2024-06-06T07:25:32.807 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-06T07:25:32.807 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --owner scheduled_teuthology@teuthology --name teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210 -- /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/orig.config.yaml 2024-06-06T07:25:32.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T07:25:32.959 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7743210' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5050 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 416/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpeEH0XzO77JVvYR8nTmyftpBbmx/TCizhL/HwhV23aRY/tlNxU1GjzfxmiUW5ayUg/sU8cXxf/tVkF8uUOyYw= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPt5jbp0lwHxS3ckq4Qm/p0zSgAQbsXnS8Cyjqg022R2qkL+BXUjg6oVlKBUQvbtoQnWLOMOb7Zpmz4Zx3OG5k= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJc4by/ciRe84OFn/gxC6DwCv4ko58qIydlLzBdXyNdQ8fCujgOlxB7HJPTUn/jiI594pLzKE7URsjN9aGnJM7c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-05_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-06T07:25:32.960 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa; will attempt to use it 2024-06-06T07:25:32.960 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks 2024-06-06T07:25:32.961 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-06T07:25:32.962 INFO:teuthology.task.internal:Checking packages... 2024-06-06T07:25:32.983 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-06T07:25:32.983 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-06T07:25:32.983 INFO:teuthology.packaging:ref: None 2024-06-06T07:25:32.983 INFO:teuthology.packaging:tag: None 2024-06-06T07:25:32.983 INFO:teuthology.packaging:branch: squid 2024-06-06T07:25:32.983 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:25:32.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-06T07:25:33.199 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2960.g3d746ca5 2024-06-06T07:25:33.201 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-06T07:25:33.207 INFO:teuthology.task.internal:no buildpackages task found 2024-06-06T07:25:33.207 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-06T07:25:33.230 INFO:teuthology.task.internal:Saving configuration 2024-06-06T07:25:33.253 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-06T07:25:33.259 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-06T07:25:33.290 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 07:20:28.599249', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpeEH0XzO77JVvYR8nTmyftpBbmx/TCizhL/HwhV23aRY/tlNxU1GjzfxmiUW5ayUg/sU8cXxf/tVkF8uUOyYw='} 2024-06-06T07:25:33.312 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 07:20:28.595800', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPt5jbp0lwHxS3ckq4Qm/p0zSgAQbsXnS8Cyjqg022R2qkL+BXUjg6oVlKBUQvbtoQnWLOMOb7Zpmz4Zx3OG5k='} 2024-06-06T07:25:33.332 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 07:20:28.597607', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJc4by/ciRe84OFn/gxC6DwCv4ko58qIydlLzBdXyNdQ8fCujgOlxB7HJPTUn/jiI594pLzKE7URsjN9aGnJM7c='} 2024-06-06T07:25:33.333 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-06T07:25:33.339 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-06T07:25:33.339 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-06T07:25:33.339 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-06T07:25:33.339 INFO:teuthology.run_tasks:Running task console_log... 2024-06-06T07:25:33.477 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fba5ff8fbe0>, signals=[15]) 2024-06-06T07:25:33.477 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-06T07:25:33.483 INFO:teuthology.task.internal:Opening connections... 2024-06-06T07:25:33.483 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:25:33.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:25:33.565 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-06-06T07:25:33.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:25:33.641 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-06-06T07:25:33.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:25:33.716 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-06T07:25:33.722 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-06-06T07:25:33.740 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-06-06T07:25:33.740 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-06-06T07:25:33.798 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-06-06T07:25:33.798 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-06-06T07:25:33.798 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T07:25:33.799 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T07:25:33.800 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-06-06T07:25:33.817 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-06-06T07:25:33.836 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-06-06T07:25:33.836 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-06-06T07:25:33.893 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T07:25:33.894 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T07:25:33.895 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-06-06T07:25:33.917 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-06-06T07:25:33.935 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-06-06T07:25:33.935 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-06-06T07:25:33.992 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-06-06T07:25:33.992 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T07:25:33.993 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T07:25:33.994 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-06-06T07:25:34.018 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-06T07:25:34.028 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-06T07:25:34.033 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-06T07:25:34.033 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-06-06T07:25:34.035 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-06-06T07:25:34.037 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-06-06T07:25:34.053 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-06T07:25:34.059 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-06T07:25:34.059 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-06-06T07:25:34.092 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-06-06T07:25:34.094 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-06-06T07:25:34.195 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-06T07:25:34.268 INFO:teuthology.run_tasks:Running task kernel... 2024-06-06T07:25:34.283 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-06T07:25:34.283 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-06T07:25:34.284 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-06T07:25:34.284 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T07:25:34.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:25:34.301 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-06-06T07:25:34.358 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-437.el9.x86_64 2024-06-06T07:25:34.358 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.14.0-437.el9.x86_64 2024-06-06T07:25:34.359 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2024-06-06T07:25:36.597 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-06-06T07:25:38.706 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-06-06T07:25:42.757 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-06-06T07:25:44.407 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 17 kB 00:00 2024-06-06T07:25:45.652 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-06-06T07:25:50.841 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-06T07:25:52.881 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T07:25:52.881 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T07:25:52.975 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-06-06T07:25:52.982 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-06-06T07:25:52.983 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-06-06T07:25:52.984 INFO:teuthology.orchestra.run.smithi087.stdout:Install 4 Packages 2024-06-06T07:25:52.984 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:25:52.984 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 80 M 2024-06-06T07:25:52.984 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 124 M 2024-06-06T07:25:52.984 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-06-06T07:25:53.214 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:25:53.216 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:25:53.222 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:25:53.409 INFO:teuthology.orchestra.run.smithi087.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.2 MB/s | 893 kB 00:00 2024-06-06T07:25:53.701 INFO:teuthology.orchestra.run.smithi087.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-06-06T07:25:54.135 INFO:teuthology.orchestra.run.smithi087.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 35 MB/s | 35 MB 00:01 2024-06-06T07:25:54.385 INFO:teuthology.orchestra.run.smithi087.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 30 MB/s | 29 MB 00:00 2024-06-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-06-06T07:25:54.387 INFO:teuthology.orchestra.run.smithi087.stdout:Total 57 MB/s | 80 MB 00:01 2024-06-06T07:25:55.085 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-06-06T07:25:55.438 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-06-06T07:25:55.438 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-06-06T07:25:56.379 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-06-06T07:25:56.380 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-06-06T07:25:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-06-06T07:25:59.515 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T07:25:59.602 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T07:26:02.258 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T07:26:02.421 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T07:26:07.147 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T07:26:07.193 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:11.899 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:41.233 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:41.273 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T07:26:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T07:26:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T07:26:42.413 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T07:26:42.413 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:26:42.414 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-06-06T07:26:42.767 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T07:26:42.858 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.859 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T07:26:42.859 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T07:26:42.859 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T07:26:42.859 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T07:26:42.859 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-06T07:26:42.859 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T07:26:42.859 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T07:26:42.911 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T07:26:42.912 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T07:26:42.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 68.el9 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-06-06T07:26:42.943 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8264254 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-06-06T07:26:42.944 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T07:26:42.946 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T07:26:42.946 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 5.14.0-452.el9.x86_64 2024-06-06T07:26:42.946 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T07:26:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-06-06T07:26:45.079 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T07:26:45.104 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-06-06T07:26:45.106 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-06-06T07:26:45.121 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.nmOEIvzvaM 2024-06-06T07:26:45.121 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nmOEIvzvaM 2024-06-06T07:26:45.187 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.nmOEIvzvaM 2024-06-06T07:26:45.319 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.nmOEIvzvaM is 6KB 2024-06-06T07:26:45.368 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.nmOEIvzvaM 2024-06-06T07:26:45.383 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T07:26:45.444 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T07:26:45.444 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T07:26:45.444 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T07:26:45.444 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T07:26:45.446 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T07:26:45.598 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-06-06T07:27:15.602 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-06T07:27:15.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:27:15.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:27:34.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-06-06T07:27:43.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:27:43.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:27:46.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-06-06T07:27:58.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:27:58.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:28:30.143 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-06-06T07:28:30.549 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:28:30.549 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-06T07:28:30.550 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-06-06T07:28:30.564 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-452.el9.x86_64 2024-06-06T07:28:30.564 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T07:28:30.565 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T07:28:30.565 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T07:28:31.566 DEBUG:teuthology.parallel:result is None 2024-06-06T07:28:31.570 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-06T07:28:31.579 INFO:teuthology.task.internal:Creating test directory... 2024-06-06T07:28:31.579 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T07:28:31.582 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T07:28:31.587 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T07:28:31.605 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-06T07:28:31.617 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-06T07:28:31.627 INFO:teuthology.task.internal:Creating archive directory... 2024-06-06T07:28:31.627 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T07:28:31.635 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T07:28:31.646 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T07:28:31.763 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-06T07:28:31.774 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-06T07:28:31.775 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T07:28:31.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:28:31.791 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T07:28:31.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:28:31.808 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T07:28:31.810 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T07:28:31.834 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T07:28:31.871 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.882 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.899 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.907 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.913 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.944 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T07:28:31.946 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-06T07:28:31.952 INFO:teuthology.task.internal:Configuring sudo... 2024-06-06T07:28:31.953 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T07:28:31.955 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T07:28:31.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T07:28:31.985 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-06T07:28:31.995 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-06T07:28:31.995 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T07:28:32.019 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T07:28:32.024 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T07:28:32.042 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.113 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.166 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.244 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:28:32.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T07:28:32.305 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.398 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.480 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.522 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:28:32.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T07:28:32.591 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.640 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T07:28:32.693 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.783 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T07:28:32.822 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:28:32.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T07:28:32.889 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-06-06T07:28:32.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-06-06T07:28:32.896 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-06-06T07:28:32.931 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T07:28:32.933 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T07:28:32.967 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T07:28:33.348 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-06T07:28:33.356 INFO:teuthology.task.internal:Starting timer... 2024-06-06T07:28:33.357 INFO:teuthology.run_tasks:Running task pcp... 2024-06-06T07:28:33.379 INFO:teuthology.run_tasks:Running task selinux... 2024-06-06T07:28:33.388 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-06T07:28:33.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-06-06T07:28:33.478 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-06-06T07:28:33.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-06-06T07:28:33.549 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-06-06T07:28:33.551 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-06-06T07:28:33.627 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-06-06T07:28:33.629 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-06T07:28:33.629 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-06-06T07:28:33.657 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-06-06T07:28:33.657 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-06-06T07:28:33.683 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-06-06T07:28:33.683 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-06-06T07:28:33.703 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-06-06T07:28:33.703 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-06-06T07:28:33.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T07:28:33.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:28:33.728 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T07:28:33.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:28:33.758 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T07:28:33.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T07:28:33.786 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-06T07:28:33.786 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-06-06T07:28:33.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-06-06T07:28:33.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-06-06T07:28:33.905 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-06T07:28:33.918 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-06T07:28:33.933 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-06T07:28:33.934 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi181.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-06T07:33:04.925 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-06-06T07:33:04.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:33:04.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:33:05.007 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-06-06T07:33:05.094 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-06-06T07:33:05.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-06-06T07:33:05.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:33:05.171 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-06-06T07:33:05.258 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-06-06T07:33:05.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-06T07:33:05.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T07:33:05.334 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-06-06T07:33:05.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-06T07:33:05.416 INFO:teuthology.run_tasks:Running task clock... 2024-06-06T07:33:05.431 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-06T07:33:05.432 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T07:33:05.432 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T07:33:05.435 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T07:33:05.435 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T07:33:05.438 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T07:33:05.438 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T07:33:05.465 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T07:33:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T07:33:05.479 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T07:33:05.481 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T07:33:05.486 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T07:33:05.500 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T07:33:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-06-06T07:33:05.506 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-06-06T07:33:05.513 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-06-06T07:33:05.518 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-06-06T07:33:05.523 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-06-06T07:33:05.526 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T07:33:05.532 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T07:33:05.534 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-06-06T07:33:05.539 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T07:33:05.546 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T07:33:05.547 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T07:33:05.560 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T07:33:05.592 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-06-06T07:33:05.596 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T07:33:05.596 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-06-06T07:33:05.597 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.597 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.597 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.597 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.600 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.603 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.618 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.622 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T07:33:05.623 INFO:teuthology.run_tasks:Running task install... 2024-06-06T07:33:05.632 DEBUG:teuthology.task.install:project ceph 2024-06-06T07:33:05.632 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-06T07:33:05.632 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-06T07:33:05.632 INFO:teuthology.task.install:Using flavor: default 2024-06-06T07:33:05.638 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-06T07:33:05.638 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-06T07:33:05.639 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T07:33:05.639 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:33:05.642 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T07:33:05.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:33:05.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T07:33:05.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:33:05.789 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T07:33:05.789 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T07:33:05.798 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T07:33:05.798 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T07:33:05.801 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T07:33:05.801 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T07:33:05.903 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T07:33:05.903 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:33:05.904 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T07:33:05.910 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T07:33:05.910 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:33:05.910 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T07:33:05.915 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T07:33:05.915 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:33:05.915 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T07:33:05.929 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T07:33:05.929 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T07:33:05.934 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T07:33:05.935 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T07:33:05.939 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T07:33:05.939 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T07:33:05.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T07:33:05.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T07:33:06.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T07:33:06.069 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T07:33:06.072 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T07:33:06.076 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T07:33:06.133 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-06-06T07:33:06.134 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-06-06T07:33:06.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-06-06T07:33:06.136 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-06-06T07:33:06.139 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-06-06T07:33:06.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-06-06T07:33:06.360 INFO:teuthology.orchestra.run.smithi195.stdout:45 files removed 2024-06-06T07:33:06.363 INFO:teuthology.orchestra.run.smithi181.stdout:45 files removed 2024-06-06T07:33:06.378 INFO:teuthology.orchestra.run.smithi087.stdout:45 files removed 2024-06-06T07:33:06.387 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T07:33:06.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T07:33:06.407 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T07:33:07.015 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 226 kB/s | 83 kB 00:00 2024-06-06T07:33:07.061 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-06-06T07:33:07.085 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 207 kB/s | 83 kB 00:00 2024-06-06T07:33:07.390 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-06-06T07:33:07.419 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-06-06T07:33:07.453 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-06-06T07:33:07.770 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-06-06T07:33:07.799 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-06-06T07:33:07.812 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-06-06T07:33:08.464 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-06-06T07:33:08.564 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-06T07:33:08.668 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.1 MB 00:00 2024-06-06T07:33:10.458 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-06-06T07:33:10.524 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-06-06T07:33:10.877 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-06-06T07:33:14.808 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-06-06T07:33:14.827 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-06-06T07:33:15.261 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-06T07:33:16.294 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 17 kB 00:00 2024-06-06T07:33:16.350 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 17 kB 00:00 2024-06-06T07:33:16.727 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-06-06T07:33:17.856 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-06T07:33:17.920 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-06T07:33:18.140 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-06T07:33:22.962 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-06-06T07:33:23.059 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-06-06T07:33:23.370 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-06-06T07:33:25.326 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.327 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.337 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.338 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.339 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T07:33:25.340 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.341 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T07:33:25.343 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.344 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.388 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.389 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.400 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.401 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.401 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T07:33:25.402 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.404 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T07:33:25.406 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.407 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.407 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T07:33:25.421 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T07:33:25.422 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T07:33:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T07:33:25.424 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T07:33:25.425 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T07:33:25.426 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T07:33:25.427 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T07:33:25.428 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T07:33:25.429 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T07:33:25.430 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T07:33:25.431 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T07:33:25.432 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout:Install 148 Packages 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 11 Packages 2024-06-06T07:33:25.433 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:33:25.434 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 225 M 2024-06-06T07:33:25.434 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-06-06T07:33:25.469 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T07:33:25.480 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T07:33:25.481 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T07:33:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T07:33:25.483 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T07:33:25.484 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T07:33:25.485 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T07:33:25.486 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T07:33:25.487 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T07:33:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T07:33:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T07:33:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T07:33:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:Install 148 Packages 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 11 Packages 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 225 M 2024-06-06T07:33:25.492 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-06-06T07:33:25.688 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.689 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.699 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.700 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T07:33:25.701 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T07:33:25.702 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.703 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T07:33:25.706 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T07:33:25.707 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T07:33:25.769 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-06-06T07:33:25.785 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-06-06T07:33:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-06-06T07:33:25.785 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T07:33:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T07:33:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T07:33:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T07:33:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T07:33:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T07:33:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T07:33:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T07:33:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T07:33:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T07:33:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T07:33:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Install 148 Packages 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 11 Packages 2024-06-06T07:33:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:33:25.798 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 225 M 2024-06-06T07:33:25.798 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-06-06T07:33:26.306 INFO:teuthology.orchestra.run.smithi087.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-06-06T07:33:26.335 INFO:teuthology.orchestra.run.smithi181.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-06-06T07:33:26.596 INFO:teuthology.orchestra.run.smithi195.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-06-06T07:33:26.663 INFO:teuthology.orchestra.run.smithi087.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.4 MB/s | 890 kB 00:00 2024-06-06T07:33:26.730 INFO:teuthology.orchestra.run.smithi087.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.1 MB/s | 145 kB 00:00 2024-06-06T07:33:26.918 INFO:teuthology.orchestra.run.smithi181.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 890 kB 00:00 2024-06-06T07:33:26.963 INFO:teuthology.orchestra.run.smithi195.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.4 MB/s | 890 kB 00:00 2024-06-06T07:33:27.035 INFO:teuthology.orchestra.run.smithi181.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.2 MB/s | 145 kB 00:00 2024-06-06T07:33:27.047 INFO:teuthology.orchestra.run.smithi195.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.7 MB/s | 145 kB 00:00 2024-06-06T07:33:27.272 INFO:teuthology.orchestra.run.smithi087.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 4.4 MB/s | 2.4 MB 00:00 2024-06-06T07:33:27.664 INFO:teuthology.orchestra.run.smithi087.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-06-06T07:33:28.239 INFO:teuthology.orchestra.run.smithi195.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-06-06T07:33:28.306 INFO:teuthology.orchestra.run.smithi195.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-06-06T07:33:28.410 INFO:teuthology.orchestra.run.smithi181.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-06-06T07:33:28.485 INFO:teuthology.orchestra.run.smithi181.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-06-06T07:33:28.872 INFO:teuthology.orchestra.run.smithi087.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-06-06T07:33:28.897 INFO:teuthology.orchestra.run.smithi195.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-06-06T07:33:28.939 INFO:teuthology.orchestra.run.smithi087.stdout:(7/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-06-06T07:33:29.269 INFO:teuthology.orchestra.run.smithi181.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-06-06T07:33:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-06-06T07:33:29.823 INFO:teuthology.orchestra.run.smithi195.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.1 MB/s | 4.7 MB 00:01 2024-06-06T07:33:30.011 INFO:teuthology.orchestra.run.smithi181.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 5.4 MB/s | 21 MB 00:03 2024-06-06T07:33:30.103 INFO:teuthology.orchestra.run.smithi181.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 275 kB/s | 25 kB 00:00 2024-06-06T07:33:32.279 INFO:teuthology.orchestra.run.smithi181.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.0 MB/s | 10 MB 00:02 2024-06-06T07:33:32.387 INFO:teuthology.orchestra.run.smithi181.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.0 MB/s | 221 kB 00:00 2024-06-06T07:33:32.495 INFO:teuthology.orchestra.run.smithi181.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 250 kB/s | 27 kB 00:00 2024-06-06T07:33:32.596 INFO:teuthology.orchestra.run.smithi181.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 317 kB/s | 32 kB 00:00 2024-06-06T07:33:32.748 INFO:teuthology.orchestra.run.smithi195.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 10 MB 00:02 2024-06-06T07:33:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.2 MB/s | 747 kB 00:00 2024-06-06T07:33:32.806 INFO:teuthology.orchestra.run.smithi195.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 432 kB/s | 25 kB 00:00 2024-06-06T07:33:32.875 INFO:teuthology.orchestra.run.smithi181.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-06T07:33:32.923 INFO:teuthology.orchestra.run.smithi087.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.1 MB/s | 21 MB 00:06 2024-06-06T07:33:32.963 INFO:teuthology.orchestra.run.smithi181.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.3 MB/s | 127 kB 00:00 2024-06-06T07:33:33.098 INFO:teuthology.orchestra.run.smithi087.stdout:(9/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 4.0 MB/s | 17 MB 00:04 2024-06-06T07:33:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 131 kB/s | 25 kB 00:00 2024-06-06T07:33:33.122 INFO:teuthology.orchestra.run.smithi181.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 3.1 MB/s | 501 kB 00:00 2024-06-06T07:33:33.282 INFO:teuthology.orchestra.run.smithi087.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.3 MB/s | 221 kB 00:00 2024-06-06T07:33:33.331 INFO:teuthology.orchestra.run.smithi195.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 3.0 MB/s | 21 MB 00:06 2024-06-06T07:33:33.373 INFO:teuthology.orchestra.run.smithi087.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 295 kB/s | 27 kB 00:00 2024-06-06T07:33:33.482 INFO:teuthology.orchestra.run.smithi087.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 292 kB/s | 32 kB 00:00 2024-06-06T07:33:33.490 INFO:teuthology.orchestra.run.smithi195.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-06T07:33:33.590 INFO:teuthology.orchestra.run.smithi195.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 270 kB/s | 27 kB 00:00 2024-06-06T07:33:33.682 INFO:teuthology.orchestra.run.smithi195.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 345 kB/s | 32 kB 00:00 2024-06-06T07:33:33.766 INFO:teuthology.orchestra.run.smithi087.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.6 MB/s | 747 kB 00:00 2024-06-06T07:33:33.883 INFO:teuthology.orchestra.run.smithi087.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.4 MB/s | 163 kB 00:00 2024-06-06T07:33:33.991 INFO:teuthology.orchestra.run.smithi087.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.1 MB/s | 127 kB 00:00 2024-06-06T07:33:33.999 INFO:teuthology.orchestra.run.smithi195.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.3 MB/s | 747 kB 00:00 2024-06-06T07:33:34.100 INFO:teuthology.orchestra.run.smithi195.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-06T07:33:34.192 INFO:teuthology.orchestra.run.smithi195.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.3 MB/s | 127 kB 00:00 2024-06-06T07:33:34.192 INFO:teuthology.orchestra.run.smithi087.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.4 MB/s | 501 kB 00:00 2024-06-06T07:33:34.409 INFO:teuthology.orchestra.run.smithi195.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.3 MB/s | 501 kB 00:00 2024-06-06T07:33:34.514 INFO:teuthology.orchestra.run.smithi181.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-06-06T07:33:34.576 INFO:teuthology.orchestra.run.smithi195.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.0 MB/s | 17 MB 00:05 2024-06-06T07:33:34.605 INFO:teuthology.orchestra.run.smithi181.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 492 kB/s | 45 kB 00:00 2024-06-06T07:33:34.625 INFO:teuthology.orchestra.run.smithi087.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 1.8 MB/s | 10 MB 00:05 2024-06-06T07:33:34.634 INFO:teuthology.orchestra.run.smithi195.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 778 kB/s | 45 kB 00:00 2024-06-06T07:33:34.689 INFO:teuthology.orchestra.run.smithi181.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-06T07:33:34.726 INFO:teuthology.orchestra.run.smithi195.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.4 MB/s | 135 kB 00:00 2024-06-06T07:33:34.734 INFO:teuthology.orchestra.run.smithi087.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 417 kB/s | 45 kB 00:00 2024-06-06T07:33:34.790 INFO:teuthology.orchestra.run.smithi181.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-06T07:33:34.826 INFO:teuthology.orchestra.run.smithi195.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-06T07:33:34.842 INFO:teuthology.orchestra.run.smithi087.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.2 MB/s | 135 kB 00:00 2024-06-06T07:33:34.960 INFO:teuthology.orchestra.run.smithi087.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.4 MB/s | 162 kB 00:00 2024-06-06T07:33:34.985 INFO:teuthology.orchestra.run.smithi195.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.0 MB/s | 321 kB 00:00 2024-06-06T07:33:34.998 INFO:teuthology.orchestra.run.smithi181.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 1.5 MB/s | 321 kB 00:00 2024-06-06T07:33:35.126 INFO:teuthology.orchestra.run.smithi181.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.3 MB/s | 300 kB 00:00 2024-06-06T07:33:35.144 INFO:teuthology.orchestra.run.smithi195.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 300 kB 00:00 2024-06-06T07:33:35.151 INFO:teuthology.orchestra.run.smithi087.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 1.6 MB/s | 321 kB 00:00 2024-06-06T07:33:35.219 INFO:teuthology.orchestra.run.smithi195.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-06T07:33:35.223 INFO:teuthology.orchestra.run.smithi181.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 992 kB/s | 99 kB 00:00 2024-06-06T07:33:35.294 INFO:teuthology.orchestra.run.smithi195.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-06T07:33:35.324 INFO:teuthology.orchestra.run.smithi181.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 851 kB/s | 85 kB 00:00 2024-06-06T07:33:35.343 INFO:teuthology.orchestra.run.smithi087.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 300 kB 00:00 2024-06-06T07:33:35.452 INFO:teuthology.orchestra.run.smithi087.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 918 kB/s | 99 kB 00:00 2024-06-06T07:33:35.619 INFO:teuthology.orchestra.run.smithi087.stdout:(25/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.7 MB/s | 5.3 MB 00:01 2024-06-06T07:33:35.644 INFO:teuthology.orchestra.run.smithi087.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 444 kB/s | 85 kB 00:00 2024-06-06T07:33:36.116 INFO:teuthology.orchestra.run.smithi181.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.9 MB/s | 3.1 MB 00:00 2024-06-06T07:33:36.153 INFO:teuthology.orchestra.run.smithi087.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 336 kB/s | 171 kB 00:00 2024-06-06T07:33:36.225 INFO:teuthology.orchestra.run.smithi181.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 171 kB 00:00 2024-06-06T07:33:36.253 INFO:teuthology.orchestra.run.smithi087.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 318 kB/s | 32 kB 00:00 2024-06-06T07:33:36.303 INFO:teuthology.orchestra.run.smithi195.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.1 MB/s | 3.1 MB 00:01 2024-06-06T07:33:36.308 INFO:teuthology.orchestra.run.smithi181.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 381 kB/s | 32 kB 00:00 2024-06-06T07:33:36.370 INFO:teuthology.orchestra.run.smithi195.stdout:(27/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 2.7 MB/s | 5.3 MB 00:01 2024-06-06T07:33:36.392 INFO:teuthology.orchestra.run.smithi181.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-06T07:33:36.395 INFO:teuthology.orchestra.run.smithi195.stdout:(28/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-06-06T07:33:36.428 INFO:teuthology.orchestra.run.smithi087.stdout:(29/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.8 MB/s | 3.1 MB 00:00 2024-06-06T07:33:36.429 INFO:teuthology.orchestra.run.smithi195.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 541 kB/s | 32 kB 00:00 2024-06-06T07:33:36.454 INFO:teuthology.orchestra.run.smithi087.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 710 kB/s | 142 kB 00:00 2024-06-06T07:33:36.462 INFO:teuthology.orchestra.run.smithi195.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-06T07:33:37.367 INFO:teuthology.orchestra.run.smithi181.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.6 MB/s | 3.5 MB 00:00 2024-06-06T07:33:37.421 INFO:teuthology.orchestra.run.smithi087.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.5 MB/s | 3.5 MB 00:00 2024-06-06T07:33:37.529 INFO:teuthology.orchestra.run.smithi087.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.3 MB/s | 249 kB 00:00 2024-06-06T07:33:37.592 INFO:teuthology.orchestra.run.smithi181.stdout:(31/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.0 MB/s | 17 MB 00:08 2024-06-06T07:33:37.638 INFO:teuthology.orchestra.run.smithi087.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 455 kB/s | 49 kB 00:00 2024-06-06T07:33:37.738 INFO:teuthology.orchestra.run.smithi087.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 164 kB/s | 16 kB 00:00 2024-06-06T07:33:37.776 INFO:teuthology.orchestra.run.smithi181.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.3 MB/s | 249 kB 00:00 2024-06-06T07:33:37.855 INFO:teuthology.orchestra.run.smithi087.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.2 MB/s | 264 kB 00:00 2024-06-06T07:33:37.884 INFO:teuthology.orchestra.run.smithi181.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 456 kB/s | 49 kB 00:00 2024-06-06T07:33:37.993 INFO:teuthology.orchestra.run.smithi181.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 152 kB/s | 16 kB 00:00 2024-06-06T07:33:38.130 INFO:teuthology.orchestra.run.smithi087.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.7 MB/s | 763 kB 00:00 2024-06-06T07:33:38.193 INFO:teuthology.orchestra.run.smithi181.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 264 kB 00:00 2024-06-06T07:33:38.230 INFO:teuthology.orchestra.run.smithi087.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 182 kB/s | 18 kB 00:00 2024-06-06T07:33:38.238 INFO:teuthology.orchestra.run.smithi195.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 1.9 MB/s | 3.5 MB 00:01 2024-06-06T07:33:38.269 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.348 INFO:teuthology.orchestra.run.smithi087.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-06-06T07:33:38.348 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.413 INFO:teuthology.orchestra.run.smithi195.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.4 MB/s | 249 kB 00:00 2024-06-06T07:33:38.439 INFO:teuthology.orchestra.run.smithi087.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-06-06T07:33:38.442 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-06T07:33:38.488 INFO:teuthology.orchestra.run.smithi195.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 657 kB/s | 49 kB 00:00 2024-06-06T07:33:38.523 INFO:teuthology.orchestra.run.smithi087.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-06-06T07:33:38.580 INFO:teuthology.orchestra.run.smithi195.stdout:(34/159): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-06-06T07:33:38.582 INFO:teuthology.orchestra.run.smithi087.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-06-06T07:33:38.605 INFO:teuthology.orchestra.run.smithi195.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 141 kB/s | 16 kB 00:00 2024-06-06T07:33:38.624 INFO:teuthology.orchestra.run.smithi087.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T07:33:38.664 INFO:teuthology.orchestra.run.smithi195.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 264 kB 00:00 2024-06-06T07:33:38.666 INFO:teuthology.orchestra.run.smithi087.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.1 MB/s | 135 kB 00:00 2024-06-06T07:33:38.706 INFO:teuthology.orchestra.run.smithi195.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 438 kB/s | 18 kB 00:00 2024-06-06T07:33:38.708 INFO:teuthology.orchestra.run.smithi087.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-06-06T07:33:38.744 INFO:teuthology.orchestra.run.smithi181.stdout:(36/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 5.6 MB/s | 48 MB 00:08 2024-06-06T07:33:38.745 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.750 INFO:teuthology.orchestra.run.smithi087.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-06-06T07:33:38.751 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.785 INFO:teuthology.orchestra.run.smithi181.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.3 MB/s | 763 kB 00:00 2024-06-06T07:33:38.811 INFO:teuthology.orchestra.run.smithi181.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 272 kB/s | 18 kB 00:00 2024-06-06T07:33:38.823 INFO:teuthology.orchestra.run.smithi195.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 690 kB/s | 80 kB 00:00 2024-06-06T07:33:38.824 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:(39/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.9 MB/s | 763 kB 00:00 2024-06-06T07:33:38.866 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.869 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.876 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.932 INFO:teuthology.orchestra.run.smithi195.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-06-06T07:33:38.934 INFO:teuthology.orchestra.run.smithi087.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-06-06T07:33:38.937 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.944 INFO:teuthology.orchestra.run.smithi181.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-06-06T07:33:38.948 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:38.974 INFO:teuthology.orchestra.run.smithi195.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-06-06T07:33:38.999 INFO:teuthology.orchestra.run.smithi195.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-06-06T07:33:39.003 INFO:teuthology.orchestra.run.smithi181.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-06-06T07:33:39.009 INFO:teuthology.orchestra.run.smithi087.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-06-06T07:33:39.012 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.037 INFO:teuthology.orchestra.run.smithi181.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-06-06T07:33:39.041 INFO:teuthology.orchestra.run.smithi195.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T07:33:39.051 INFO:teuthology.orchestra.run.smithi087.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-06T07:33:39.083 INFO:teuthology.orchestra.run.smithi195.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-06-06T07:33:39.093 INFO:teuthology.orchestra.run.smithi087.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-06-06T07:33:39.109 INFO:teuthology.orchestra.run.smithi195.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-06-06T07:33:39.129 INFO:teuthology.orchestra.run.smithi181.stdout:(42/159): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-06-06T07:33:39.134 INFO:teuthology.orchestra.run.smithi195.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T07:33:39.135 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.144 INFO:teuthology.orchestra.run.smithi087.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T07:33:39.154 INFO:teuthology.orchestra.run.smithi181.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-06-06T07:33:39.159 INFO:teuthology.orchestra.run.smithi195.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T07:33:39.160 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.185 INFO:teuthology.orchestra.run.smithi087.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-06-06T07:33:39.188 INFO:teuthology.orchestra.run.smithi181.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-06-06T07:33:39.213 INFO:teuthology.orchestra.run.smithi181.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-06T07:33:39.238 INFO:teuthology.orchestra.run.smithi181.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T07:33:39.263 INFO:teuthology.orchestra.run.smithi181.stdout:(47/159): python3-ply-3.11-14.el9.noarch.rpm 793 kB/s | 106 kB 00:00 2024-06-06T07:33:39.264 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.297 INFO:teuthology.orchestra.run.smithi181.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-06-06T07:33:39.298 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.300 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.303 INFO:teuthology.orchestra.run.smithi087.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-06-06T07:33:39.326 INFO:teuthology.orchestra.run.smithi195.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-06-06T07:33:39.336 INFO:teuthology.orchestra.run.smithi087.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 551 kB/s | 18 kB 00:00 2024-06-06T07:33:39.339 INFO:teuthology.orchestra.run.smithi181.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-06-06T07:33:39.360 INFO:teuthology.orchestra.run.smithi195.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-06-06T07:33:39.360 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.363 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:39.364 INFO:teuthology.orchestra.run.smithi181.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-06-06T07:33:39.370 INFO:teuthology.orchestra.run.smithi087.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-06-06T07:33:39.390 INFO:teuthology.orchestra.run.smithi181.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-06-06T07:33:39.402 INFO:teuthology.orchestra.run.smithi195.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-06-06T07:33:39.412 INFO:teuthology.orchestra.run.smithi087.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-06T07:33:39.427 INFO:teuthology.orchestra.run.smithi195.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-06-06T07:33:39.432 INFO:teuthology.orchestra.run.smithi181.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-06-06T07:33:39.452 INFO:teuthology.orchestra.run.smithi195.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T07:33:39.454 INFO:teuthology.orchestra.run.smithi087.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-06T07:33:39.457 INFO:teuthology.orchestra.run.smithi181.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-06-06T07:33:39.478 INFO:teuthology.orchestra.run.smithi195.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-06T07:33:39.482 INFO:teuthology.orchestra.run.smithi181.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-06-06T07:33:39.504 INFO:teuthology.orchestra.run.smithi087.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-06T07:33:39.507 INFO:teuthology.orchestra.run.smithi181.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-06-06T07:33:39.519 INFO:teuthology.orchestra.run.smithi195.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-06-06T07:33:39.533 INFO:teuthology.orchestra.run.smithi181.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-06-06T07:33:39.558 INFO:teuthology.orchestra.run.smithi181.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T07:33:39.578 INFO:teuthology.orchestra.run.smithi195.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-06-06T07:33:39.583 INFO:teuthology.orchestra.run.smithi181.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-06-06T07:33:39.588 INFO:teuthology.orchestra.run.smithi087.stdout:(59/159): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-06-06T07:33:39.603 INFO:teuthology.orchestra.run.smithi195.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-06T07:33:39.617 INFO:teuthology.orchestra.run.smithi181.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-06T07:33:39.629 INFO:teuthology.orchestra.run.smithi195.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T07:33:39.630 INFO:teuthology.orchestra.run.smithi087.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-06-06T07:33:39.654 INFO:teuthology.orchestra.run.smithi195.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-06-06T07:33:39.659 INFO:teuthology.orchestra.run.smithi181.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-06-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi087.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-06-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi181.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-06-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi195.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi181.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-06-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi087.stdout:(62/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-06-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi195.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi181.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi087.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-06-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi195.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-06-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi181.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi195.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi181.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 35 kB 00:00 2024-06-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi087.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 845 kB/s | 35 kB 00:00 2024-06-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi195.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi087.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-06-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi195.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi181.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-06-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi195.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-06-06T07:33:39.907 INFO:teuthology.orchestra.run.smithi087.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-06-06T07:33:39.936 INFO:teuthology.orchestra.run.smithi181.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-06-06T07:33:39.965 INFO:teuthology.orchestra.run.smithi195.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-06-06T07:33:39.970 INFO:teuthology.orchestra.run.smithi181.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-06T07:33:39.974 INFO:teuthology.orchestra.run.smithi087.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-06-06T07:33:40.023 INFO:teuthology.orchestra.run.smithi195.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-06-06T07:33:40.028 INFO:teuthology.orchestra.run.smithi181.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-06T07:33:40.049 INFO:teuthology.orchestra.run.smithi087.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-06T07:33:40.053 INFO:teuthology.orchestra.run.smithi181.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-06-06T07:33:40.074 INFO:teuthology.orchestra.run.smithi195.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-06T07:33:40.087 INFO:teuthology.orchestra.run.smithi181.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-06-06T07:33:40.091 INFO:teuthology.orchestra.run.smithi087.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-06-06T07:33:40.107 INFO:teuthology.orchestra.run.smithi195.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-06-06T07:33:40.108 INFO:teuthology.orchestra.run.smithi087.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T07:33:40.129 INFO:teuthology.orchestra.run.smithi181.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-06-06T07:33:40.133 INFO:teuthology.orchestra.run.smithi195.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T07:33:40.154 INFO:teuthology.orchestra.run.smithi181.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-06-06T07:33:40.175 INFO:teuthology.orchestra.run.smithi087.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-06-06T07:33:40.196 INFO:teuthology.orchestra.run.smithi181.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-06T07:33:40.234 INFO:teuthology.orchestra.run.smithi087.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-06T07:33:40.280 INFO:teuthology.orchestra.run.smithi181.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-06-06T07:33:40.355 INFO:teuthology.orchestra.run.smithi181.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-06-06T07:33:40.380 INFO:teuthology.orchestra.run.smithi181.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-06-06T07:33:40.391 INFO:teuthology.orchestra.run.smithi195.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-06-06T07:33:40.406 INFO:teuthology.orchestra.run.smithi181.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T07:33:40.431 INFO:teuthology.orchestra.run.smithi181.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-06-06T07:33:40.456 INFO:teuthology.orchestra.run.smithi181.stdout:(80/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-06-06T07:33:40.467 INFO:teuthology.orchestra.run.smithi195.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-06-06T07:33:40.482 INFO:teuthology.orchestra.run.smithi181.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-06-06T07:33:40.507 INFO:teuthology.orchestra.run.smithi181.stdout:(82/159): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-06-06T07:33:40.540 INFO:teuthology.orchestra.run.smithi181.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-06-06T07:33:40.566 INFO:teuthology.orchestra.run.smithi181.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-06-06T07:33:40.591 INFO:teuthology.orchestra.run.smithi181.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T07:33:40.592 INFO:teuthology.orchestra.run.smithi195.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-06-06T07:33:40.592 INFO:teuthology.orchestra.run.smithi087.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-06-06T07:33:40.616 INFO:teuthology.orchestra.run.smithi181.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-06-06T07:33:40.634 INFO:teuthology.orchestra.run.smithi087.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-06T07:33:40.641 INFO:teuthology.orchestra.run.smithi181.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T07:33:40.642 INFO:teuthology.orchestra.run.smithi195.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-06-06T07:33:40.667 INFO:teuthology.orchestra.run.smithi181.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T07:33:40.685 INFO:teuthology.orchestra.run.smithi087.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-06T07:33:40.692 INFO:teuthology.orchestra.run.smithi181.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T07:33:40.717 INFO:teuthology.orchestra.run.smithi181.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T07:33:40.743 INFO:teuthology.orchestra.run.smithi181.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T07:33:40.785 INFO:teuthology.orchestra.run.smithi181.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-06T07:33:40.792 INFO:teuthology.orchestra.run.smithi195.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-06-06T07:33:40.852 INFO:teuthology.orchestra.run.smithi181.stdout:(93/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-06-06T07:33:40.868 INFO:teuthology.orchestra.run.smithi195.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-06-06T07:33:40.877 INFO:teuthology.orchestra.run.smithi181.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-06-06T07:33:40.893 INFO:teuthology.orchestra.run.smithi195.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-06-06T07:33:40.902 INFO:teuthology.orchestra.run.smithi181.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T07:33:40.918 INFO:teuthology.orchestra.run.smithi195.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-06-06T07:33:40.928 INFO:teuthology.orchestra.run.smithi181.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 711 kB/s | 36 kB 00:00 2024-06-06T07:33:40.944 INFO:teuthology.orchestra.run.smithi195.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-06-06T07:33:40.953 INFO:teuthology.orchestra.run.smithi181.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-06-06T07:33:40.977 INFO:teuthology.orchestra.run.smithi195.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-06T07:33:40.978 INFO:teuthology.orchestra.run.smithi181.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T07:33:41.002 INFO:teuthology.orchestra.run.smithi195.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-06-06T07:33:41.004 INFO:teuthology.orchestra.run.smithi181.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T07:33:41.029 INFO:teuthology.orchestra.run.smithi181.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T07:33:41.054 INFO:teuthology.orchestra.run.smithi181.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T07:33:41.061 INFO:teuthology.orchestra.run.smithi195.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-06-06T07:33:41.079 INFO:teuthology.orchestra.run.smithi181.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-06-06T07:33:41.080 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:41.081 INFO:teuthology.orchestra.run.smithi181.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:41.121 INFO:teuthology.orchestra.run.smithi181.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-06-06T07:33:41.127 INFO:teuthology.orchestra.run.smithi087.stdout:(76/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.0 MB/s | 48 MB 00:08 2024-06-06T07:33:41.128 INFO:teuthology.orchestra.run.smithi195.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-06-06T07:33:41.152 INFO:teuthology.orchestra.run.smithi087.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 531 kB/s | 248 kB 00:00 2024-06-06T07:33:41.153 INFO:teuthology.orchestra.run.smithi195.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-06-06T07:33:41.163 INFO:teuthology.orchestra.run.smithi181.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-06-06T07:33:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T07:33:41.197 INFO:teuthology.orchestra.run.smithi181.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-06-06T07:33:41.204 INFO:teuthology.orchestra.run.smithi195.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-06-06T07:33:41.222 INFO:teuthology.orchestra.run.smithi181.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-06-06T07:33:41.227 INFO:teuthology.orchestra.run.smithi087.stdout:(78/159): python3-babel-2.9.1-2.el9.noarch.rpm 6.0 MB/s | 6.0 MB 00:00 2024-06-06T07:33:41.229 INFO:teuthology.orchestra.run.smithi195.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T07:33:41.247 INFO:teuthology.orchestra.run.smithi181.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-06-06T07:33:41.252 INFO:teuthology.orchestra.run.smithi087.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-06-06T07:33:41.255 INFO:teuthology.orchestra.run.smithi195.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T07:33:41.273 INFO:teuthology.orchestra.run.smithi181.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-06T07:33:41.277 INFO:teuthology.orchestra.run.smithi087.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-06-06T07:33:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T07:33:41.290 INFO:teuthology.orchestra.run.smithi181.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-06-06T07:33:41.303 INFO:teuthology.orchestra.run.smithi087.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-06-06T07:33:41.305 INFO:teuthology.orchestra.run.smithi195.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T07:33:41.324 INFO:teuthology.orchestra.run.smithi181.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-06-06T07:33:41.328 INFO:teuthology.orchestra.run.smithi087.stdout:(82/159): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-06-06T07:33:41.331 INFO:teuthology.orchestra.run.smithi195.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T07:33:41.353 INFO:teuthology.orchestra.run.smithi087.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-06-06T07:33:41.357 INFO:teuthology.orchestra.run.smithi181.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-06-06T07:33:41.379 INFO:teuthology.orchestra.run.smithi087.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T07:33:41.404 INFO:teuthology.orchestra.run.smithi087.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T07:33:41.429 INFO:teuthology.orchestra.run.smithi087.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-06-06T07:33:41.454 INFO:teuthology.orchestra.run.smithi087.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T07:33:41.479 INFO:teuthology.orchestra.run.smithi087.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T07:33:41.505 INFO:teuthology.orchestra.run.smithi087.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T07:33:41.522 INFO:teuthology.orchestra.run.smithi087.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-06-06T07:33:41.532 INFO:teuthology.orchestra.run.smithi181.stdout:(112/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-06T07:33:41.539 INFO:teuthology.orchestra.run.smithi087.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-06-06T07:33:41.591 INFO:teuthology.orchestra.run.smithi181.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-06-06T07:33:41.614 INFO:teuthology.orchestra.run.smithi087.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-06-06T07:33:41.616 INFO:teuthology.orchestra.run.smithi181.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-06-06T07:33:41.639 INFO:teuthology.orchestra.run.smithi087.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-06-06T07:33:41.641 INFO:teuthology.orchestra.run.smithi181.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-06T07:33:41.665 INFO:teuthology.orchestra.run.smithi087.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-06-06T07:33:41.667 INFO:teuthology.orchestra.run.smithi181.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 597 kB/s | 15 kB 00:00 2024-06-06T07:33:41.698 INFO:teuthology.orchestra.run.smithi087.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-06-06T07:33:41.700 INFO:teuthology.orchestra.run.smithi181.stdout:(117/159): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-06-06T07:33:41.724 INFO:teuthology.orchestra.run.smithi087.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-06-06T07:33:41.734 INFO:teuthology.orchestra.run.smithi181.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-06-06T07:33:41.749 INFO:teuthology.orchestra.run.smithi087.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-06-06T07:33:41.774 INFO:teuthology.orchestra.run.smithi087.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T07:33:41.800 INFO:teuthology.orchestra.run.smithi087.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T07:33:41.801 INFO:teuthology.orchestra.run.smithi181.stdout:(119/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-06-06T07:33:41.822 INFO:teuthology.orchestra.run.smithi195.stdout:(92/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 5.4 MB/s | 48 MB 00:09 2024-06-06T07:33:41.825 INFO:teuthology.orchestra.run.smithi087.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T07:33:41.826 INFO:teuthology.orchestra.run.smithi181.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-06-06T07:33:41.851 INFO:teuthology.orchestra.run.smithi087.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T07:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T07:33:41.873 INFO:teuthology.orchestra.run.smithi195.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 42 kB 00:00 2024-06-06T07:33:41.876 INFO:teuthology.orchestra.run.smithi087.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-06-06T07:33:41.876 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:41.877 INFO:teuthology.orchestra.run.smithi181.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-06-06T07:33:41.877 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:41.902 INFO:teuthology.orchestra.run.smithi181.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-06-06T07:33:41.918 INFO:teuthology.orchestra.run.smithi087.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-06-06T07:33:41.923 INFO:teuthology.orchestra.run.smithi195.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T07:33:41.927 INFO:teuthology.orchestra.run.smithi181.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-06-06T07:33:41.953 INFO:teuthology.orchestra.run.smithi181.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-06-06T07:33:41.957 INFO:teuthology.orchestra.run.smithi195.stdout:(95/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 330 kB/s | 44 kB 00:00 2024-06-06T07:33:41.960 INFO:teuthology.orchestra.run.smithi087.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-06-06T07:33:41.978 INFO:teuthology.orchestra.run.smithi181.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-06-06T07:33:41.982 INFO:teuthology.orchestra.run.smithi195.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-06-06T07:33:42.003 INFO:teuthology.orchestra.run.smithi181.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-06-06T07:33:42.024 INFO:teuthology.orchestra.run.smithi195.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-06-06T07:33:42.027 INFO:teuthology.orchestra.run.smithi087.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-06-06T07:33:42.029 INFO:teuthology.orchestra.run.smithi181.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-06T07:33:42.050 INFO:teuthology.orchestra.run.smithi195.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-06-06T07:33:42.060 INFO:teuthology.orchestra.run.smithi087.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 633 kB/s | 19 kB 00:00 2024-06-06T07:33:42.075 INFO:teuthology.orchestra.run.smithi195.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T07:33:42.100 INFO:teuthology.orchestra.run.smithi195.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T07:33:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-06-06T07:33:42.125 INFO:teuthology.orchestra.run.smithi195.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T07:33:42.141 INFO:teuthology.orchestra.run.smithi181.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 415 kB/s | 46 kB 00:00 2024-06-06T07:33:42.144 INFO:teuthology.orchestra.run.smithi087.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 785 kB/s | 26 kB 00:00 2024-06-06T07:33:42.151 INFO:teuthology.orchestra.run.smithi195.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-06-06T07:33:42.151 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:42.154 INFO:teuthology.orchestra.run.smithi195.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T07:33:42.196 INFO:teuthology.orchestra.run.smithi181.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 1.4 MB/s | 79 kB 00:00 2024-06-06T07:33:42.201 INFO:teuthology.orchestra.run.smithi195.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-06-06T07:33:42.221 INFO:teuthology.orchestra.run.smithi181.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-06T07:33:42.247 INFO:teuthology.orchestra.run.smithi181.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-06-06T07:33:42.288 INFO:teuthology.orchestra.run.smithi181.stdout:(133/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-06-06T07:33:42.314 INFO:teuthology.orchestra.run.smithi181.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-06-06T07:33:42.339 INFO:teuthology.orchestra.run.smithi181.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-06-06T07:33:42.361 INFO:teuthology.orchestra.run.smithi087.stdout:(109/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-06-06T07:33:42.364 INFO:teuthology.orchestra.run.smithi181.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-06T07:33:42.368 INFO:teuthology.orchestra.run.smithi195.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-06-06T07:33:42.386 INFO:teuthology.orchestra.run.smithi087.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 200 kB/s | 49 kB 00:00 2024-06-06T07:33:42.390 INFO:teuthology.orchestra.run.smithi181.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-06T07:33:42.415 INFO:teuthology.orchestra.run.smithi181.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-06-06T07:33:42.440 INFO:teuthology.orchestra.run.smithi181.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-06-06T07:33:42.444 INFO:teuthology.orchestra.run.smithi195.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-06-06T07:33:42.454 INFO:teuthology.orchestra.run.smithi087.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.7 MB/s | 151 kB 00:00 2024-06-06T07:33:42.466 INFO:teuthology.orchestra.run.smithi181.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-06-06T07:33:42.479 INFO:teuthology.orchestra.run.smithi087.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-06-06T07:33:42.491 INFO:teuthology.orchestra.run.smithi181.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-06-06T07:33:42.502 INFO:teuthology.orchestra.run.smithi195.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-06-06T07:33:42.516 INFO:teuthology.orchestra.run.smithi181.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-06-06T07:33:42.527 INFO:teuthology.orchestra.run.smithi195.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-06-06T07:33:42.541 INFO:teuthology.orchestra.run.smithi181.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-06T07:33:42.544 INFO:teuthology.orchestra.run.smithi195.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-06-06T07:33:42.554 INFO:teuthology.orchestra.run.smithi087.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-06-06T07:33:42.561 INFO:teuthology.orchestra.run.smithi195.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 773 kB/s | 26 kB 00:00 2024-06-06T07:33:42.578 INFO:teuthology.orchestra.run.smithi195.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-06-06T07:33:42.588 INFO:teuthology.orchestra.run.smithi087.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-06-06T07:33:42.591 INFO:teuthology.orchestra.run.smithi181.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-06-06T07:33:42.604 INFO:teuthology.orchestra.run.smithi195.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-06-06T07:33:42.613 INFO:teuthology.orchestra.run.smithi087.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-06-06T07:33:42.617 INFO:teuthology.orchestra.run.smithi181.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-06-06T07:33:42.629 INFO:teuthology.orchestra.run.smithi195.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-06-06T07:33:42.638 INFO:teuthology.orchestra.run.smithi087.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-06-06T07:33:42.642 INFO:teuthology.orchestra.run.smithi181.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T07:33:42.655 INFO:teuthology.orchestra.run.smithi087.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 357 kB/s | 15 kB 00:00 2024-06-06T07:33:42.667 INFO:teuthology.orchestra.run.smithi181.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-06-06T07:33:42.696 INFO:teuthology.orchestra.run.smithi195.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-06-06T07:33:42.718 INFO:teuthology.orchestra.run.smithi181.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-06T07:33:42.721 INFO:teuthology.orchestra.run.smithi195.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-06T07:33:42.722 INFO:teuthology.orchestra.run.smithi087.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-06-06T07:33:42.746 INFO:teuthology.orchestra.run.smithi195.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-06-06T07:33:42.747 INFO:teuthology.orchestra.run.smithi087.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-06-06T07:33:42.772 INFO:teuthology.orchestra.run.smithi195.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-06-06T07:33:42.773 INFO:teuthology.orchestra.run.smithi087.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-06T07:33:42.797 INFO:teuthology.orchestra.run.smithi195.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-06-06T07:33:42.798 INFO:teuthology.orchestra.run.smithi087.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-06-06T07:33:42.822 INFO:teuthology.orchestra.run.smithi195.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-06T07:33:42.823 INFO:teuthology.orchestra.run.smithi087.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-06-06T07:33:42.848 INFO:teuthology.orchestra.run.smithi087.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-06T07:33:42.851 INFO:teuthology.orchestra.run.smithi181.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 17 MB/s | 2.3 MB 00:00 2024-06-06T07:33:42.856 INFO:teuthology.orchestra.run.smithi195.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-06T07:33:42.874 INFO:teuthology.orchestra.run.smithi087.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-06-06T07:33:42.881 INFO:teuthology.orchestra.run.smithi195.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-06-06T07:33:42.899 INFO:teuthology.orchestra.run.smithi087.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-06-06T07:33:42.906 INFO:teuthology.orchestra.run.smithi195.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T07:33:42.918 INFO:teuthology.orchestra.run.smithi181.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T07:33:42.924 INFO:teuthology.orchestra.run.smithi087.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-06-06T07:33:42.940 INFO:teuthology.orchestra.run.smithi195.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-06-06T07:33:42.950 INFO:teuthology.orchestra.run.smithi087.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-06-06T07:33:42.952 INFO:teuthology.orchestra.run.smithi181.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-06T07:33:42.975 INFO:teuthology.orchestra.run.smithi087.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-06-06T07:33:42.990 INFO:teuthology.orchestra.run.smithi195.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-06-06T07:33:42.994 INFO:teuthology.orchestra.run.smithi181.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T07:33:43.000 INFO:teuthology.orchestra.run.smithi087.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T07:33:43.015 INFO:teuthology.orchestra.run.smithi195.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 715 kB/s | 18 kB 00:00 2024-06-06T07:33:43.026 INFO:teuthology.orchestra.run.smithi087.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-06T07:33:43.036 INFO:teuthology.orchestra.run.smithi181.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 886 kB/s | 37 kB 00:00 2024-06-06T07:33:43.041 INFO:teuthology.orchestra.run.smithi195.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-06-06T07:33:43.051 INFO:teuthology.orchestra.run.smithi087.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-06T07:33:43.066 INFO:teuthology.orchestra.run.smithi195.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 771 kB/s | 19 kB 00:00 2024-06-06T07:33:43.070 INFO:teuthology.orchestra.run.smithi181.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 6.0 MB/s | 207 kB 00:00 2024-06-06T07:33:43.085 INFO:teuthology.orchestra.run.smithi087.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-06-06T07:33:43.110 INFO:teuthology.orchestra.run.smithi087.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-06-06T07:33:43.112 INFO:teuthology.orchestra.run.smithi181.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-06-06T07:33:43.133 INFO:teuthology.orchestra.run.smithi195.stdout:(127/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-06-06T07:33:43.152 INFO:teuthology.orchestra.run.smithi087.stdout:(134/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-06-06T07:33:43.158 INFO:teuthology.orchestra.run.smithi195.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-06-06T07:33:43.170 INFO:teuthology.orchestra.run.smithi181.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T07:33:43.177 INFO:teuthology.orchestra.run.smithi087.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-06-06T07:33:43.200 INFO:teuthology.orchestra.run.smithi195.stdout:(129/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-06-06T07:33:43.202 INFO:teuthology.orchestra.run.smithi087.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-06-06T07:33:43.226 INFO:teuthology.orchestra.run.smithi195.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-06-06T07:33:43.228 INFO:teuthology.orchestra.run.smithi087.stdout:(137/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 765 kB/s | 90 kB 00:00 2024-06-06T07:33:43.246 INFO:teuthology.orchestra.run.smithi181.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T07:33:43.251 INFO:teuthology.orchestra.run.smithi195.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-06-06T07:33:43.253 INFO:teuthology.orchestra.run.smithi087.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-06-06T07:33:43.277 INFO:teuthology.orchestra.run.smithi195.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-06T07:33:43.278 INFO:teuthology.orchestra.run.smithi087.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-06-06T07:33:43.302 INFO:teuthology.orchestra.run.smithi195.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-06-06T07:33:43.304 INFO:teuthology.orchestra.run.smithi087.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-06-06T07:33:43.327 INFO:teuthology.orchestra.run.smithi195.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 334 kB/s | 16 kB 00:00 2024-06-06T07:33:43.329 INFO:teuthology.orchestra.run.smithi087.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-06T07:33:43.329 INFO:teuthology.orchestra.run.smithi181.stdout:(158/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.7 MB/s | 3.1 MB 00:00 2024-06-06T07:33:43.352 INFO:teuthology.orchestra.run.smithi195.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-06-06T07:33:43.354 INFO:teuthology.orchestra.run.smithi087.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-06T07:33:43.377 INFO:teuthology.orchestra.run.smithi195.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-06-06T07:33:43.379 INFO:teuthology.orchestra.run.smithi087.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-06T07:33:43.403 INFO:teuthology.orchestra.run.smithi195.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-06T07:33:43.413 INFO:teuthology.orchestra.run.smithi087.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-06-06T07:33:43.428 INFO:teuthology.orchestra.run.smithi195.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-06T07:33:43.438 INFO:teuthology.orchestra.run.smithi087.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-06-06T07:33:43.453 INFO:teuthology.orchestra.run.smithi195.stdout:(139/159): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-06-06T07:33:43.464 INFO:teuthology.orchestra.run.smithi087.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-06-06T07:33:43.479 INFO:teuthology.orchestra.run.smithi195.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-06-06T07:33:43.489 INFO:teuthology.orchestra.run.smithi087.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-06T07:33:43.504 INFO:teuthology.orchestra.run.smithi195.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-06-06T07:33:43.529 INFO:teuthology.orchestra.run.smithi195.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-06-06T07:33:43.539 INFO:teuthology.orchestra.run.smithi087.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-06T07:33:43.555 INFO:teuthology.orchestra.run.smithi195.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-06T07:33:43.580 INFO:teuthology.orchestra.run.smithi195.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-06-06T07:33:43.596 INFO:teuthology.orchestra.run.smithi181.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 3.4 MB 00:00 2024-06-06T07:33:43.602 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-06-06T07:33:43.602 INFO:teuthology.orchestra.run.smithi181.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-06T07:33:43.605 INFO:teuthology.orchestra.run.smithi195.stdout:(145/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-06-06T07:33:43.622 INFO:teuthology.orchestra.run.smithi195.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-06T07:33:43.647 INFO:teuthology.orchestra.run.smithi195.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-06-06T07:33:43.689 INFO:teuthology.orchestra.run.smithi087.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-06T07:33:43.698 INFO:teuthology.orchestra.run.smithi195.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-06-06T07:33:43.882 INFO:teuthology.orchestra.run.smithi087.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-06-06T07:33:43.923 INFO:teuthology.orchestra.run.smithi087.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-06-06T07:33:43.982 INFO:teuthology.orchestra.run.smithi087.stdout:(152/159): librados2-19.0.0-2946.g3d89accf.el9. 6.5 MB/s | 3.4 MB 00:00 2024-06-06T07:33:43.998 INFO:teuthology.orchestra.run.smithi195.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 7.7 MB/s | 2.3 MB 00:00 2024-06-06T07:33:44.007 INFO:teuthology.orchestra.run.smithi087.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-06-06T07:33:44.049 INFO:teuthology.orchestra.run.smithi087.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T07:33:44.065 INFO:teuthology.orchestra.run.smithi195.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T07:33:44.066 INFO:teuthology.orchestra.run.smithi087.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 438 kB/s | 37 kB 00:00 2024-06-06T07:33:44.092 INFO:teuthology.orchestra.run.smithi087.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.0 MB/s | 260 kB 00:00 2024-06-06T07:33:44.107 INFO:teuthology.orchestra.run.smithi195.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-06-06T07:33:44.149 INFO:teuthology.orchestra.run.smithi195.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T07:33:44.167 INFO:teuthology.orchestra.run.smithi087.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T07:33:44.207 INFO:teuthology.orchestra.run.smithi195.stdout:(153/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-06-06T07:33:44.217 INFO:teuthology.orchestra.run.smithi087.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-06-06T07:33:44.225 INFO:teuthology.orchestra.run.smithi195.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 487 kB/s | 37 kB 00:00 2024-06-06T07:33:44.267 INFO:teuthology.orchestra.run.smithi195.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-06-06T07:33:44.325 INFO:teuthology.orchestra.run.smithi195.stdout:(156/159): librados2-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 3.4 MB 00:00 2024-06-06T07:33:44.367 INFO:teuthology.orchestra.run.smithi195.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.3 MB/s | 207 kB 00:00 2024-06-06T07:33:44.409 INFO:teuthology.orchestra.run.smithi195.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 9.5 MB/s | 1.3 MB 00:00 2024-06-06T07:33:44.493 INFO:teuthology.orchestra.run.smithi087.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-06-06T07:33:44.499 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-06-06T07:33:44.499 INFO:teuthology.orchestra.run.smithi087.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-06T07:33:44.542 INFO:teuthology.orchestra.run.smithi195.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-06-06T07:33:44.549 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-06-06T07:33:44.549 INFO:teuthology.orchestra.run.smithi195.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-06T07:33:44.816 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-06-06T07:33:44.974 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-06-06T07:33:44.974 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-06-06T07:33:45.735 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-06-06T07:33:45.758 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-06-06T07:33:45.893 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-06-06T07:33:45.893 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-06-06T07:33:45.916 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-06-06T07:33:45.917 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-06-06T07:33:46.784 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-06-06T07:33:46.784 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-06-06T07:33:47.714 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-06-06T07:33:47.715 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-06-06T07:33:48.024 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-06-06T07:33:48.024 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-06-06T07:33:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-06-06T07:33:53.233 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T07:33:53.271 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T07:33:53.325 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T07:33:53.358 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T07:33:53.385 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T07:33:53.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T07:33:53.757 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T07:33:53.852 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:53.928 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-06-06T07:33:53.974 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:54.003 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:54.005 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T07:33:54.043 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T07:33:54.095 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:54.096 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T07:33:54.124 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:33:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T07:33:54.156 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T07:33:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T07:33:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T07:33:54.229 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-06-06T07:33:54.322 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T07:33:54.337 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T07:33:54.360 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T07:33:54.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T07:33:54.429 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T07:33:54.433 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:54.464 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T07:33:54.490 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T07:33:54.520 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:54.537 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T07:33:54.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:54.547 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T07:33:54.589 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:54.611 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:54.637 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:54.800 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:54.802 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:54.870 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T07:33:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:54.904 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:33:54.927 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:54.937 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T07:33:54.974 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:55.079 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:33:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:55.117 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T07:33:55.168 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T07:33:55.198 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:33:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:55.230 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:33:55.271 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T07:33:55.272 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:55.317 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:55.340 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:55.361 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:55.402 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:55.424 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:55.443 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:55.461 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T07:33:55.519 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T07:33:55.547 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:55.572 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:55.599 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:55.650 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:33:55.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T07:33:55.698 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:55.744 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T07:33:55.766 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:55.778 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T07:33:55.785 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T07:33:55.809 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T07:33:55.843 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T07:33:55.880 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:55.907 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T07:33:55.940 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:55.942 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T07:33:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:55.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T07:33:56.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T07:33:56.035 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.046 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T07:33:56.116 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T07:33:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.161 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T07:33:56.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T07:33:56.201 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T07:33:56.237 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T07:33:56.312 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T07:33:56.313 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T07:33:56.373 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T07:33:56.382 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T07:33:56.411 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T07:33:56.448 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T07:33:56.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T07:33:56.485 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.526 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T07:33:56.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T07:33:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.581 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T07:33:56.600 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T07:33:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T07:33:56.697 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T07:33:56.731 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T07:33:56.733 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T07:33:56.775 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T07:33:56.786 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T07:33:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T07:33:56.807 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T07:33:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T07:33:56.896 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T07:33:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T07:33:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T07:33:56.992 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T07:33:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T07:33:57.031 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T07:33:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T07:33:57.065 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T07:33:57.117 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T07:33:57.129 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T07:33:57.152 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T07:33:57.164 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T07:33:57.195 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T07:33:57.207 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T07:33:57.225 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T07:33:57.240 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T07:33:57.268 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T07:33:57.293 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T07:33:57.296 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T07:33:57.337 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T07:33:57.338 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T07:33:57.374 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T07:33:57.390 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T07:33:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T07:33:57.412 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T07:33:57.421 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T07:33:57.439 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T07:33:57.456 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T07:33:57.480 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T07:33:57.518 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T07:33:57.532 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T07:33:57.551 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T07:33:57.553 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T07:33:57.567 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T07:33:57.609 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T07:33:57.611 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T07:33:57.639 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T07:33:57.708 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T07:33:57.708 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T07:33:57.753 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T07:33:57.780 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T07:33:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T07:33:57.834 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T07:33:57.907 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T07:33:57.939 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T07:33:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T07:33:57.975 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T07:33:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T07:33:58.022 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T07:33:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T07:33:58.093 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T07:33:58.134 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T07:33:58.134 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T07:33:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T07:33:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T07:33:58.269 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T07:33:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T07:33:58.312 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T07:33:58.327 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T07:33:58.353 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T07:33:58.365 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T07:33:58.406 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T07:33:58.446 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T07:33:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T07:33:58.554 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T07:33:58.558 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T07:33:58.593 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T07:33:58.610 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T07:33:58.619 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T07:33:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T07:33:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T07:33:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T07:33:58.665 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T07:33:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T07:33:58.709 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T07:33:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T07:33:58.738 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T07:33:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T07:33:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T07:33:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T07:33:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T07:33:58.838 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T07:33:58.860 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T07:33:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T07:33:58.897 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T07:33:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T07:33:58.939 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T07:33:58.955 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T07:33:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T07:33:58.981 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T07:33:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T07:33:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T07:33:59.027 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T07:33:59.056 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T07:33:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T07:33:59.090 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T07:33:59.117 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T07:33:59.124 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T07:33:59.156 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T07:33:59.159 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T07:33:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T07:33:59.191 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T07:33:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T07:33:59.229 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T07:33:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T07:33:59.278 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T07:33:59.309 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T07:33:59.312 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T07:33:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T07:33:59.353 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T07:33:59.363 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T07:33:59.400 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T07:33:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T07:33:59.405 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T07:33:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T07:33:59.474 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T07:33:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T07:33:59.500 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T07:33:59.524 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T07:33:59.542 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T07:33:59.598 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T07:33:59.641 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T07:33:59.732 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T07:33:59.734 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T07:33:59.765 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T07:33:59.776 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T07:33:59.793 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T07:33:59.826 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T07:33:59.835 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T07:33:59.874 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T07:33:59.875 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T07:33:59.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T07:33:59.915 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T07:33:59.936 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T07:33:59.947 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T07:33:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T07:33:59.972 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T07:34:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T07:34:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T07:34:00.063 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T07:34:00.064 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T07:34:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T07:34:00.191 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T07:34:00.194 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T07:34:00.229 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T07:34:00.233 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T07:34:00.257 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T07:34:00.281 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T07:34:00.292 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T07:34:00.331 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T07:34:00.363 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T07:34:00.380 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T07:34:00.392 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T07:34:00.427 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T07:34:00.443 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T07:34:00.460 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T07:34:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T07:34:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T07:34:00.606 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T07:34:00.623 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T07:34:00.651 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T07:34:00.675 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T07:34:00.685 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T07:34:00.706 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T07:34:00.753 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T07:34:00.784 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T07:34:00.832 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T07:34:00.851 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T07:34:00.877 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T07:34:00.910 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T07:34:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T07:34:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T07:34:01.160 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T07:34:01.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T07:34:01.194 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T07:34:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T07:34:01.241 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T07:34:01.272 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T07:34:01.338 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T07:34:01.396 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T07:34:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T07:34:01.520 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T07:34:01.568 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T07:34:01.622 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T07:34:01.672 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T07:34:01.705 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T07:34:01.754 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T07:34:01.783 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T07:34:01.804 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T07:34:01.852 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T07:34:01.860 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T07:34:01.897 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T07:34:01.902 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T07:34:01.984 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T07:34:02.017 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T07:34:02.033 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T07:34:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T07:34:02.248 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T07:34:02.326 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T07:34:02.501 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T07:34:02.614 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T07:34:02.971 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T07:34:03.096 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T07:34:03.231 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T07:34:03.315 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T07:34:03.344 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T07:34:03.377 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T07:34:03.686 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T07:34:03.756 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:03.831 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T07:34:03.861 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T07:34:03.897 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T07:34:03.928 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T07:34:03.956 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T07:34:03.989 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T07:34:04.302 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T07:34:04.344 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:04.365 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T07:34:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T07:34:04.401 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T07:34:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T07:34:04.472 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T07:34:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T07:34:04.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T07:34:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T07:34:04.823 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T07:34:04.868 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:04.958 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T07:34:04.959 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T07:34:04.989 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T07:34:04.989 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:05.026 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T07:34:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T07:34:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T07:34:05.521 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:05.592 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T07:34:05.622 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T07:34:06.780 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:06.816 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.352 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.391 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.394 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:07.420 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.469 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:07.469 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T07:34:07.469 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T07:34:07.469 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:07.495 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:07.824 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.860 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:07.958 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:08.007 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:08.099 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T07:34:08.099 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T07:34:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:08.400 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T07:34:08.439 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:08.530 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T07:34:08.530 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T07:34:08.530 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T07:34:08.530 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:08.557 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:18.948 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-06-06T07:34:18.949 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:19.116 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:19.132 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:19.132 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:19.132 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T07:34:19.132 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:19.976 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-06-06T07:34:19.976 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-06-06T07:34:19.976 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-06-06T07:34:19.977 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-06-06T07:34:19.977 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-06-06T07:34:19.977 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-06-06T07:34:19.977 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-06-06T07:34:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-06-06T07:34:20.114 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-06-06T07:34:20.115 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-06-06T07:34:20.115 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-06-06T07:34:20.115 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:20.141 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:20.157 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:20.157 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:20.158 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T07:34:20.158 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:20.281 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:20.297 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T07:34:20.297 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:20.297 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T07:34:20.297 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:20.594 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:20.620 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:20.844 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:20.867 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:20.868 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:20.868 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T07:34:20.868 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:20.868 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:20.868 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:21.312 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:21.335 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:21.442 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T07:34:21.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T07:34:21.571 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T07:34:21.646 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:21.673 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:21.756 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T07:34:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T07:34:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:21.804 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T07:34:21.883 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T07:34:21.905 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:21.928 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:21.971 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T07:34:22.018 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:22.040 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T07:34:22.040 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:22.041 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T07:34:22.041 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:22.041 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T07:34:22.041 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:22.109 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T07:34:22.155 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T07:34:22.187 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T07:34:22.214 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T07:34:22.380 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:22.403 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:22.404 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:22.404 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T07:34:22.404 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:22.404 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:22.404 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:22.502 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:22.502 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T07:34:22.525 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T07:34:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T07:34:22.632 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T07:34:22.666 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:22.681 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T07:34:22.696 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:22.736 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T07:34:22.736 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T07:34:22.791 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T07:34:22.798 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T07:34:22.828 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:22.839 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T07:34:22.909 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T07:34:22.920 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T07:34:22.978 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T07:34:22.995 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T07:34:23.056 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T07:34:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T07:34:23.143 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T07:34:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T07:34:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T07:34:23.233 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T07:34:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T07:34:23.328 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T07:34:23.360 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T07:34:23.387 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T07:34:23.627 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:23.657 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:23.696 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T07:34:23.758 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T07:34:23.788 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:23.802 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T07:34:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T07:34:23.860 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:23.896 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:23.920 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T07:34:23.951 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:24.818 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:24.848 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:24.864 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:24.908 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:24.974 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T07:34:25.009 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:25.011 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T07:34:25.230 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:25.253 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:25.253 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:25.253 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T07:34:25.253 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:25.253 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:25.254 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:25.299 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:25.311 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:25.409 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T07:34:25.465 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T07:34:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:25.840 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:25.941 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:25.968 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T07:34:25.995 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:26.094 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T07:34:26.097 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T07:34:26.117 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:26.184 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:26.196 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:26.274 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T07:34:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T07:34:26.311 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T07:34:26.335 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T07:34:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T07:34:26.378 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:26.390 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T07:34:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:26.417 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.417 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T07:34:26.417 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:26.417 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:26.461 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:26.483 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:26.484 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.484 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T07:34:26.484 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:26.486 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T07:34:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:26.538 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T07:34:26.551 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:26.551 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.551 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T07:34:26.551 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:26.551 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:26.552 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:26.885 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:26.886 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T07:34:26.886 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:26.886 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:27.232 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T07:34:27.262 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:27.285 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:27.285 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.285 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T07:34:27.285 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:27.285 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:27.286 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:27.351 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:27.351 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.352 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T07:34:27.352 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:27.405 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:27.427 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:27.427 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.428 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T07:34:27.428 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:27.428 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:27.428 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:27.430 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T07:34:27.464 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:27.487 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T07:34:27.487 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.487 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T07:34:27.487 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:27.488 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T07:34:27.488 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T07:34:27.562 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.562 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T07:34:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T07:34:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:27.739 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:27.761 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:27.761 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.761 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T07:34:27.762 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:27.762 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:27.762 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:27.934 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:27.955 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T07:34:27.956 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T07:34:27.956 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T07:34:27.956 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:27.956 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T07:34:27.956 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:31.400 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T07:34:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T07:34:31.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T07:34:31.514 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T07:34:31.541 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T07:34:31.725 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T07:34:31.771 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T07:34:31.803 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T07:34:31.844 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T07:34:31.873 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T07:34:31.920 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:32.267 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T07:34:32.308 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T07:34:32.349 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T07:34:32.381 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T07:34:32.409 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T07:34:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T07:34:32.503 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T07:34:32.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:32.560 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T07:34:32.563 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:32.589 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T07:34:32.592 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T07:34:32.619 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T07:34:32.638 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T07:34:32.670 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T07:34:32.711 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T07:34:32.741 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T07:34:32.787 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:32.807 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T07:34:32.857 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T07:34:32.890 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T07:34:32.921 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T07:34:32.951 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T07:34:32.989 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:33.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T07:34:33.407 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.443 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.463 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.489 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T07:34:33.539 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T07:34:33.620 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T07:34:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.659 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T07:34:33.678 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.684 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:33.698 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T07:34:33.709 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T07:34:33.736 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:33.736 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:33.820 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:33.845 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T07:34:33.845 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:33.929 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:33.954 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T07:34:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:34.148 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:34.149 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:34.241 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:34.266 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T07:34:34.300 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T07:34:34.329 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:34.390 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T07:34:34.422 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T07:34:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T07:34:34.534 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T07:34:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T07:34:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T07:34:34.628 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:34.628 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:34.632 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T07:34:34.673 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T07:34:34.713 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:34.737 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T07:34:34.738 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:34.822 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:34.840 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T07:34:34.846 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T07:34:34.852 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:34.852 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:34.868 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T07:34:34.893 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:34.919 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T07:34:34.945 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:34.945 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:35.030 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T07:34:35.032 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:35.032 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:35.063 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T07:34:35.063 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:35.146 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:35.155 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T07:34:35.174 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T07:34:35.180 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T07:34:35.187 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:35.187 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:35.238 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:35.383 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T07:34:35.383 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:35.508 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T07:34:35.533 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T07:34:35.599 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:40.027 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:34:40.028 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T07:34:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T07:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T07:34:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T07:34:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T07:34:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T07:34:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T07:34:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T07:34:40.036 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T07:34:40.037 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T07:34:40.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T07:34:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T07:34:40.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T07:34:40.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T07:34:40.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T07:34:40.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T07:34:40.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T07:34:40.816 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T07:34:40.817 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T07:34:40.829 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T07:34:40.829 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T07:34:40.829 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T07:34:40.829 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T07:34:40.829 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T07:34:40.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T07:34:40.831 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T07:34:40.832 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T07:34:40.833 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T07:34:40.834 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T07:34:40.835 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T07:34:40.836 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T07:34:40.837 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T07:34:40.838 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T07:34:40.839 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T07:34:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T07:34:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T07:34:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T07:34:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T07:34:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T07:34:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T07:34:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T07:34:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T07:34:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T07:34:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T07:34:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T07:34:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T07:34:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T07:34:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T07:34:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T07:34:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T07:34:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T07:34:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T07:34:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T07:34:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T07:34:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T07:34:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T07:34:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T07:34:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T07:34:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-06-06T07:34:42.724 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-06-06T07:34:43.175 DEBUG:teuthology.parallel:result is None 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.701 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-06-06T07:34:43.702 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.703 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.704 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.705 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T07:34:43.706 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T07:34:43.707 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T07:34:43.708 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T07:34:43.709 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T07:34:43.710 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T07:34:43.711 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T07:34:43.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T07:34:43.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T07:34:43.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T07:34:43.715 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T07:34:43.716 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T07:34:43.716 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:34:43.716 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-06-06T07:34:43.809 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T07:34:43.809 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:43.809 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.810 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.811 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.812 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.813 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T07:34:43.814 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T07:34:43.815 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T07:34:43.816 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.817 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T07:34:43.818 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T07:34:43.819 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T07:34:43.820 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T07:34:43.821 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T07:34:43.822 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T07:34:43.823 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T07:34:43.824 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T07:34:43.824 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:34:43.824 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-06-06T07:34:44.126 DEBUG:teuthology.parallel:result is None 2024-06-06T07:34:44.233 DEBUG:teuthology.parallel:result is None 2024-06-06T07:34:44.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:44.463 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T07:34:44.485 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.486 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.486 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T07:34:44.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:44.628 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T07:34:44.653 INFO:teuthology.orchestra.run.smithi181.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.654 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.654 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T07:34:44.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:44.869 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T07:34:44.894 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.894 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T07:34:44.894 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T07:34:44.896 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-06T07:34:44.897 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:34:44.897 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T07:34:44.924 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:34:44.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T07:34:44.955 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:34:44.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T07:34:44.984 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-06T07:34:44.984 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:34:44.984 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T07:34:45.010 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T07:34:45.074 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:34:45.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T07:34:45.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T07:34:45.172 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:34:45.172 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T07:34:45.199 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T07:34:45.267 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-06T07:34:45.267 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:34:45.267 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T07:34:45.292 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T07:34:45.356 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:34:45.356 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T07:34:45.384 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T07:34:45.452 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:34:45.452 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T07:34:45.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T07:34:45.548 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-06T07:34:45.548 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:34:45.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T07:34:45.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T07:34:45.638 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:34:45.638 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T07:34:45.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T07:34:45.732 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:34:45.732 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T07:34:45.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T07:34:45.827 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-06T07:34:45.924 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T07:34:45.925 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:45.925 INFO:tasks.cephadm:Cluster fsid is 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:34:45.925 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-06T07:34:45.926 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.87', 'mon.b': '172.21.15.181', 'mon.c': '172.21.15.195'} 2024-06-06T07:34:45.926 INFO:tasks.cephadm:First mon is mon.a on smithi087 2024-06-06T07:34:45.926 INFO:tasks.cephadm:First mgr is x 2024-06-06T07:34:45.926 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-06T07:34:45.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname $(hostname -s) 2024-06-06T07:34:45.952 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-06-06T07:34:45.980 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname $(hostname -s) 2024-06-06T07:34:46.008 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-06T07:34:46.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:46.228 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-06T07:34:46.437 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:34:46.439 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-06T07:34:46.439 DEBUG:teuthology.orchestra.run.smithi087:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:46.975 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 07:34 /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:46.975 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.376 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 07:34 /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.376 DEBUG:teuthology.orchestra.run.smithi195:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.786 INFO:teuthology.orchestra.run.smithi195.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 07:34 /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.786 DEBUG:teuthology.orchestra.run.smithi087:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.806 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.825 DEBUG:teuthology.orchestra.run.smithi195:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T07:34:47.858 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-06T07:34:47.859 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T07:34:47.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T07:34:47.868 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T07:34:48.726 INFO:teuthology.orchestra.run.smithi087.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T07:34:48.728 INFO:teuthology.orchestra.run.smithi195.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T07:34:48.755 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: "repo_digests": [ 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-06-06T07:35:26.126 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-06-06T07:35:38.254 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-06-06T07:35:38.255 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-06-06T07:35:40.563 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:35:40.563 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T07:35:40.563 INFO:teuthology.orchestra.run.smithi087.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T07:35:40.563 INFO:teuthology.orchestra.run.smithi087.stdout: "repo_digests": [ 2024-06-06T07:35:40.564 INFO:teuthology.orchestra.run.smithi087.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T07:35:40.564 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T07:35:40.564 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:35:40.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph 2024-06-06T07:35:40.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-06-06T07:35:40.680 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph 2024-06-06T07:35:40.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /etc/ceph 2024-06-06T07:35:40.737 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-06-06T07:35:40.765 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /etc/ceph 2024-06-06T07:35:40.792 INFO:tasks.cephadm:Writing seed config... 2024-06-06T07:35:40.793 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-06T07:35:40.793 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-06T07:35:40.793 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-06T07:35:40.793 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-06T07:35:40.793 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-06T07:35:40.794 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-06T07:35:40.795 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-06T07:35:40.796 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-06T07:35:40.797 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-06T07:35:40.797 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:35:40.797 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-06T07:35:40.813 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3f55109c-23d7-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-06T07:35:40.813 DEBUG:teuthology.orchestra.run.smithi087:mon.a> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service 2024-06-06T07:35:40.856 DEBUG:teuthology.orchestra.run.smithi087:mgr.x> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-06T07:35:40.898 INFO:tasks.cephadm:Bootstrapping... 2024-06-06T07:35:40.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.87 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:35:41.100 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-06-06T07:35:41.100 INFO:teuthology.orchestra.run.smithi087.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '3f55109c-23d7-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.87', '--skip-admin-label'] 2024-06-06T07:35:41.101 INFO:teuthology.orchestra.run.smithi087.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-06T07:35:41.101 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying podman|docker is present... 2024-06-06T07:35:41.127 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 5.0.2 2024-06-06T07:35:41.127 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying lvm2 is present... 2024-06-06T07:35:41.127 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying time synchronization is in place... 2024-06-06T07:35:41.134 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T07:35:41.134 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T07:35:41.140 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T07:35:41.140 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:35:41.147 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-06-06T07:35:41.153 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-06-06T07:35:41.153 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-06-06T07:35:41.154 INFO:teuthology.orchestra.run.smithi087.stdout:Repeating the final host check... 2024-06-06T07:35:41.177 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 5.0.2 2024-06-06T07:35:41.177 INFO:teuthology.orchestra.run.smithi087.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-06T07:35:41.177 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl is present 2024-06-06T07:35:41.177 INFO:teuthology.orchestra.run.smithi087.stdout:lvcreate is present 2024-06-06T07:35:41.184 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T07:35:41.184 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T07:35:41.189 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T07:35:41.189 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:35:41.196 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-06-06T07:35:41.202 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-06-06T07:35:41.202 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-06-06T07:35:41.202 INFO:teuthology.orchestra.run.smithi087.stdout:Host looks OK 2024-06-06T07:35:41.202 INFO:teuthology.orchestra.run.smithi087.stdout:Cluster fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:41.203 INFO:teuthology.orchestra.run.smithi087.stdout:Acquiring lock 140508622316928 on /run/cephadm/3f55109c-23d7-11ef-bc9b-c7b262605968.lock 2024-06-06T07:35:41.203 INFO:teuthology.orchestra.run.smithi087.stdout:Lock 140508622316928 acquired on /run/cephadm/3f55109c-23d7-11ef-bc9b-c7b262605968.lock 2024-06-06T07:35:41.203 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 3300 ... 2024-06-06T07:35:41.203 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 6789 ... 2024-06-06T07:35:41.203 INFO:teuthology.orchestra.run.smithi087.stdout:Base mon IP(s) is [172.21.15.87:3300, 172.21.15.87:6789], mon addrv is [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-06-06T07:35:41.206 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.87 metric 100 2024-06-06T07:35:41.207 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.87 metric 100 2024-06-06T07:35:41.209 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-06T07:35:41.209 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-06T07:35:41.210 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T07:35:41.215 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-06-06T07:35:41.216 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-06-06T07:35:41.216 INFO:teuthology.orchestra.run.smithi087.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-06T07:35:41.216 INFO:teuthology.orchestra.run.smithi087.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-06T07:35:41.216 INFO:teuthology.orchestra.run.smithi087.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T07:35:41.707 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Getting image source signatures 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T07:35:41.708 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-06T07:35:42.703 INFO:teuthology.orchestra.run.smithi087.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:35:42.703 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:35:42.703 INFO:teuthology.orchestra.run.smithi087.stdout:Extracting ceph user uid/gid from container image... 2024-06-06T07:35:43.683 INFO:teuthology.orchestra.run.smithi087.stdout:stat: stdout 167 167 2024-06-06T07:35:43.684 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial keys... 2024-06-06T07:35:44.530 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQDQZmFm218AABAAeRlT1qq5aF5ggk5UxQeKvA== 2024-06-06T07:35:45.426 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQDQZmFmYGz2MxAAIWRXvA97nv8saYGUSZZSuw== 2024-06-06T07:35:46.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQDRZmFmLfbqKxAAI8BX5az5ftDRdkVcZoku2Q== 2024-06-06T07:35:46.358 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial monmap... 2024-06-06T07:35:47.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T07:35:47.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-06T07:35:47.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:47.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T07:35:47.247 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool for a [v2:172.21.15.87:3300,v1:172.21.15.87:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T07:35:47.248 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-06-06T07:35:47.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: set fsid to 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:47.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T07:35:47.248 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:35:47.248 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mon... 2024-06-06T07:35:48.169 INFO:teuthology.orchestra.run.smithi087.stdout:create mon.a on 2024-06-06T07:35:48.379 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-06T07:35:48.526 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-06T07:35:48.696 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target. 2024-06-06T07:35:48.696 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target. 2024-06-06T07:35:48.947 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a 2024-06-06T07:35:48.947 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-06T07:35:49.109 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target.wants/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@.service. 2024-06-06T07:35:49.652 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T07:35:49.652 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-06-06T07:35:49.659 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T07:35:49.660 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:35:49.660 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-06-06T07:35:49.660 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T07:35:49.660 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon to start... 2024-06-06T07:35:49.660 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon... 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout cluster: 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout id: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout services: 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.553764s) 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-06T07:35:50.719 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout data: 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pgs: 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.072+0000 7fb0d663f700 1 Processor -- start 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.072+0000 7fb0d663f700 1 -- start start 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.072+0000 7fb0d663f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.072+0000 7fb0d663f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0108a10 con 0x7fb0d0108020 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:50.720 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34562/0 (socket says 172.21.15.87:34562) 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cffff700 1 -- 172.21.15.87:0/2113066657 learned_addr learned my addr 172.21.15.87:0/2113066657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cffff700 1 -- 172.21.15.87:0/2113066657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d0109220 con 0x7fb0d0108020 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cffff700 1 --2- 172.21.15.87:0/2113066657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1bf2ce0153fa15 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cf7fe700 1 -- 172.21.15.87:0/2113066657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0004160 con 0x7fb0d0108020 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cf7fe700 1 -- 172.21.15.87:0/2113066657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0004300 con 0x7fb0d0108020 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.073+0000 7fb0cf7fe700 1 -- 172.21.15.87:0/2113066657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c00036a0 con 0x7fb0d0108020 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 -- 172.21.15.87:0/2113066657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 msgr2=0x7fb0d0108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 --2- 172.21.15.87:0/2113066657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c0009bb0 comp rx=0 tx=0).stop 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 -- 172.21.15.87:0/2113066657 shutdown_connections 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 --2- 172.21.15.87:0/2113066657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d0108440 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 -- 172.21.15.87:0/2113066657 >> 172.21.15.87:0/2113066657 conn(0x7fb0d01035e0 msgr2=0x7fb0d0105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:50.721 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 -- 172.21.15.87:0/2113066657 shutdown_connections 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.074+0000 7fb0d663f700 1 -- 172.21.15.87:0/2113066657 wait complete. 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0d663f700 1 Processor -- start 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0d663f700 1 -- start start 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0d663f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0d663f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0109750 con 0x7fb0d0108020 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34578/0 (socket says 172.21.15.87:34578) 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0cffff700 1 -- 172.21.15.87:0/3539986855 learned_addr learned my addr 172.21.15.87:0/3539986855 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0cffff700 1 -- 172.21.15.87:0/3539986855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d019bbc0 con 0x7fb0d0108020 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.075+0000 7fb0cffff700 1 --2- 172.21.15.87:0/3539986855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0000c00 tx=0x7fb0c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0003db0 con 0x7fb0d0108020 2024-06-06T07:35:50.722 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0011630 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d019be50 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0011950 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d019eb40 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0c001a460 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.076+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb0c001b780 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.077+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d0063af0 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.079+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb0c001b980 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.151+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb0d002def0 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.151+0000 7fb0cdffb700 1 -- 172.21.15.87:0/3539986855 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb0c001bb60 con 0x7fb0d0108020 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 msgr2=0x7fb0d019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 --2- 172.21.15.87:0/3539986855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0000c00 tx=0x7fb0c0004750 comp rx=0 tx=0).stop 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 shutdown_connections 2024-06-06T07:35:50.723 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 --2- 172.21.15.87:0/3539986855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0d0108020 0x7fb0d019b680 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:50.724 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 >> 172.21.15.87:0/3539986855 conn(0x7fb0d01035e0 msgr2=0x7fb0d01922e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:50.724 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 shutdown_connections 2024-06-06T07:35:50.724 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:50.152+0000 7fb0d663f700 1 -- 172.21.15.87:0/3539986855 wait complete. 2024-06-06T07:35:50.724 INFO:teuthology.orchestra.run.smithi087.stdout:mon is available 2024-06-06T07:35:50.724 INFO:teuthology.orchestra.run.smithi087.stdout:Assimilating anything we can from ceph.conf... 2024-06-06T07:35:51.659 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:51.659 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T07:35:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.118+0000 7fa8628cb700 1 Processor -- start 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.118+0000 7fa8628cb700 1 -- start start 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa8628cb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa8628cb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10a010 con 0x7fa85c109620 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34584/0 (socket says 172.21.15.87:34584) 2024-06-06T07:35:51.661 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85bfff700 1 -- 172.21.15.87:0/2468572566 learned_addr learned my addr 172.21.15.87:0/2468572566 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85bfff700 1 -- 172.21.15.87:0/2468572566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c10a190 con 0x7fa85c109620 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85bfff700 1 --2- 172.21.15.87:0/2468572566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0098a0 tx=0x7fa84c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa1c8fd0ea42a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.119+0000 7fa85b7fe700 1 -- 172.21.15.87:0/2468572566 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c004160 con 0x7fa85c109620 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa85b7fe700 1 -- 172.21.15.87:0/2468572566 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c004300 con 0x7fa85c109620 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa85b7fe700 1 -- 172.21.15.87:0/2468572566 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c0036a0 con 0x7fa85c109620 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 -- 172.21.15.87:0/2468572566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 msgr2=0x7fa85c109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:51.662 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 --2- 172.21.15.87:0/2468572566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0098a0 tx=0x7fa84c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 -- 172.21.15.87:0/2468572566 shutdown_connections 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 --2- 172.21.15.87:0/2468572566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c109a40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 -- 172.21.15.87:0/2468572566 >> 172.21.15.87:0/2468572566 conn(0x7fa85c101c40 msgr2=0x7fa85c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 -- 172.21.15.87:0/2468572566 shutdown_connections 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.120+0000 7fa8628cb700 1 -- 172.21.15.87:0/2468572566 wait complete. 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa8628cb700 1 Processor -- start 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa8628cb700 1 -- start start 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa8628cb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa8628cb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1063a0 con 0x7fa85c109620 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa85bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa85bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34586/0 (socket says 172.21.15.87:34586) 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa85bfff700 1 -- 172.21.15.87:0/1680602206 learned_addr learned my addr 172.21.15.87:0/1680602206 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:51.663 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.121+0000 7fa85bfff700 1 -- 172.21.15.87:0/1680602206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c19bc30 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa85bfff700 1 --2- 172.21.15.87:0/1680602206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c003db0 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c011630 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c011950 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c19bf20 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c19ebb0 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa84c011af0 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.122+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa84c01c450 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.123+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c063af0 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.125+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa84c032020 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.192+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa85c02de60 con 0x7fa85c109620 2024-06-06T07:35:51.664 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.195+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fa84c004260 con 0x7fa85c109620 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa859ffb700 1 -- 172.21.15.87:0/1680602206 <== mon.0 v2:172.21.15.87:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa84c01b450 con 0x7fa85c109620 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 msgr2=0x7fa85c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa8628cb700 1 --2- 172.21.15.87:0/1680602206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c004750 comp rx=0 tx=0).stop 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 shutdown_connections 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa8628cb700 1 --2- 172.21.15.87:0/1680602206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa85c109620 0x7fa85c19b6f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.196+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 >> 172.21.15.87:0/1680602206 conn(0x7fa85c101c40 msgr2=0x7fa85c18fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.197+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 shutdown_connections 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:51.197+0000 7fa8628cb700 1 -- 172.21.15.87:0/1680602206 wait complete. 2024-06-06T07:35:51.665 INFO:teuthology.orchestra.run.smithi087.stdout:Generating new minimal ceph.conf... 2024-06-06T07:35:52.729 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.074+0000 7fd29c4c5700 1 Processor -- start 2024-06-06T07:35:52.729 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29c4c5700 1 -- start start 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29c4c5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29c4c5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941089e0 con 0x7fd294107ff0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29a261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29a261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34602/0 (socket says 172.21.15.87:34602) 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29a261700 1 -- 172.21.15.87:0/2020782486 learned_addr learned my addr 172.21.15.87:0/2020782486 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.075+0000 7fd29a261700 1 -- 172.21.15.87:0/2020782486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941091f0 con 0x7fd294107ff0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29a261700 1 --2- 172.21.15.87:0/2020782486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd2840098a0 tx=0x7fd284009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44f21c7fd2cd823 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29925f700 1 -- 172.21.15.87:0/2020782486 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd284004160 con 0x7fd294107ff0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29925f700 1 -- 172.21.15.87:0/2020782486 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd284004300 con 0x7fd294107ff0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29925f700 1 -- 172.21.15.87:0/2020782486 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd284011b50 con 0x7fd294107ff0 2024-06-06T07:35:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 -- 172.21.15.87:0/2020782486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 msgr2=0x7fd294108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 --2- 172.21.15.87:0/2020782486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd2840098a0 tx=0x7fd284009bb0 comp rx=0 tx=0).stop 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 -- 172.21.15.87:0/2020782486 shutdown_connections 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 --2- 172.21.15.87:0/2020782486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd294108410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 -- 172.21.15.87:0/2020782486 >> 172.21.15.87:0/2020782486 conn(0x7fd294103590 msgr2=0x7fd2941059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 -- 172.21.15.87:0/2020782486 shutdown_connections 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.076+0000 7fd29c4c5700 1 -- 172.21.15.87:0/2020782486 wait complete. 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29c4c5700 1 Processor -- start 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29c4c5700 1 -- start start 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29c4c5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29c4c5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd29410c420 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29a261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29a261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34614/0 (socket says 172.21.15.87:34614) 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29a261700 1 -- 172.21.15.87:0/1324301452 learned_addr learned my addr 172.21.15.87:0/1324301452 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29a261700 1 -- 172.21.15.87:0/1324301452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29419c4d0 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.077+0000 7fd29a261700 1 --2- 172.21.15.87:0/1324301452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd284011850 tx=0x7fd284011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd284003af0 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd284003c90 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2840236b0 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29419c760 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29419f450 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd284022030 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.078+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd284011e00 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.079+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29410c5a0 con 0x7fd294107ff0 2024-06-06T07:35:52.732 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.081+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd28401a980 con 0x7fd294107ff0 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.147+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd294063af0 con 0x7fd294107ff0 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.147+0000 7fd28b7fe700 1 -- 172.21.15.87:0/1324301452 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fd28401a430 con 0x7fd294107ff0 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 msgr2=0x7fd29419bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 --2- 172.21.15.87:0/1324301452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd284011850 tx=0x7fd284011c10 comp rx=0 tx=0).stop 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 shutdown_connections 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 --2- 172.21.15.87:0/1324301452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd294107ff0 0x7fd29419bf90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 >> 172.21.15.87:0/1324301452 conn(0x7fd294103590 msgr2=0x7fd294104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 shutdown_connections 2024-06-06T07:35:52.733 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:52.148+0000 7fd29c4c5700 1 -- 172.21.15.87:0/1324301452 wait complete. 2024-06-06T07:35:52.734 INFO:teuthology.orchestra.run.smithi087.stdout:Restarting the monitor... 2024-06-06T07:35:52.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:52 smithi087 systemd[1]: Stopping Ceph mon.a for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:35:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[31902]: 2024-06-06T07:35:52.999+0000 7f513100c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:35:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[31902]: 2024-06-06T07:35:52.999+0000 7f513100c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-06T07:35:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 podman[32151]: 2024-06-06 07:35:53.12344629 +0000 UTC m=+0.230897164 container died 219a9d49d2c79d5602ab299f27b1eae918f80b10e607c3967caf271ca408d3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T07:35:53.709 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 podman[32151]: 2024-06-06 07:35:53.474951409 +0000 UTC m=+0.582402281 container cleanup 219a9d49d2c79d5602ab299f27b1eae918f80b10e607c3967caf271ca408d3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:35:53.709 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 bash[32151]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a 2024-06-06T07:35:54.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 podman[32163]: 2024-06-06 07:35:53.708976913 +0000 UTC m=+0.585168415 container remove 219a9d49d2c79d5602ab299f27b1eae918f80b10e607c3967caf271ca408d3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T07:35:54.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 systemd[1]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T07:35:54.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 systemd[1]: Stopped Ceph mon.a for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:35:54.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:53 smithi087 systemd[1]: Starting Ceph mon.a for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:35:54.302 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 podman[32257]: 2024-06-06 07:35:54.051338036 +0000 UTC m=+0.109073700 container create d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 podman[32257]: 2024-06-06 07:35:53.959084696 +0000 UTC m=+0.016820379 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 podman[32257]: 2024-06-06 07:35:54.270214941 +0000 UTC m=+0.327950606 container init d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 podman[32257]: 2024-06-06 07:35:54.273466091 +0000 UTC m=+0.331201760 container start d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: pidfile_write: ignore empty --pid-file 2024-06-06T07:35:54.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: load: jerasure load: lrc 2024-06-06T07:35:54.358 INFO:teuthology.orchestra.run.smithi087.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-06T07:35:54.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: RocksDB version: 7.9.2 2024-06-06T07:35:54.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Git sha 0 2024-06-06T07:35:54.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T07:35:54.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: DB SUMMARY 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: DB Session ID: RWR3UQ5F4YKKKZDOZOLE 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: CURRENT file: CURRENT 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105308 ; 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.error_if_exists: 0 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.create_if_missing: 0 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.paranoid_checks: 1 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T07:35:54.562 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.env: 0x556efcbc1c60 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.info_log: 0x556eff08bb20 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.statistics: (nil) 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.use_fsync: 0 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_log_file_size: 0 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T07:35:54.563 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_fallocate: 1 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.use_direct_reads: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.db_log_dir: 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.wal_dir: 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T07:35:54.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.write_buffer_manager: 0x556eff05c1e0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.unordered_write: 0 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T07:35:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.row_cache: None 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.wal_filter: None 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.two_write_queues: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.wal_compression: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.atomic_flush: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T07:35:54.566 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.log_readahead_size: 0 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_background_jobs: 2 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_background_compactions: -1 2024-06-06T07:35:54.567 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_subcompactions: 1 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_open_files: -1 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T07:35:54.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_background_flushes: -1 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Compression algorithms supported: 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kZSTD supported: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kXpressCompression supported: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kZlibCompression supported: 1 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kSnappyCompression supported: 1 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kLZ4Compression supported: 1 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: kBZip2Compression supported: 0 2024-06-06T07:35:54.570 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.merge_operator: 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_filter: None 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556eff08bc40) 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks: 1 2024-06-06T07:35:54.571 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_top_level_index_and_filter: 1 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_type: 0 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_index_type: 0 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_shortening: 1 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: checksum: 4 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: no_block_cache: 0 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache: 0x556eff08d090 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_name: BinnedLRUCache 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_options: 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: capacity : 536870912 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_shard_bits : 4 2024-06-06T07:35:54.572 INFO:journalctl@ceph.mon.a.smithi087.stdout: strict_capacity_limit : 0 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: high_pri_pool_ratio: 0.000 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_compressed: (nil) 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: persistent_cache: (nil) 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size: 4096 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size_deviation: 10 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_restart_interval: 16 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_block_restart_interval: 1 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: metadata_block_size: 4096 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: partition_filters: 0 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: use_delta_encoding: 1 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: filter_policy: bloomfilter 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: whole_key_filtering: 1 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: verify_compression: 0 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: read_amp_bytes_per_bit: 0 2024-06-06T07:35:54.573 INFO:journalctl@ceph.mon.a.smithi087.stdout: format_version: 5 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_index_compression: 1 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_align: 0 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: max_auto_readahead_size: 262144 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: prepopulate_block_cache: 0 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: initial_auto_readahead_size: 8192 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression: NoCompression 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T07:35:54.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.num_levels: 7 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T07:35:54.575 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T07:35:54.576 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T07:35:54.577 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T07:35:54.578 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.table_properties_collectors: 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.inplace_update_support: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.bloom_locality: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.max_successive_merges: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T07:35:54.579 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.ttl: 2592000 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enable_blob_files: false 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.min_blob_size: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-06T07:35:54.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2ff5031c-d041-40cd-88d4-fcc2797d139e 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659354304842, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659354306203, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100341, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98261, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14308, "raw_average_key_size": 49, "raw_value_size": 90515, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717659354, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2ff5031c-d041-40cd-88d4-fcc2797d139e", "db_session_id": "RWR3UQ5F4YKKKZDOZOLE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659354306285, "job": 1, "event": "recovery_finished"} 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556eff17c000 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: DB pointer 0x556eff166000 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T07:35:54.581 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** DB Stats ** 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: L0 2/0 99.82 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Sum 2/0 99.82 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T07:35:54.582 INFO:journalctl@ceph.mon.a.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T07:35:54.583 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative compaction: 0.00 GB write, 28.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval compaction: 0.00 GB write, 28.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache BinnedLRUCache@0x556eff08d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: starting mon.a rank 0 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???) e1 preinit fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).mds e1 new map 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).mds e1 print_map 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: e1 2024-06-06T07:35:54.584 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout: legacy client fscid: -1 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout: No filesystems configured 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 bash[32257]: d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 systemd[1]: Started Ceph mon.a for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: monmap epoch 1 2024-06-06T07:35:54.585 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: last_changed 2024-06-06T07:35:46.713783+0000 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: min_mon_release 19 (squid) 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: election_strategy: 1 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: fsmap 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T07:35:54.586 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:54 smithi087 ceph-mon[32276]: mgrmap e1: no daemons active 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.783+0000 7f42e3301700 1 Processor -- start 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.783+0000 7f42e3301700 1 -- start start 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.783+0000 7f42e3301700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.783+0000 7f42e3301700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc108a10 con 0x7f42dc108020 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.783+0000 7f42e109d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42e109d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34616/0 (socket says 172.21.15.87:34616) 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42e109d700 1 -- 172.21.15.87:0/3976727027 learned_addr learned my addr 172.21.15.87:0/3976727027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:55.324 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42e109d700 1 -- 172.21.15.87:0/3976727027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc109220 con 0x7f42dc108020 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42e109d700 1 --2- 172.21.15.87:0/3976727027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42d80098a0 tx=0x7f42d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e307d4cea24fca7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42d3fff700 1 -- 172.21.15.87:0/3976727027 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8004160 con 0x7f42dc108020 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42d3fff700 1 -- 172.21.15.87:0/3976727027 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42d8004300 con 0x7f42dc108020 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.784+0000 7f42d3fff700 1 -- 172.21.15.87:0/3976727027 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8011b50 con 0x7f42dc108020 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 -- 172.21.15.87:0/3976727027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 msgr2=0x7f42dc108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 --2- 172.21.15.87:0/3976727027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42d80098a0 tx=0x7f42d8009bb0 comp rx=0 tx=0).stop 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 -- 172.21.15.87:0/3976727027 shutdown_connections 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 --2- 172.21.15.87:0/3976727027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc108440 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 -- 172.21.15.87:0/3976727027 >> 172.21.15.87:0/3976727027 conn(0x7f42dc1035e0 msgr2=0x7f42dc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 -- 172.21.15.87:0/3976727027 shutdown_connections 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.785+0000 7f42e3301700 1 -- 172.21.15.87:0/3976727027 wait complete. 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e3301700 1 Processor -- start 2024-06-06T07:35:55.325 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e3301700 1 -- start start 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e3301700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e3301700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc10c450 con 0x7f42dc108020 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e109d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e109d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34630/0 (socket says 172.21.15.87:34630) 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e109d700 1 -- 172.21.15.87:0/1016165688 learned_addr learned my addr 172.21.15.87:0/1016165688 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e109d700 1 -- 172.21.15.87:0/1016165688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc07e930 con 0x7f42dc108020 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e109d700 1 --2- 172.21.15.87:0/1016165688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011850 tx=0x7f42d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d80039e0 con 0x7f42dc108020 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42d8011e50 con 0x7f42dc108020 2024-06-06T07:35:55.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d80245a0 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.786+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc07cb20 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.787+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc07d060 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.787+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42d8022020 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.787+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42d802dab0 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.787+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc10c5d0 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.790+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f42d8043dd0 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.855+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f42dc07d800 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.857+0000 7f42d27fc700 1 -- 172.21.15.87:0/1016165688 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f42d8032030 con 0x7f42dc108020 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 msgr2=0x7f42dc07e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 --2- 172.21.15.87:0/1016165688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011850 tx=0x7f42d8011c10 comp rx=0 tx=0).stop 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 shutdown_connections 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 --2- 172.21.15.87:0/1016165688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42dc108020 0x7f42dc07e3f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:55.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 >> 172.21.15.87:0/1016165688 conn(0x7f42dc1035e0 msgr2=0x7f42dc1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:55.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 shutdown_connections 2024-06-06T07:35:55.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:54.858+0000 7f42e3301700 1 -- 172.21.15.87:0/1016165688 wait complete. 2024-06-06T07:35:55.328 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-06T07:35:55.342 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:35:55.342 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mgr... 2024-06-06T07:35:55.342 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-06T07:35:55.343 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-06T07:35:55.584 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T07:35:55.585 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-06T07:35:55.733 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968.target.wants/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@.service. 2024-06-06T07:35:55.858 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:55 smithi087 systemd[1]: Starting Ceph mgr.x for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:35:56.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:55 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1016165688' entity='client.admin' 2024-06-06T07:35:56.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:55 smithi087 podman[32488]: 2024-06-06 07:35:55.952831996 +0000 UTC m=+0.099644917 container create 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T07:35:56.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:55 smithi087 podman[32488]: 2024-06-06 07:35:55.870597786 +0000 UTC m=+0.017410721 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T07:35:56.329 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T07:35:56.329 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-06-06T07:35:56.338 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T07:35:56.339 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:35:56.339 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-06-06T07:35:56.339 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T07:35:56.347 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T07:35:56.347 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-06-06T07:35:56.354 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T07:35:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:35:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-06-06T07:35:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-06T07:35:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr to start... 2024-06-06T07:35:56.355 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr... 2024-06-06T07:35:56.498 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 podman[32488]: 2024-06-06 07:35:56.238518876 +0000 UTC m=+0.385331807 container init 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-06T07:35:56.498 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 podman[32488]: 2024-06-06 07:35:56.241258351 +0000 UTC m=+0.388071280 container start 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T07:35:56.498 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 bash[32488]: 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 2024-06-06T07:35:56.498 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 systemd[1]: Started Ceph mgr.x for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:35:56.498 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:56.401+0000 7f288e88e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:35:56.775 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:56.588+0000 7f288e88e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:35:56.775 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:56.694+0000 7f288e88e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:35:57.104 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:35:56 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1240993899' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T07:35:57.104 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:57.017+0000 7f288e88e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:35:57.104 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:57.103+0000 7f288e88e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T07:35:57.458 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:35:57.459 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T07:35:57.460 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T07:35:57.460 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T07:35:57.467 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T07:35:57.468 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T07:35:49.600107+0000", 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:35:57.469 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.809+0000 7f6614efa700 1 Processor -- start 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.809+0000 7f6614efa700 1 -- start start 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.809+0000 7f6614efa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.809+0000 7f6614efa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6610108a60 con 0x7f6610108070 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34676/0 (socket says 172.21.15.87:34676) 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660e59c700 1 -- 172.21.15.87:0/2571393684 learned_addr learned my addr 172.21.15.87:0/2571393684 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660e59c700 1 -- 172.21.15.87:0/2571393684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6610109220 con 0x7f6610108070 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660e59c700 1 --2- 172.21.15.87:0/2571393684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66000098a0 tx=0x7f6600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e771674b6ea73ddd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660d59a700 1 -- 172.21.15.87:0/2571393684 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6600004160 con 0x7f6610108070 2024-06-06T07:35:57.470 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660d59a700 1 -- 172.21.15.87:0/2571393684 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6600004300 con 0x7f6610108070 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.810+0000 7f660d59a700 1 -- 172.21.15.87:0/2571393684 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6600011b50 con 0x7f6610108070 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 -- 172.21.15.87:0/2571393684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 msgr2=0x7f6610108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 --2- 172.21.15.87:0/2571393684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66000098a0 tx=0x7f6600009bb0 comp rx=0 tx=0).stop 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 -- 172.21.15.87:0/2571393684 shutdown_connections 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 --2- 172.21.15.87:0/2571393684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f6610108490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 -- 172.21.15.87:0/2571393684 >> 172.21.15.87:0/2571393684 conn(0x7f6610103610 msgr2=0x7f6610105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 -- 172.21.15.87:0/2571393684 shutdown_connections 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.811+0000 7f6614efa700 1 -- 172.21.15.87:0/2571393684 wait complete. 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f6614efa700 1 Processor -- start 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f6614efa700 1 -- start start 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f6614efa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f6614efa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f661010c450 con 0x7f6610108070 2024-06-06T07:35:57.471 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f660e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f660e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34682/0 (socket says 172.21.15.87:34682) 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.812+0000 7f660e59c700 1 -- 172.21.15.87:0/1240993899 learned_addr learned my addr 172.21.15.87:0/1240993899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f660e59c700 1 -- 172.21.15.87:0/1240993899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661019c510 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f660e59c700 1 --2- 172.21.15.87:0/1240993899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6600011850 tx=0x7f6600011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66000039e0 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6600011e50 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661019c7a0 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66000245a0 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661019f490 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6600022020 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.813+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f660002db00 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.814+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661010c5d0 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.816+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6600043dd0 con 0x7f6610108070 2024-06-06T07:35:57.472 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.889+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f661019f9c0 con 0x7f6610108070 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.890+0000 7f65ff7fe700 1 -- 172.21.15.87:0/1240993899 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6600032030 con 0x7f6610108070 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 msgr2=0x7f661019bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 --2- 172.21.15.87:0/1240993899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6600011850 tx=0x7f6600011c10 comp rx=0 tx=0).stop 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 shutdown_connections 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 --2- 172.21.15.87:0/1240993899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6610108070 0x7f661019bfd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 >> 172.21.15.87:0/1240993899 conn(0x7f6610103610 msgr2=0x7f6610104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 shutdown_connections 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:56.891+0000 7f6614efa700 1 -- 172.21.15.87:0/1240993899 wait complete. 2024-06-06T07:35:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (1/15)... 2024-06-06T07:35:58.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:58.294+0000 7f288e88e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:35:58.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:58.468+0000 7f288e88e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:35:58.924 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:58.637+0000 7f288e88e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:35:58.924 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:58.923+0000 7f288e88e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:35:59.218 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:59.016+0000 7f288e88e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:35:59.218 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:59.217+0000 7f288e88e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:35:59.586 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:35:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:35:59.409+0000 7f288e88e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:36:00.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1425927082' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T07:36:00.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:00.026+0000 7f288e88e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:36:00.756 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:00.455+0000 7f288e88e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:36:00.756 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:00.553+0000 7f288e88e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:36:00.756 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:00.754+0000 7f288e88e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:36:01.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.273+0000 7f288e88e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:36:01.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.366+0000 7f288e88e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:36:01.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.608+0000 7f288e88e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:36:01.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.740+0000 7f288e88e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:36:01.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.834+0000 7f288e88e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:36:02.249 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:01.950+0000 7f288e88e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:36:02.249 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:02 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:02.247+0000 7f288e88e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:36:03.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:02 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:02.853+0000 7f288e88e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:36:03.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:03.072+0000 7f288e88e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:36:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: Activating manager daemon x 2024-06-06T07:36:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: mgrmap e2: x(active, starting, since 0.00308765s) 2024-06-06T07:36:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: Manager daemon x is now available 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' 2024-06-06T07:36:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:03 smithi087 ceph-mon[32276]: from='mgr.14100 172.21.15.87:0/3280264169' entity='mgr.x' 2024-06-06T07:36:05.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:05 smithi087 ceph-mon[32276]: mgrmap e3: x(active, since 1.00744s) 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T07:36:06.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T07:36:06.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:06.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T07:36:06.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T07:36:06.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T07:36:06.060 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.061 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T07:35:49.600107+0000", 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:36:06.062 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.907+0000 7f65c9245700 1 Processor -- start 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c9245700 1 -- start start 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c9245700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c9245700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4108ed0 con 0x7f65c41084e0 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38466/0 (socket says 172.21.15.87:38466) 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c2d9d700 1 -- 172.21.15.87:0/443263434 learned_addr learned my addr 172.21.15.87:0/443263434 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c2d9d700 1 -- 172.21.15.87:0/443263434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41070f0 con 0x7f65c41084e0 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.908+0000 7f65c2d9d700 1 --2- 172.21.15.87:0/443263434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65b40098a0 tx=0x7f65b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9b5470bf1db4be server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c1d9b700 1 -- 172.21.15.87:0/443263434 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b4004160 con 0x7f65c41084e0 2024-06-06T07:36:06.063 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c1d9b700 1 -- 172.21.15.87:0/443263434 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65b4004300 con 0x7f65c41084e0 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c1d9b700 1 -- 172.21.15.87:0/443263434 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b4011b50 con 0x7f65c41084e0 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 -- 172.21.15.87:0/443263434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 msgr2=0x7f65c4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 --2- 172.21.15.87:0/443263434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65b40098a0 tx=0x7f65b4009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 -- 172.21.15.87:0/443263434 shutdown_connections 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 --2- 172.21.15.87:0/443263434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4108900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 -- 172.21.15.87:0/443263434 >> 172.21.15.87:0/443263434 conn(0x7f65c4101c40 msgr2=0x7f65c4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 -- 172.21.15.87:0/443263434 shutdown_connections 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.909+0000 7f65c9245700 1 -- 172.21.15.87:0/443263434 wait complete. 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.910+0000 7f65c9245700 1 Processor -- start 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c9245700 1 -- start start 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c9245700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c9245700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4105fe0 con 0x7f65c41084e0 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:06.064 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38468/0 (socket says 172.21.15.87:38468) 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c2d9d700 1 -- 172.21.15.87:0/1425927082 learned_addr learned my addr 172.21.15.87:0/1425927082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c2d9d700 1 -- 172.21.15.87:0/1425927082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c419a2a0 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c2d9d700 1 --2- 172.21.15.87:0/1425927082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65b4011670 tx=0x7f65b4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b40039c0 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65b4011e50 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b401a5c0 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.911+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c419a530 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.912+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c419d1c0 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.912+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f65b401ac50 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.912+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65b4019700 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.912+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c4063af0 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.915+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f65b4003b60 con 0x7f65c41084e0 2024-06-06T07:36:06.065 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.988+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f65c41932c0 con 0x7f65c41084e0 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.989+0000 7f65abfff700 1 -- 172.21.15.87:0/1425927082 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f65b4019900 con 0x7f65c41084e0 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 msgr2=0x7f65c4199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 --2- 172.21.15.87:0/1425927082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65b4011670 tx=0x7f65b4011b60 comp rx=0 tx=0).stop 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 shutdown_connections 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 --2- 172.21.15.87:0/1425927082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c41084e0 0x7f65c4199d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 >> 172.21.15.87:0/1425927082 conn(0x7f65c4101c40 msgr2=0x7f65c4190380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 shutdown_connections 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:35:59.990+0000 7f65c9245700 1 -- 172.21.15.87:0/1425927082 wait complete. 2024-06-06T07:36:06.066 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (2/15)... 2024-06-06T07:36:06.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:06 smithi087 ceph-mon[32276]: mgrmap e4: x(active, since 2s) 2024-06-06T07:36:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2260697368' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T07:36:11.871 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:36:11.871 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:11.871 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:36:11.871 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T07:36:11.872 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T07:36:11.873 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.874 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T07:35:49.600107+0000", 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T07:36:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef61d6a700 1 Processor -- start 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef61d6a700 1 -- start start 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef61d6a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef61d6a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c07c6f0 con 0x7fef5c106920 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef5b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef5b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33184/0 (socket says 172.21.15.87:33184) 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.038+0000 7fef5b7fe700 1 -- 172.21.15.87:0/1576991889 learned_addr learned my addr 172.21.15.87:0/1576991889 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.039+0000 7fef5b7fe700 1 -- 172.21.15.87:0/1576991889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c107280 con 0x7fef5c106920 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.039+0000 7fef5b7fe700 1 --2- 172.21.15.87:0/1576991889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0098a0 tx=0x7fef4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362f85b2fb27f99f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.039+0000 7fef5a7fc700 1 -- 172.21.15.87:0/1576991889 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c004160 con 0x7fef5c106920 2024-06-06T07:36:11.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.039+0000 7fef5a7fc700 1 -- 172.21.15.87:0/1576991889 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef4c004300 con 0x7fef5c106920 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.039+0000 7fef5a7fc700 1 -- 172.21.15.87:0/1576991889 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c011b50 con 0x7fef5c106920 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 -- 172.21.15.87:0/1576991889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 msgr2=0x7fef5c106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 --2- 172.21.15.87:0/1576991889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0098a0 tx=0x7fef4c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 -- 172.21.15.87:0/1576991889 shutdown_connections 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 --2- 172.21.15.87:0/1576991889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c106d40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 -- 172.21.15.87:0/1576991889 >> 172.21.15.87:0/1576991889 conn(0x7fef5c102520 msgr2=0x7fef5c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 -- 172.21.15.87:0/1576991889 shutdown_connections 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.040+0000 7fef61d6a700 1 -- 172.21.15.87:0/1576991889 wait complete. 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef61d6a700 1 Processor -- start 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef61d6a700 1 -- start start 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef61d6a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef61d6a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c108230 con 0x7fef5c106920 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef5b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:11.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef5b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33194/0 (socket says 172.21.15.87:33194) 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef5b7fe700 1 -- 172.21.15.87:0/2260697368 learned_addr learned my addr 172.21.15.87:0/2260697368 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef5b7fe700 1 -- 172.21.15.87:0/2260697368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c19e610 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.041+0000 7fef5b7fe700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef4c011670 tx=0x7fef4c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c0039c0 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef4c011e50 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c19e8a0 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c01a5c0 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1a1590 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.042+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fef4c01a870 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.043+0000 7fef58ff9700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 0x7fef4403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.043+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef4c04d3c0 con 0x7fef5c106920 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.043+0000 7fef5affd700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 0x7fef4403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:11.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.043+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c063af0 con 0x7fef5c106920 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.043+0000 7fef5affd700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 0x7fef4403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.048+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef4c019c70 con 0x7fef5c106920 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.348+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fef5c02de90 con 0x7fef5c106920 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.349+0000 7fef58ff9700 1 -- 172.21.15.87:0/2260697368 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fef4c02c9c0 con 0x7fef5c106920 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 msgr2=0x7fef4403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 0x7fef4403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 msgr2=0x7fef5c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef4c011670 tx=0x7fef4c011b60 comp rx=0 tx=0).stop 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 shutdown_connections 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fef44039000 0x7fef4403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 --2- 172.21.15.87:0/2260697368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef5c106920 0x7fef5c19e0d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 >> 172.21.15.87:0/2260697368 conn(0x7fef5c102520 msgr2=0x7fef5c194d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 shutdown_connections 2024-06-06T07:36:11.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:11.351+0000 7fef61d6a700 1 -- 172.21.15.87:0/2260697368 wait complete. 2024-06-06T07:36:11.880 INFO:teuthology.orchestra.run.smithi087.stdout:mgr is available 2024-06-06T07:36:12.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1545995939' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T07:36:12.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:36:12.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T07:36:12.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:12.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T07:36:12.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.259+0000 7fd7b1f2c700 1 Processor -- start 2024-06-06T07:36:12.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.259+0000 7fd7b1f2c700 1 -- start start 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.259+0000 7fd7b1f2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.259+0000 7fd7b1f2c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac108a60 con 0x7fd7ac108070 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.259+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33206/0 (socket says 172.21.15.87:33206) 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7ab7fe700 1 -- 172.21.15.87:0/2415557335 learned_addr learned my addr 172.21.15.87:0/2415557335 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7ab7fe700 1 -- 172.21.15.87:0/2415557335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac109220 con 0x7fd7ac108070 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7ab7fe700 1 --2- 172.21.15.87:0/2415557335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098a0 tx=0x7fd79c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c883fd8800e73f7b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7aa7fc700 1 -- 172.21.15.87:0/2415557335 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd79c004160 con 0x7fd7ac108070 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7aa7fc700 1 -- 172.21.15.87:0/2415557335 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd79c004300 con 0x7fd7ac108070 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.260+0000 7fd7aa7fc700 1 -- 172.21.15.87:0/2415557335 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd79c011b50 con 0x7fd7ac108070 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/2415557335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 msgr2=0x7fd7ac108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/2415557335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098a0 tx=0x7fd79c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:12.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/2415557335 shutdown_connections 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/2415557335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac108490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/2415557335 >> 172.21.15.87:0/2415557335 conn(0x7fd7ac103610 msgr2=0x7fd7ac105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/2415557335 shutdown_connections 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.261+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/2415557335 wait complete. 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7b1f2c700 1 Processor -- start 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7b1f2c700 1 -- start start 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7b1f2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7b1f2c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10c450 con 0x7fd7ac108070 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33218/0 (socket says 172.21.15.87:33218) 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7ab7fe700 1 -- 172.21.15.87:0/1545995939 learned_addr learned my addr 172.21.15.87:0/1545995939 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.262+0000 7fd7ab7fe700 1 -- 172.21.15.87:0/1545995939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac07e930 con 0x7fd7ac108070 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7ab7fe700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd79c011850 tx=0x7fd79c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:12.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd79c003af0 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd79c003c90 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac07cb20 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd79c0236b0 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.263+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac07d060 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd79c022030 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7a8ff9700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 0x7fd79403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd79c011e00 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac063af0 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7aaffd700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 0x7fd79403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.264+0000 7fd7aaffd700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 0x7fd79403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0006fd0 tx=0x7fd7a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.269+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd79c020220 con 0x7fd7ac108070 2024-06-06T07:36:12.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.463+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd7ac07d2e0 con 0x7fd7ac108070 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.464+0000 7fd7a8ff9700 1 -- 172.21.15.87:0/1545995939 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fd79c01a980 con 0x7fd7ac108070 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 msgr2=0x7fd79403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 0x7fd79403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0006fd0 tx=0x7fd7a0006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 msgr2=0x7fd7ac07e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd79c011850 tx=0x7fd79c011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 shutdown_connections 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd794039050 0x7fd79403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 --2- 172.21.15.87:0/1545995939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7ac108070 0x7fd7ac07e3f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 >> 172.21.15.87:0/1545995939 conn(0x7fd7ac103610 msgr2=0x7fd7ac1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 shutdown_connections 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:12.466+0000 7fd7b1f2c700 1 -- 172.21.15.87:0/1545995939 wait complete. 2024-06-06T07:36:12.971 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling cephadm module... 2024-06-06T07:36:13.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:13 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3092186899' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-06T07:36:13.979 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setuser ceph since I am not root 2024-06-06T07:36:13.979 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setgroup ceph since I am not root 2024-06-06T07:36:13.979 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:13.849+0000 7f4128d62200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:36:14.238 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:14.035+0000 7f4128d62200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:36:14.238 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:14.137+0000 7f4128d62200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:36:14.245 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.367+0000 7f91e176c700 1 Processor -- start 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.367+0000 7f91e176c700 1 -- start start 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91e176c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91e176c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc07c730 con 0x7f91dc108a90 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33220/0 (socket says 172.21.15.87:33220) 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91daffd700 1 -- 172.21.15.87:0/1033904535 learned_addr learned my addr 172.21.15.87:0/1033904535 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91daffd700 1 -- 172.21.15.87:0/1033904535 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc10b3c0 con 0x7f91dc108a90 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91daffd700 1 --2- 172.21.15.87:0/1033904535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098a0 tx=0x7f91cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f50de676f31af61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91da7fc700 1 -- 172.21.15.87:0/1033904535 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc004160 con 0x7f91dc108a90 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91da7fc700 1 -- 172.21.15.87:0/1033904535 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91cc004300 con 0x7f91dc108a90 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.368+0000 7f91da7fc700 1 -- 172.21.15.87:0/1033904535 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc011b50 con 0x7f91dc108a90 2024-06-06T07:36:14.246 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 -- 172.21.15.87:0/1033904535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 msgr2=0x7f91dc10ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 --2- 172.21.15.87:0/1033904535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098a0 tx=0x7f91cc009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 -- 172.21.15.87:0/1033904535 shutdown_connections 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 --2- 172.21.15.87:0/1033904535 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc10ae80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 -- 172.21.15.87:0/1033904535 >> 172.21.15.87:0/1033904535 conn(0x7f91dc1024c0 msgr2=0x7f91dc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 -- 172.21.15.87:0/1033904535 shutdown_connections 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.369+0000 7f91e176c700 1 -- 172.21.15.87:0/1033904535 wait complete. 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91e176c700 1 Processor -- start 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91e176c700 1 -- start start 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91e176c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91e176c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10c310 con 0x7f91dc108a90 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33226/0 (socket says 172.21.15.87:33226) 2024-06-06T07:36:14.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91daffd700 1 -- 172.21.15.87:0/3092186899 learned_addr learned my addr 172.21.15.87:0/3092186899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91daffd700 1 -- 172.21.15.87:0/3092186899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc19c450 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91daffd700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009fb0 tx=0x7f91cc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.370+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc0038b0 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91cc011e50 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc019420 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc19c6e0 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc19f3d0 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f91cc01a430 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.371+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc063af0 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.372+0000 7f91d8ff9700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 0x7f91c803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.372+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91cc04d1c0 con 0x7f91dc108a90 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.372+0000 7f91d3fff700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 0x7f91c803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:14.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.372+0000 7f91d3fff700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 0x7f91c803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.377+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91cc003b00 con 0x7f91dc108a90 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.627+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f91dc19f5a0 con 0x7f91dc108a90 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.679+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f91dc19f5a0 con 0x7f91dc108a90 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.680+0000 7f91d8ff9700 1 -- 172.21.15.87:0/3092186899 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f91cc0196d0 con 0x7f91dc108a90 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.681+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 msgr2=0x7f91c803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.681+0000 7f91e176c700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 0x7f91c803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.681+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 msgr2=0x7f91dc19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.681+0000 7f91e176c700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009fb0 tx=0x7f91cc011bf0 comp rx=0 tx=0).stop 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 shutdown_connections 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f91c8039000 0x7f91c803b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 --2- 172.21.15.87:0/3092186899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91dc108a90 0x7f91dc19bf10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 >> 172.21.15.87:0/3092186899 conn(0x7f91dc1024c0 msgr2=0x7f91dc103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:14.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 shutdown_connections 2024-06-06T07:36:14.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:13.682+0000 7f91e176c700 1 -- 172.21.15.87:0/3092186899 wait complete. 2024-06-06T07:36:14.491 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:14.455+0000 7f4128d62200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:36:14.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:14.545+0000 7f4128d62200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:36:14.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3092186899' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-06T07:36:14.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:14 smithi087 ceph-mon[32276]: mgrmap e5: x(active, since 10s) 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T07:36:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fc10a9700 1 Processor -- start 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fc10a9700 1 -- start start 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fc10a9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fc10a9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc07c6d0 con 0x7f9fbc108b50 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33248/0 (socket says 172.21.15.87:33248) 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fbad9d700 1 -- 172.21.15.87:0/384888551 learned_addr learned my addr 172.21.15.87:0/384888551 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fbad9d700 1 -- 172.21.15.87:0/384888551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc10b480 con 0x7f9fbc108b50 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.673+0000 7f9fbad9d700 1 --2- 172.21.15.87:0/384888551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098a0 tx=0x7f9fac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b90d06df285dde8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fb9d9b700 1 -- 172.21.15.87:0/384888551 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac004160 con 0x7f9fbc108b50 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fb9d9b700 1 -- 172.21.15.87:0/384888551 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fac004300 con 0x7f9fbc108b50 2024-06-06T07:36:15.486 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fb9d9b700 1 -- 172.21.15.87:0/384888551 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac011b50 con 0x7f9fbc108b50 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 -- 172.21.15.87:0/384888551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 msgr2=0x7f9fbc10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/384888551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098a0 tx=0x7f9fac009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 -- 172.21.15.87:0/384888551 shutdown_connections 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/384888551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc10af40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 -- 172.21.15.87:0/384888551 >> 172.21.15.87:0/384888551 conn(0x7f9fbc102520 msgr2=0x7f9fbc104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 -- 172.21.15.87:0/384888551 shutdown_connections 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.674+0000 7f9fc10a9700 1 -- 172.21.15.87:0/384888551 wait complete. 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.675+0000 7f9fc10a9700 1 Processor -- start 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.675+0000 7f9fc10a9700 1 -- start start 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.675+0000 7f9fc10a9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.675+0000 7f9fc10a9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc10c370 con 0x7f9fbc108b50 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33264/0 (socket says 172.21.15.87:33264) 2024-06-06T07:36:15.487 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fbad9d700 1 -- 172.21.15.87:0/3369619771 learned_addr learned my addr 172.21.15.87:0/3369619771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fbad9d700 1 -- 172.21.15.87:0/3369619771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc108910 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fbad9d700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011670 tx=0x7f9fac011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac0039c0 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc106b00 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fac011e50 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fac01a5c0 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.676+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc107040 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.677+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc063af0 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.678+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9fac01a8c0 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.678+0000 7f9fa3fff700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 0x7f9fa403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.678+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fac04d710 con 0x7f9fbc108b50 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.678+0000 7f9fba59c700 1 -- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 msgr2=0x7f9fa403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:15.488 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.678+0000 7f9fba59c700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 0x7f9fa403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.683+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9fac016260 con 0x7f9fbc108b50 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.879+0000 7f9fba59c700 1 -- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 msgr2=0x7f9fa403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.879+0000 7f9fba59c700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 0x7f9fa403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.960+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9fbc1079b0 con 0x7f9fbc108b50 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.961+0000 7f9fa3fff700 1 -- 172.21.15.87:0/3369619771 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9fac019db0 con 0x7f9fbc108b50 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.964+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 msgr2=0x7f9fa403b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.964+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 0x7f9fa403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.964+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 msgr2=0x7f9fbc1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.964+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011670 tx=0x7f9fac011b60 comp rx=0 tx=0).stop 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.964+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 shutdown_connections 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.965+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7f9fa4039000 0x7f9fa403b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.965+0000 7f9fc10a9700 1 --2- 172.21.15.87:0/3369619771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fbc108b50 0x7f9fbc1083d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.965+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 >> 172.21.15.87:0/3369619771 conn(0x7f9fbc102520 msgr2=0x7f9fbc103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:15.489 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.965+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 shutdown_connections 2024-06-06T07:36:15.490 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:14.965+0000 7f9fc10a9700 1 -- 172.21.15.87:0/3369619771 wait complete. 2024-06-06T07:36:15.490 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-06-06T07:36:15.490 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 5... 2024-06-06T07:36:15.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3369619771' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T07:36:15.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:15 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:15.760+0000 7f4128d62200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:36:16.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:15 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:15.937+0000 7f4128d62200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:36:16.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:16.106+0000 7f4128d62200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:36:16.689 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:16.395+0000 7f4128d62200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:36:16.690 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:16.487+0000 7f4128d62200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:36:16.690 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:16.689+0000 7f4128d62200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:36:17.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:16.882+0000 7f4128d62200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:36:17.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:17.480+0000 7f4128d62200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:36:18.199 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:17.901+0000 7f4128d62200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:36:18.199 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:17.998+0000 7f4128d62200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:36:18.199 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:18.198+0000 7f4128d62200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:36:19.057 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:18.721+0000 7f4128d62200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:36:19.058 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:18.813+0000 7f4128d62200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:36:19.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:19.057+0000 7f4128d62200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:36:19.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:19.190+0000 7f4128d62200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:36:19.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:19.285+0000 7f4128d62200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:36:19.697 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:19.401+0000 7f4128d62200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:36:19.697 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:19.696+0000 7f4128d62200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:36:20.562 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:20.291+0000 7f4128d62200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:36:20.562 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:20.508+0000 7f4128d62200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:36:20.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: Active manager daemon x restarted 2024-06-06T07:36:20.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: Activating manager daemon x 2024-06-06T07:36:20.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: osdmap e2: 0 total, 0 up, 0 in 2024-06-06T07:36:20.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: mgrmap e6: x(active, starting, since 0.00333119s) 2024-06-06T07:36:20.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: Manager daemon x is now available 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:36:20.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:20 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:36:21.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:21 smithi087 ceph-mon[32276]: Found migration_current of "None". Setting to last migration. 2024-06-06T07:36:21.809 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:21 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:21.809 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:21 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:21.809 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:21 smithi087 ceph-mon[32276]: mgrmap e7: x(active, since 1.00604s) 2024-06-06T07:36:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-06T07:36:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T07:36:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.880+0000 7fd892532700 1 Processor -- start 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.880+0000 7fd892532700 1 -- start start 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd892532700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd892532700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c07be20 con 0x7fd88c1068e0 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33274/0 (socket says 172.21.15.87:33274) 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88bfff700 1 -- 172.21.15.87:0/3353862244 learned_addr learned my addr 172.21.15.87:0/3353862244 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88bfff700 1 -- 172.21.15.87:0/3353862244 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c107240 con 0x7fd88c1068e0 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88bfff700 1 --2- 172.21.15.87:0/3353862244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893264847203342 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88affd700 1 -- 172.21.15.87:0/3353862244 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd87c004160 con 0x7fd88c1068e0 2024-06-06T07:36:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88affd700 1 -- 172.21.15.87:0/3353862244 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd87c004300 con 0x7fd88c1068e0 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.881+0000 7fd88affd700 1 -- 172.21.15.87:0/3353862244 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd87c011b50 con 0x7fd88c1068e0 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 -- 172.21.15.87:0/3353862244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 msgr2=0x7fd88c106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 --2- 172.21.15.87:0/3353862244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 -- 172.21.15.87:0/3353862244 shutdown_connections 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 --2- 172.21.15.87:0/3353862244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c106d00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 -- 172.21.15.87:0/3353862244 >> 172.21.15.87:0/3353862244 conn(0x7fd88c1024c0 msgr2=0x7fd88c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 -- 172.21.15.87:0/3353862244 shutdown_connections 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.882+0000 7fd892532700 1 -- 172.21.15.87:0/3353862244 wait complete. 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd892532700 1 Processor -- start 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd892532700 1 -- start start 2024-06-06T07:36:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd892532700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd892532700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1081f0 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd88bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd88bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33286/0 (socket says 172.21.15.87:33286) 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.883+0000 7fd88bfff700 1 -- 172.21.15.87:0/2975750891 learned_addr learned my addr 172.21.15.87:0/2975750891 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd88bfff700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c19e600 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd88bfff700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011b60 tx=0x7fd87c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd87c003b20 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd87c003cc0 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c19e890 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd87c024840 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.884+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a1580 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd87c022020 con 0x7fd88c1068e0 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd8897fa700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:22.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd87403bc20 con 0x7fd874039050 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd87c011e00 con 0x7fd88c1068e0 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd88b7fe700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:15.885+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:16.086+0000 7fd88b7fe700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:16.086+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:16.487+0000 7fd88b7fe700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:16.487+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:17.288+0000 7fd88b7fe700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:17.288+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:18.889+0000 7fd88b7fe700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/4176793720 2024-06-06T07:36:22.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:18.889+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:20.512+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fd87c01a960 con 0x7fd88c1068e0 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:20.512+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 msgr2=0x7fd87403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:20.512+0000 7fd8897fa700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.514+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fd87c04ee50 con 0x7fd88c1068e0 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.514+0000 7fd8897fa700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 0x7fd87403ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.514+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd87c00ce90 con 0x7fd87403c980 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.514+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 0x7fd87403ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.515+0000 7fd88b7fe700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 0x7fd87403ed70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd880003a10 tx=0x7fd880009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.515+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fd87c00ce90 con 0x7fd87403c980 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd88c063af0 con 0x7fd87403c980 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd8897fa700 1 -- 172.21.15.87:0/2975750891 <== mgr.14118 v2:172.21.15.87:6800/1605115594 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd88c063af0 con 0x7fd87403c980 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 msgr2=0x7fd87403ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:22.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 0x7fd87403ed70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd880003a10 tx=0x7fd880009290 comp rx=0 tx=0).stop 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 msgr2=0x7fd88c19e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011b60 tx=0x7fd87c011c40 comp rx=0 tx=0).stop 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 shutdown_connections 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fd87403c980 0x7fd87403ed70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd88c1068e0 0x7fd88c19e0c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.522+0000 7fd892532700 1 --2- 172.21.15.87:0/2975750891 >> [v2:172.21.15.87:6800/4176793720,v1:172.21.15.87:6801/4176793720] conn(0x7fd874039050 0x7fd87403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.523+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 >> 172.21.15.87:0/2975750891 conn(0x7fd88c1024c0 msgr2=0x7fd88c194d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.523+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 shutdown_connections 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:21.523+0000 7fd892532700 1 -- 172.21.15.87:0/2975750891 wait complete. 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 5 is available 2024-06-06T07:36:22.091 INFO:teuthology.orchestra.run.smithi087.stdout:Setting orchestrator backend to cephadm... 2024-06-06T07:36:22.701 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:22 smithi087 ceph-mon[32276]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T07:36:22.701 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:22 smithi087 ceph-mon[32276]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T07:36:22.701 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:22 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:21] ENGINE Bus STARTING 2024-06-06T07:36:22.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:22 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:21] ENGINE Serving on https://172.21.15.87:7150 2024-06-06T07:36:22.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:22 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:23.172 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.477+0000 7f1befbbe700 1 Processor -- start 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.477+0000 7f1befbbe700 1 -- start start 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.477+0000 7f1befbbe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.477+0000 7f1befbbe700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1be807d150 con 0x7f1be807e720 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bed95a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bed95a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59884/0 (socket says 172.21.15.87:59884) 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bed95a700 1 -- 172.21.15.87:0/2061814577 learned_addr learned my addr 172.21.15.87:0/2061814577 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bed95a700 1 -- 172.21.15.87:0/2061814577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be807d2d0 con 0x7f1be807e720 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bed95a700 1 --2- 172.21.15.87:0/2061814577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009e80 tx=0x7f1be400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54192d313168c1ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bec958700 1 -- 172.21.15.87:0/2061814577 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be4017070 con 0x7f1be807e720 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bec958700 1 -- 172.21.15.87:0/2061814577 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1be400b8c0 con 0x7f1be807e720 2024-06-06T07:36:23.173 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.478+0000 7f1bec958700 1 -- 172.21.15.87:0/2061814577 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be40045c0 con 0x7f1be807e720 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 -- 172.21.15.87:0/2061814577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 msgr2=0x7f1be807cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 --2- 172.21.15.87:0/2061814577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009e80 tx=0x7f1be400b330 comp rx=0 tx=0).stop 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 -- 172.21.15.87:0/2061814577 shutdown_connections 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 --2- 172.21.15.87:0/2061814577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be807cb80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 -- 172.21.15.87:0/2061814577 >> 172.21.15.87:0/2061814577 conn(0x7f1be81035e0 msgr2=0x7f1be8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 -- 172.21.15.87:0/2061814577 shutdown_connections 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 -- 172.21.15.87:0/2061814577 wait complete. 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.479+0000 7f1befbbe700 1 Processor -- start 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1befbbe700 1 -- start start 2024-06-06T07:36:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1befbbe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1befbbe700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1be810e560 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bed95a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bed95a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59892/0 (socket says 172.21.15.87:59892) 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bed95a700 1 -- 172.21.15.87:0/3159515155 learned_addr learned my addr 172.21.15.87:0/3159515155 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bed95a700 1 -- 172.21.15.87:0/3159515155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be819e650 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bed95a700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009e50 tx=0x7f1be400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be4020020 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1be4003b50 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be819e8e0 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be81a15d0 con 0x7f1be807e720 2024-06-06T07:36:23.175 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.480+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be4017040 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.481+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f1be4023020 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.481+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be8063af0 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.481+0000 7f1bdeffd700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 0x7f1bd403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.482+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1be400f050 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.482+0000 7f1bed159700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 0x7f1bd403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.482+0000 7f1bed159700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 0x7f1bd403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8006fd0 tx=0x7f1bd8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.487+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1be402a020 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.564+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1be4025400 con 0x7f1be807e720 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.700+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1be8104820 con 0x7f1bd4038f20 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.705+0000 7f1bdeffd700 1 -- 172.21.15.87:0/3159515155 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1be8104820 con 0x7f1bd4038f20 2024-06-06T07:36:23.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 msgr2=0x7f1bd403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 0x7f1bd403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8006fd0 tx=0x7f1bd8006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 msgr2=0x7f1be819e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009e50 tx=0x7f1be400bd30 comp rx=0 tx=0).stop 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 shutdown_connections 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1bd4038f20 0x7f1bd403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 --2- 172.21.15.87:0/3159515155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1be807e720 0x7f1be819e110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 >> 172.21.15.87:0/3159515155 conn(0x7f1be81035e0 msgr2=0x7f1be8104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 shutdown_connections 2024-06-06T07:36:23.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:22.707+0000 7f1befbbe700 1 -- 172.21.15.87:0/3159515155 wait complete. 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:21] ENGINE Serving on http://172.21.15.87:8765 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:21] ENGINE Bus STARTED 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: mgrmap e8: x(active, since 2s) 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:23 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:24.363 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-06T07:36:24.363 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff6bb9700 1 Processor -- start 2024-06-06T07:36:24.363 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff6bb9700 1 -- start start 2024-06-06T07:36:24.363 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff6bb9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:24.363 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff6bb9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feff007c650 con 0x7feff0108b70 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff4955700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff4955700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59894/0 (socket says 172.21.15.87:59894) 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.549+0000 7feff4955700 1 -- 172.21.15.87:0/3311950577 learned_addr learned my addr 172.21.15.87:0/3311950577 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.550+0000 7feff4955700 1 -- 172.21.15.87:0/3311950577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff010b4a0 con 0x7feff0108b70 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.550+0000 7feff4955700 1 --2- 172.21.15.87:0/3311950577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefe40098a0 tx=0x7fefe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dada0fecde525 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.550+0000 7fefef7fe700 1 -- 172.21.15.87:0/3311950577 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe4004160 con 0x7feff0108b70 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.550+0000 7fefef7fe700 1 -- 172.21.15.87:0/3311950577 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefe4004300 con 0x7feff0108b70 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.550+0000 7fefef7fe700 1 -- 172.21.15.87:0/3311950577 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe4011b50 con 0x7feff0108b70 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- 172.21.15.87:0/3311950577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 msgr2=0x7feff010af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3311950577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefe40098a0 tx=0x7fefe4009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- 172.21.15.87:0/3311950577 shutdown_connections 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3311950577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff010af60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:24.364 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- 172.21.15.87:0/3311950577 >> 172.21.15.87:0/3311950577 conn(0x7feff01024c0 msgr2=0x7feff0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- 172.21.15.87:0/3311950577 shutdown_connections 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- 172.21.15.87:0/3311950577 wait complete. 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 Processor -- start 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- start start 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.551+0000 7feff6bb9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feff010c320 con 0x7feff0108b70 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff4955700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff4955700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59902/0 (socket says 172.21.15.87:59902) 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff4955700 1 -- 172.21.15.87:0/3256557289 learned_addr learned my addr 172.21.15.87:0/3256557289 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff4955700 1 -- 172.21.15.87:0/3256557289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff019c400 con 0x7feff0108b70 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff4955700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fefe4011b60 tx=0x7fefe4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:24.365 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe4003970 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefe4003b10 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff019c690 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe4024610 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.552+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff019f380 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.553+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fefe4024910 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.553+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0063af0 con 0x7feff0108b70 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.553+0000 7fefedffb700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 0x7fefe003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:24.366 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.553+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fefe4011e00 con 0x7feff0108b70 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.554+0000 7fefeffff700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 0x7fefe003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.554+0000 7fefeffff700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 0x7fefe003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.558+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fefe40172b0 con 0x7feff0108b70 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.756+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7feff0103710 con 0x7fefe0039050 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.757+0000 7fefedffb700 1 -- 172.21.15.87:0/3256557289 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7feff0103710 con 0x7fefe0039050 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 msgr2=0x7fefe003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 0x7fefe003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 msgr2=0x7feff019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fefe4011b60 tx=0x7fefe4011c40 comp rx=0 tx=0).stop 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 shutdown_connections 2024-06-06T07:36:24.367 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fefe0039050 0x7fefe003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:24.368 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 --2- 172.21.15.87:0/3256557289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff0108b70 0x7feff019bec0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:24.368 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 >> 172.21.15.87:0/3256557289 conn(0x7feff01024c0 msgr2=0x7feff0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:24.368 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 shutdown_connections 2024-06-06T07:36:24.368 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:23.759+0000 7feff6bb9700 1 -- 172.21.15.87:0/3256557289 wait complete. 2024-06-06T07:36:24.368 INFO:teuthology.orchestra.run.smithi087.stdout:Generating ssh key... 2024-06-06T07:36:25.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:24 smithi087 ceph-mon[32276]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: Generating public/private rsa key pair. 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: Your identification has been saved in /tmp/tmpu7pudbg_/key. 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: Your public key has been saved in /tmp/tmpu7pudbg_/key.pub. 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: The key fingerprint is: 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: SHA256:3HYll6c/78sS+5c7pAn8lBOqNvpPrsJUnaregty7k0c ceph-3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: The key's randomart image is: 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: +---[RSA 3072]----+ 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | | 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | . | 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | . o + .| 2024-06-06T07:36:25.611 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | . o o = o | 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | S = o + | 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | . E = = o | 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | . = + ..+ B.o| 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | o O.=o *.o+| 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: | .=@++o o*B| 2024-06-06T07:36:25.612 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: +----[SHA256]-----+ 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.805+0000 7f85e2a8c700 1 Processor -- start 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.805+0000 7f85e2a8c700 1 -- start start 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.805+0000 7f85e2a8c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.805+0000 7f85e2a8c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc07d110 con 0x7f85dc07e6e0 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85e0828700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:25.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85e0828700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59910/0 (socket says 172.21.15.87:59910) 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85e0828700 1 -- 172.21.15.87:0/1842176707 learned_addr learned my addr 172.21.15.87:0/1842176707 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85e0828700 1 -- 172.21.15.87:0/1842176707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc07d290 con 0x7f85dc07e6e0 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85e0828700 1 --2- 172.21.15.87:0/1842176707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009e80 tx=0x7f85d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d252525aeee9e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85d37fe700 1 -- 172.21.15.87:0/1842176707 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8017070 con 0x7f85dc07e6e0 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85d37fe700 1 -- 172.21.15.87:0/1842176707 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85d800b8c0 con 0x7f85dc07e6e0 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.806+0000 7f85d37fe700 1 -- 172.21.15.87:0/1842176707 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d80045c0 con 0x7f85dc07e6e0 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- 172.21.15.87:0/1842176707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 msgr2=0x7f85dc07cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/1842176707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009e80 tx=0x7f85d800b330 comp rx=0 tx=0).stop 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- 172.21.15.87:0/1842176707 shutdown_connections 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/1842176707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc07cb40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- 172.21.15.87:0/1842176707 >> 172.21.15.87:0/1842176707 conn(0x7f85dc1035e0 msgr2=0x7f85dc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- 172.21.15.87:0/1842176707 shutdown_connections 2024-06-06T07:36:25.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- 172.21.15.87:0/1842176707 wait complete. 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 Processor -- start 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.807+0000 7f85e2a8c700 1 -- start start 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e2a8c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e2a8c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc10e540 con 0x7f85dc07e6e0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e0828700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e0828700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59926/0 (socket says 172.21.15.87:59926) 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e0828700 1 -- 172.21.15.87:0/3601044679 learned_addr learned my addr 172.21.15.87:0/3601044679 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e0828700 1 -- 172.21.15.87:0/3601044679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc19e630 con 0x7f85dc07e6e0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e0828700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009e50 tx=0x7f85d800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8020020 con 0x7f85dc07e6e0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc19e8c0 con 0x7f85dc07e6e0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85d8003b50 con 0x7f85dc07e6e0 2024-06-06T07:36:25.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.808+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d8017040 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.809+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a15b0 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.809+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85d8023020 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.809+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc063af0 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.810+0000 7f85d1ffb700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 0x7f85c403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.810+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85d800f050 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.810+0000 7f85d3fff700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 0x7f85c403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.810+0000 7f85d3fff700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 0x7f85c403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85cc006fd0 tx=0x7f85cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:24.815+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f85d802a020 con 0x7f85dc07e6e0 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.011+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f85dc104800 con 0x7f85c4039000 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.177+0000 7f85d1ffb700 1 -- 172.21.15.87:0/3601044679 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85dc104800 con 0x7f85c4039000 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 msgr2=0x7f85c403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 0x7f85c403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85cc006fd0 tx=0x7f85cc006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 msgr2=0x7f85dc19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009e50 tx=0x7f85d800bd30 comp rx=0 tx=0).stop 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 shutdown_connections 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f85c4039000 0x7f85c403b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 --2- 172.21.15.87:0/3601044679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85dc07e6e0 0x7f85dc19e0f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 >> 172.21.15.87:0/3601044679 conn(0x7f85dc1035e0 msgr2=0x7f85dc1040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 shutdown_connections 2024-06-06T07:36:25.755 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:25.179+0000 7f85e2a8c700 1 -- 172.21.15.87:0/3601044679 wait complete. 2024-06-06T07:36:26.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:26 smithi087 ceph-mon[32276]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:26.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:26 smithi087 ceph-mon[32276]: Generating ssh key... 2024-06-06T07:36:26.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:26 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:26.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:26 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:26.966 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6aca40f700 1 Processor -- start 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6aca40f700 1 -- start start 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6aca40f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6aca40f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac407be60 con 0x7f6ac4108ab0 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59934/0 (socket says 172.21.15.87:59934) 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.153+0000 7f6ac3fff700 1 -- 172.21.15.87:0/1334932989 learned_addr learned my addr 172.21.15.87:0/1334932989 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.154+0000 7f6ac3fff700 1 -- 172.21.15.87:0/1334932989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac410b3e0 con 0x7f6ac4108ab0 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.154+0000 7f6ac3fff700 1 --2- 172.21.15.87:0/1334932989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098a0 tx=0x7f6ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470afebea98a6e09 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:26.967 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.154+0000 7f6ac2ffd700 1 -- 172.21.15.87:0/1334932989 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4004160 con 0x7f6ac4108ab0 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.154+0000 7f6ac2ffd700 1 -- 172.21.15.87:0/1334932989 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ab4004300 con 0x7f6ac4108ab0 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.154+0000 7f6ac2ffd700 1 -- 172.21.15.87:0/1334932989 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4011b50 con 0x7f6ac4108ab0 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- 172.21.15.87:0/1334932989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 msgr2=0x7f6ac410aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1334932989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098a0 tx=0x7f6ab4009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- 172.21.15.87:0/1334932989 shutdown_connections 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1334932989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac410aea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- 172.21.15.87:0/1334932989 >> 172.21.15.87:0/1334932989 conn(0x7f6ac41024c0 msgr2=0x7f6ac41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- 172.21.15.87:0/1334932989 shutdown_connections 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- 172.21.15.87:0/1334932989 wait complete. 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 Processor -- start 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.155+0000 7f6aca40f700 1 -- start start 2024-06-06T07:36:26.968 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6aca40f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6aca40f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac410c320 con 0x7f6ac4108ab0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59942/0 (socket says 172.21.15.87:59942) 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac3fff700 1 -- 172.21.15.87:0/1564424258 learned_addr learned my addr 172.21.15.87:0/1564424258 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac3fff700 1 -- 172.21.15.87:0/1564424258 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac419c410 con 0x7f6ac4108ab0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac3fff700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011b60 tx=0x7f6ab4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4003b20 con 0x7f6ac4108ab0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ab4003cc0 con 0x7f6ac4108ab0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac419c6a0 con 0x7f6ac4108ab0 2024-06-06T07:36:26.969 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab4024840 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.156+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac419f390 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.157+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6ab4022020 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.157+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac4063af0 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.157+0000 7f6ac17fa700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 0x7f6aac03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.157+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ab4011e00 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.158+0000 7f6ac37fe700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 0x7f6aac03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.158+0000 7f6ac37fe700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 0x7f6aac03b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006fd0 tx=0x7f6ab8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.162+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ab402d3f0 con 0x7f6ac4108ab0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.363+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6ac41036e0 con 0x7f6aac0390a0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.363+0000 7f6ac17fa700 1 -- 172.21.15.87:0/1564424258 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6ac41036e0 con 0x7f6aac0390a0 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.365+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 msgr2=0x7f6aac03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:26.970 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.365+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 0x7f6aac03b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006fd0 tx=0x7f6ab8006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.365+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 msgr2=0x7f6ac419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.365+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011b60 tx=0x7f6ab4011c40 comp rx=0 tx=0).stop 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 shutdown_connections 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f6aac0390a0 0x7f6aac03b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 --2- 172.21.15.87:0/1564424258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ac4108ab0 0x7f6ac419bed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 >> 172.21.15.87:0/1564424258 conn(0x7f6ac41024c0 msgr2=0x7f6ac4102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 shutdown_connections 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:26.366+0000 7f6aca40f700 1 -- 172.21.15.87:0/1564424258 wait complete. 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:Adding key to root@localhost authorized_keys... 2024-06-06T07:36:26.971 INFO:teuthology.orchestra.run.smithi087.stdout:Adding host smithi087... 2024-06-06T07:36:27.352 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:27 smithi087 ceph-mon[32276]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:28.472 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:28 smithi087 ceph-mon[32276]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "addr": "172.21.15.87", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:29.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:29 smithi087 ceph-mon[32276]: Deploying cephadm binary to smithi087 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Added host 'smithi087' with addr '172.21.15.87' 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.388+0000 7f8e290c4700 1 Processor -- start 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.388+0000 7f8e290c4700 1 -- start start 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.388+0000 7f8e290c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.388+0000 7f8e290c4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24107860 con 0x7f8e24106f00 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e22d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e22d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59094/0 (socket says 172.21.15.87:59094) 2024-06-06T07:36:32.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e22d9d700 1 -- 172.21.15.87:0/2701344636 learned_addr learned my addr 172.21.15.87:0/2701344636 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e22d9d700 1 -- 172.21.15.87:0/2701344636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e24108060 con 0x7f8e24106f00 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e22d9d700 1 --2- 172.21.15.87:0/2701344636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098a0 tx=0x7f8e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4246342c14fee34c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e2259c700 1 -- 172.21.15.87:0/2701344636 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e14004160 con 0x7f8e24106f00 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e2259c700 1 -- 172.21.15.87:0/2701344636 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e14004300 con 0x7f8e24106f00 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.389+0000 7f8e2259c700 1 -- 172.21.15.87:0/2701344636 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e14011b50 con 0x7f8e24106f00 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 -- 172.21.15.87:0/2701344636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 msgr2=0x7f8e24107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 --2- 172.21.15.87:0/2701344636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098a0 tx=0x7f8e14009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 -- 172.21.15.87:0/2701344636 shutdown_connections 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 --2- 172.21.15.87:0/2701344636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e24107320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 -- 172.21.15.87:0/2701344636 >> 172.21.15.87:0/2701344636 conn(0x7f8e241024c0 msgr2=0x7f8e241048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 -- 172.21.15.87:0/2701344636 shutdown_connections 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.390+0000 7f8e290c4700 1 -- 172.21.15.87:0/2701344636 wait complete. 2024-06-06T07:36:32.200 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e290c4700 1 Processor -- start 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e290c4700 1 -- start start 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e290c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e290c4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410b320 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e22d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e22d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59108/0 (socket says 172.21.15.87:59108) 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e22d9d700 1 -- 172.21.15.87:0/1145455137 learned_addr learned my addr 172.21.15.87:0/1145455137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.391+0000 7f8e22d9d700 1 -- 172.21.15.87:0/1145455137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2419c450 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e22d9d700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e14011670 tx=0x7f8e14011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e140039c0 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e14011e50 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2419c6e0 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e1401a5c0 con 0x7f8e24106f00 2024-06-06T07:36:32.201 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.392+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2419f3d0 con 0x7f8e24106f00 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.393+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8e1401a870 con 0x7f8e24106f00 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.393+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e24063af0 con 0x7f8e24106f00 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.393+0000 7f8e20d99700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 0x7f8e1003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.393+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e1404d340 con 0x7f8e24106f00 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.393+0000 7f8e1a59c700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 0x7f8e1003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.394+0000 7f8e1a59c700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 0x7f8e1003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006fd0 tx=0x7f8e0c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.398+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e1402cc00 con 0x7f8e24106f00 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:27.598+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi087", "addr": "172.21.15.87", "target": ["mon-mgr", ""]}) -- 0x7f8e240008d0 con 0x7f8e10039050 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.672+0000 7f8e20d99700 1 -- 172.21.15.87:0/1145455137 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8e240008d0 con 0x7f8e10039050 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 msgr2=0x7f8e1003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 0x7f8e1003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006fd0 tx=0x7f8e0c006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 msgr2=0x7f8e2419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:32.202 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8e14011670 tx=0x7f8e14011b60 comp rx=0 tx=0).stop 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 shutdown_connections 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f8e10039050 0x7f8e1003b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 --2- 172.21.15.87:0/1145455137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e24106f00 0x7f8e2419bf10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 >> 172.21.15.87:0/1145455137 conn(0x7f8e241024c0 msgr2=0x7f8e241033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 shutdown_connections 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:31.675+0000 7f8e290c4700 1 -- 172.21.15.87:0/1145455137 wait complete. 2024-06-06T07:36:32.203 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mon service... 2024-06-06T07:36:32.822 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:32 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:32.822 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:32 smithi087 ceph-mon[32276]: Added host smithi087 2024-06-06T07:36:32.823 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:32 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:33.534 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-06T07:36:33.534 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.671+0000 7fb75f8ef700 1 Processor -- start 2024-06-06T07:36:33.534 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75f8ef700 1 -- start start 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75f8ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75f8ef700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb758107850 con 0x7fb758106ef0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75d68b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75d68b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59122/0 (socket says 172.21.15.87:59122) 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75d68b700 1 -- 172.21.15.87:0/2950991296 learned_addr learned my addr 172.21.15.87:0/2950991296 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75d68b700 1 -- 172.21.15.87:0/2950991296 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb758108090 con 0x7fb758106ef0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.672+0000 7fb75d68b700 1 --2- 172.21.15.87:0/2950991296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d7aa7080dd47 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.673+0000 7fb74ffff700 1 -- 172.21.15.87:0/2950991296 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754004160 con 0x7fb758106ef0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.673+0000 7fb74ffff700 1 -- 172.21.15.87:0/2950991296 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb754004300 con 0x7fb758106ef0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.673+0000 7fb74ffff700 1 -- 172.21.15.87:0/2950991296 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754011b50 con 0x7fb758106ef0 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 -- 172.21.15.87:0/2950991296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 msgr2=0x7fb758107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:33.535 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/2950991296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 -- 172.21.15.87:0/2950991296 shutdown_connections 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/2950991296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb758107310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 -- 172.21.15.87:0/2950991296 >> 172.21.15.87:0/2950991296 conn(0x7fb758102520 msgr2=0x7fb758104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 -- 172.21.15.87:0/2950991296 shutdown_connections 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.674+0000 7fb75f8ef700 1 -- 172.21.15.87:0/2950991296 wait complete. 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75f8ef700 1 Processor -- start 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75f8ef700 1 -- start start 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75f8ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75f8ef700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581079d0 con 0x7fb758106ef0 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75d68b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75d68b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59130/0 (socket says 172.21.15.87:59130) 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75d68b700 1 -- 172.21.15.87:0/1448990615 learned_addr learned my addr 172.21.15.87:0/1448990615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:33.536 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75d68b700 1 -- 172.21.15.87:0/1448990615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819ec30 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75d68b700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754003b20 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819ee00 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb754003cc0 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.675+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754024910 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.676+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75819f1a0 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.676+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb754022020 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.676+0000 7fb74e7fc700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 0x7fb74403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.676+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758063af0 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.676+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb754011e00 con 0x7fb758106ef0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.677+0000 7fb75ce8a700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 0x7fb74403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.677+0000 7fb75ce8a700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 0x7fb74403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb748006fd0 tx=0x7fb748006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:33.537 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.682+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb75400bdd0 con 0x7fb758106ef0 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.902+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb75819f4e0 con 0x7fb744039050 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.910+0000 7fb74e7fc700 1 -- 172.21.15.87:0/1448990615 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb75819f4e0 con 0x7fb744039050 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 msgr2=0x7fb74403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 0x7fb74403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb748006fd0 tx=0x7fb748006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 msgr2=0x7fb75807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 shutdown_connections 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fb744039050 0x7fb74403b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 --2- 172.21.15.87:0/1448990615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb758106ef0 0x7fb75807c2d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 >> 172.21.15.87:0/1448990615 conn(0x7fb758102520 msgr2=0x7fb758103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 shutdown_connections 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:32.912+0000 7fb75f8ef700 1 -- 172.21.15.87:0/1448990615 wait complete. 2024-06-06T07:36:33.538 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mgr service... 2024-06-06T07:36:34.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:33 smithi087 ceph-mon[32276]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:34.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:33 smithi087 ceph-mon[32276]: Saving service mon spec with placement count:5 2024-06-06T07:36:34.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:33 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:34.845 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-06T07:36:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d9ad6700 1 Processor -- start 2024-06-06T07:36:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d9ad6700 1 -- start start 2024-06-06T07:36:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d9ad6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d9ad6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d407c6f0 con 0x7fa4d4108aa0 2024-06-06T07:36:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59146/0 (socket says 172.21.15.87:59146) 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d37fe700 1 -- 172.21.15.87:0/2678806125 learned_addr learned my addr 172.21.15.87:0/2678806125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.943+0000 7fa4d37fe700 1 -- 172.21.15.87:0/2678806125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d410b3d0 con 0x7fa4d4108aa0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d37fe700 1 --2- 172.21.15.87:0/2678806125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40098a0 tx=0x7fa4c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfa28232cda252a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d27fc700 1 -- 172.21.15.87:0/2678806125 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c4004160 con 0x7fa4d4108aa0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d27fc700 1 -- 172.21.15.87:0/2678806125 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa4c4004300 con 0x7fa4d4108aa0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d27fc700 1 -- 172.21.15.87:0/2678806125 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c4011b50 con 0x7fa4d4108aa0 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/2678806125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 msgr2=0x7fa4d410ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/2678806125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40098a0 tx=0x7fa4c4009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/2678806125 shutdown_connections 2024-06-06T07:36:34.847 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/2678806125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d410ae90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/2678806125 >> 172.21.15.87:0/2678806125 conn(0x7fa4d4102470 msgr2=0x7fa4d41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/2678806125 shutdown_connections 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.944+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/2678806125 wait complete. 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d9ad6700 1 Processor -- start 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d9ad6700 1 -- start start 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d9ad6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d9ad6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410c310 con 0x7fa4d4108aa0 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59154/0 (socket says 172.21.15.87:59154) 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.945+0000 7fa4d37fe700 1 -- 172.21.15.87:0/1288874468 learned_addr learned my addr 172.21.15.87:0/1288874468 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d37fe700 1 -- 172.21.15.87:0/1288874468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419c480 con 0x7fa4d4108aa0 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d37fe700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4011b60 tx=0x7fa4c4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:34.848 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c4003970 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa4c4003b10 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d419c710 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c4024610 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.946+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d419f3b0 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.947+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa4c4024910 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.947+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4063af0 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.947+0000 7fa4d0ff9700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 0x7fa4bc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.947+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4c4011e00 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.947+0000 7fa4d2ffd700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 0x7fa4bc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.948+0000 7fa4d2ffd700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 0x7fa4bc03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006fd0 tx=0x7fa4c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:33.952+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa4c400ba00 con 0x7fa4d4108aa0 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.167+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa4d4103700 con 0x7fa4bc039050 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.175+0000 7fa4d0ff9700 1 -- 172.21.15.87:0/1288874468 <== mgr.14118 v2:172.21.15.87:6800/1605115594 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa4d4103700 con 0x7fa4bc039050 2024-06-06T07:36:34.849 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 msgr2=0x7fa4bc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 0x7fa4bc03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006fd0 tx=0x7fa4c8006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 msgr2=0x7fa4d419bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4011b60 tx=0x7fa4c4011c40 comp rx=0 tx=0).stop 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 shutdown_connections 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fa4bc039050 0x7fa4bc03b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 --2- 172.21.15.87:0/1288874468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4d4108aa0 0x7fa4d419bf40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 >> 172.21.15.87:0/1288874468 conn(0x7fa4d4102470 msgr2=0x7fa4d4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:34.850 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 shutdown_connections 2024-06-06T07:36:34.851 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:34.178+0000 7fa4d9ad6700 1 -- 172.21.15.87:0/1288874468 wait complete. 2024-06-06T07:36:35.476 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:35 smithi087 ceph-mon[32276]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:35.476 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:35 smithi087 ceph-mon[32276]: Saving service mgr spec with placement count:2 2024-06-06T07:36:35.476 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:35 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.269+0000 7f3ff78a1700 1 Processor -- start 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.269+0000 7f3ff78a1700 1 -- start start 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.269+0000 7f3ff78a1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.269+0000 7f3ff78a1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff007be00 con 0x7f3ff0108b10 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3ff563d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:36.176 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3ff563d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59170/0 (socket says 172.21.15.87:59170) 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3ff563d700 1 -- 172.21.15.87:0/3575998380 learned_addr learned my addr 172.21.15.87:0/3575998380 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3ff563d700 1 -- 172.21.15.87:0/3575998380 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff010b440 con 0x7f3ff0108b10 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3ff563d700 1 --2- 172.21.15.87:0/3575998380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009e80 tx=0x7f3fec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc7712be09099e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3fe7fff700 1 -- 172.21.15.87:0/3575998380 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fec017070 con 0x7f3ff0108b10 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3fe7fff700 1 -- 172.21.15.87:0/3575998380 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3fec00b8c0 con 0x7f3ff0108b10 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.270+0000 7f3fe7fff700 1 -- 172.21.15.87:0/3575998380 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fec0045c0 con 0x7f3ff0108b10 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3575998380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 msgr2=0x7f3ff010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3575998380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009e80 tx=0x7f3fec00b330 comp rx=0 tx=0).stop 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3575998380 shutdown_connections 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3575998380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff010af00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3575998380 >> 172.21.15.87:0/3575998380 conn(0x7f3ff01024c0 msgr2=0x7f3ff01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3575998380 shutdown_connections 2024-06-06T07:36:36.177 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3575998380 wait complete. 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 Processor -- start 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.271+0000 7f3ff78a1700 1 -- start start 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff78a1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff78a1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff010c320 con 0x7f3ff0108b10 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff563d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff563d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59184/0 (socket says 172.21.15.87:59184) 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff563d700 1 -- 172.21.15.87:0/3912717148 learned_addr learned my addr 172.21.15.87:0/3912717148 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff563d700 1 -- 172.21.15.87:0/3912717148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff019c3b0 con 0x7f3ff0108b10 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff563d700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0072a0 tx=0x7f3fec00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fec020020 con 0x7f3ff0108b10 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3fec003b50 con 0x7f3ff0108b10 2024-06-06T07:36:36.178 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fec017040 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff019c640 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.272+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff019f330 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.273+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3fec023020 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.273+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff0063af0 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.273+0000 7f3fe67fc700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 0x7f3fdc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.273+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3fec00f050 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.273+0000 7f3ff4e3c700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 0x7f3fdc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.274+0000 7f3ff4e3c700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 0x7f3fdc03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.278+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3fec02a020 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.475+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3ff019f9a0 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.480+0000 7f3fe67fc700 1 -- 172.21.15.87:0/3912717148 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f3fec01ce20 con 0x7f3ff0108b10 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 msgr2=0x7f3fdc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:36.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 0x7f3fdc03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 msgr2=0x7f3ff019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0072a0 tx=0x7f3fec00bd50 comp rx=0 tx=0).stop 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 shutdown_connections 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f3fdc039050 0x7f3fdc03b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 --2- 172.21.15.87:0/3912717148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ff0108b10 0x7f3ff019be70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 >> 172.21.15.87:0/3912717148 conn(0x7f3ff01024c0 msgr2=0x7f3ff0102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 shutdown_connections 2024-06-06T07:36:36.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:35.482+0000 7f3ff78a1700 1 -- 172.21.15.87:0/3912717148 wait complete. 2024-06-06T07:36:36.549 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:36 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3912717148' entity='client.admin' 2024-06-06T07:36:36.549 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:36 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:36.549 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:36 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc99daf6700 1 Processor -- start 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc99daf6700 1 -- start start 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc99daf6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc99daf6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc998106ce0 con 0x7fc9981062f0 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59192/0 (socket says 172.21.15.87:59192) 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc9977fe700 1 -- 172.21.15.87:0/82646349 learned_addr learned my addr 172.21.15.87:0/82646349 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.649+0000 7fc9977fe700 1 -- 172.21.15.87:0/82646349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc998109280 con 0x7fc9981062f0 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc9977fe700 1 --2- 172.21.15.87:0/82646349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0098a0 tx=0x7fc98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b6667d3b2ea370 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc9967fc700 1 -- 172.21.15.87:0/82646349 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc98c004160 con 0x7fc9981062f0 2024-06-06T07:36:37.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc9967fc700 1 -- 172.21.15.87:0/82646349 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c004300 con 0x7fc9981062f0 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc9967fc700 1 -- 172.21.15.87:0/82646349 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc98c011b20 con 0x7fc9981062f0 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 -- 172.21.15.87:0/82646349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 msgr2=0x7fc998106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 --2- 172.21.15.87:0/82646349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0098a0 tx=0x7fc98c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 -- 172.21.15.87:0/82646349 shutdown_connections 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 --2- 172.21.15.87:0/82646349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998106710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 -- 172.21.15.87:0/82646349 >> 172.21.15.87:0/82646349 conn(0x7fc998101c40 msgr2=0x7fc998104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 -- 172.21.15.87:0/82646349 shutdown_connections 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.650+0000 7fc99daf6700 1 -- 172.21.15.87:0/82646349 wait complete. 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc99daf6700 1 Processor -- start 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc99daf6700 1 -- start start 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc99daf6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc99daf6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc99810c630 con 0x7fc9981062f0 2024-06-06T07:36:37.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59208/0 (socket says 172.21.15.87:59208) 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.651+0000 7fc9977fe700 1 -- 172.21.15.87:0/3405723635 learned_addr learned my addr 172.21.15.87:0/3405723635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc9977fe700 1 -- 172.21.15.87:0/3405723635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc998109040 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc9977fe700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0117e0 tx=0x7fc98c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc98c003bf0 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc998107230 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c01a910 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc98c024970 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.652+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981076f0 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.653+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc98c022020 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.653+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998063af0 con 0x7fc9981062f0 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.653+0000 7fc994ff9700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 0x7fc98003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:37.830 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.653+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc98c04d8c0 con 0x7fc9981062f0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.653+0000 7fc996ffd700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 0x7fc98003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.654+0000 7fc996ffd700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 0x7fc98003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc988006fd0 tx=0x7fc988006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.658+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc98c02cdf0 con 0x7fc9981062f0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.873+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc998107970 con 0x7fc9981062f0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.876+0000 7fc994ff9700 1 -- 172.21.15.87:0/3405723635 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc98c020c10 con 0x7fc9981062f0 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 msgr2=0x7fc98003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 0x7fc98003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc988006fd0 tx=0x7fc988006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 msgr2=0x7fc998108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0117e0 tx=0x7fc98c011ba0 comp rx=0 tx=0).stop 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 shutdown_connections 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fc980039050 0x7fc98003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 --2- 172.21.15.87:0/3405723635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981062f0 0x7fc998108b00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:37.831 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 >> 172.21.15.87:0/3405723635 conn(0x7fc998101c40 msgr2=0x7fc9981026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:37.832 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 shutdown_connections 2024-06-06T07:36:37.832 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:36.878+0000 7fc99daf6700 1 -- 172.21.15.87:0/3405723635 wait complete. 2024-06-06T07:36:37.832 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling the dashboard module... 2024-06-06T07:36:38.181 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:37 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3405723635' entity='client.admin' 2024-06-06T07:36:39.386 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-mon[32276]: from='mgr.14118 172.21.15.87:0/915778248' entity='mgr.x' 2024-06-06T07:36:39.386 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1479221165' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-06T07:36:39.386 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setuser ceph since I am not root 2024-06-06T07:36:39.386 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setgroup ceph since I am not root 2024-06-06T07:36:39.387 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:39.297+0000 7f0b62d3a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:36:39.743 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:39.492+0000 7f0b62d3a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:36:39.743 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:39.596+0000 7f0b62d3a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:36:39.750 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.285+0000 7fdcc4827700 1 Processor -- start 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.285+0000 7fdcc4827700 1 -- start start 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.285+0000 7fdcc4827700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.285+0000 7fdcc4827700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc007c6b0 con 0x7fdcc0106960 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbe59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbe59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44886/0 (socket says 172.21.15.87:44886) 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbe59c700 1 -- 172.21.15.87:0/898664487 learned_addr learned my addr 172.21.15.87:0/898664487 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbe59c700 1 -- 172.21.15.87:0/898664487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01072c0 con 0x7fdcc0106960 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbe59c700 1 --2- 172.21.15.87:0/898664487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a300f7049cb16931 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbd59a700 1 -- 172.21.15.87:0/898664487 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4004160 con 0x7fdcc0106960 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbd59a700 1 -- 172.21.15.87:0/898664487 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4004300 con 0x7fdcc0106960 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.286+0000 7fdcbd59a700 1 -- 172.21.15.87:0/898664487 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4011b20 con 0x7fdcc0106960 2024-06-06T07:36:39.751 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 -- 172.21.15.87:0/898664487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 msgr2=0x7fdcc0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 --2- 172.21.15.87:0/898664487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 -- 172.21.15.87:0/898664487 shutdown_connections 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 --2- 172.21.15.87:0/898664487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc0106d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 -- 172.21.15.87:0/898664487 >> 172.21.15.87:0/898664487 conn(0x7fdcc0102520 msgr2=0x7fdcc0104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 -- 172.21.15.87:0/898664487 shutdown_connections 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.287+0000 7fdcc4827700 1 -- 172.21.15.87:0/898664487 wait complete. 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcc4827700 1 Processor -- start 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcc4827700 1 -- start start 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcc4827700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcc4827700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0108390 con 0x7fdcc0106960 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcbe59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcbe59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44898/0 (socket says 172.21.15.87:44898) 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcbe59c700 1 -- 172.21.15.87:0/1479221165 learned_addr learned my addr 172.21.15.87:0/1479221165 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:39.752 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.288+0000 7fdcbe59c700 1 -- 172.21.15.87:0/1479221165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc019e6a0 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcbe59c700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009fb0 tx=0x7fdcb4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4003930 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4003ad0 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4019690 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc019e930 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.289+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc019ed90 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.290+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdcb4019990 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.290+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0063af0 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.290+0000 7fdcaf7fe700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 0x7fdca803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.290+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdcb404d690 con 0x7fdcc0106960 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.290+0000 7fdcbdd9b700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 0x7fdca803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.291+0000 7fdcbdd9b700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 0x7fdca803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0006fd0 tx=0x7fdcb0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:39.753 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.295+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcb4011e20 con 0x7fdcc0106960 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:38.557+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fdcc00630a0 con 0x7fdcc0106960 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.115+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fdcb4019c80 con 0x7fdcc0106960 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.117+0000 7fdcaf7fe700 1 -- 172.21.15.87:0/1479221165 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fdcb400fd80 con 0x7fdcc0106960 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 msgr2=0x7fdca803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 0x7fdca803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0006fd0 tx=0x7fdcb0006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 msgr2=0x7fdcc019e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009fb0 tx=0x7fdcb4011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 shutdown_connections 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fdca8039000 0x7fdca803b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 --2- 172.21.15.87:0/1479221165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc0106960 0x7fdcc019e160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 >> 172.21.15.87:0/1479221165 conn(0x7fdcc0102520 msgr2=0x7fdcc0194d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 shutdown_connections 2024-06-06T07:36:39.754 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:39.118+0000 7fdcc4827700 1 -- 172.21.15.87:0/1479221165 wait complete. 2024-06-06T07:36:40.011 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:39.921+0000 7f0b62d3a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:36:40.011 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:40.010+0000 7f0b62d3a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:36:40.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:40 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1479221165' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-06T07:36:40.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:40 smithi087 ceph-mon[32276]: mgrmap e9: x(active, since 18s) 2024-06-06T07:36:41.050 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:41.050 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.223+0000 7fcca9829700 1 Processor -- start 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.223+0000 7fcca9829700 1 -- start start 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.223+0000 7fcca9829700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.223+0000 7fcca9829700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcca407c6a0 con 0x7fcca4108af0 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44928/0 (socket says 172.21.15.87:44928) 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca2ffd700 1 -- 172.21.15.87:0/651505749 learned_addr learned my addr 172.21.15.87:0/651505749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:41.051 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca2ffd700 1 -- 172.21.15.87:0/651505749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca410b420 con 0x7fcca4108af0 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca2ffd700 1 --2- 172.21.15.87:0/651505749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc980098a0 tx=0x7fcc98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1436b77dd7b79fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca1ffb700 1 -- 172.21.15.87:0/651505749 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc98004160 con 0x7fcca4108af0 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca1ffb700 1 -- 172.21.15.87:0/651505749 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc98004300 con 0x7fcca4108af0 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.224+0000 7fcca1ffb700 1 -- 172.21.15.87:0/651505749 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc98011b20 con 0x7fcca4108af0 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 -- 172.21.15.87:0/651505749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 msgr2=0x7fcca410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 --2- 172.21.15.87:0/651505749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc980098a0 tx=0x7fcc98009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 -- 172.21.15.87:0/651505749 shutdown_connections 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 --2- 172.21.15.87:0/651505749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca410aee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 -- 172.21.15.87:0/651505749 >> 172.21.15.87:0/651505749 conn(0x7fcca4102560 msgr2=0x7fcca4104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 -- 172.21.15.87:0/651505749 shutdown_connections 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.225+0000 7fcca9829700 1 -- 172.21.15.87:0/651505749 wait complete. 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca9829700 1 Processor -- start 2024-06-06T07:36:41.052 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca9829700 1 -- start start 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca9829700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca9829700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcca410c420 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44944/0 (socket says 172.21.15.87:44944) 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca2ffd700 1 -- 172.21.15.87:0/3704846860 learned_addr learned my addr 172.21.15.87:0/3704846860 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.226+0000 7fcca2ffd700 1 -- 172.21.15.87:0/3704846860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca419c360 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca2ffd700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc980117e0 tx=0x7fcc98011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc98003bf0 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9801a910 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca419c5f0 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc98024970 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.227+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca419ca50 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fcc98022020 con 0x7fcca4108af0 2024-06-06T07:36:41.053 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca4063af0 con 0x7fcca4108af0 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca8827700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 0x7fcc8c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc9804d700 con 0x7fcca4108af0 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca27fc700 1 -- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 msgr2=0x7fcc8c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.228+0000 7fcca27fc700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 0x7fcc8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.237+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc98011e20 con 0x7fcca4108af0 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.429+0000 7fcca27fc700 1 -- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 msgr2=0x7fcc8c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.429+0000 7fcca27fc700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 0x7fcc8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.530+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcca419f400 con 0x7fcca4108af0 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.531+0000 7fcca8827700 1 -- 172.21.15.87:0/3704846860 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcc980172b0 con 0x7fcca4108af0 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 msgr2=0x7fcc8c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 0x7fcc8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 msgr2=0x7fcca419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:41.054 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc980117e0 tx=0x7fcc98011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 shutdown_connections 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7fcc8c039050 0x7fcc8c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 --2- 172.21.15.87:0/3704846860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcca4108af0 0x7fcca419be20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 >> 172.21.15.87:0/3704846860 conn(0x7fcca4102560 msgr2=0x7fcca4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 shutdown_connections 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:40.533+0000 7fcca9829700 1 -- 172.21.15.87:0/3704846860 wait complete. 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-06-06T07:36:41.055 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 9... 2024-06-06T07:36:41.402 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3704846860' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T07:36:41.403 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:41.250+0000 7f0b62d3a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:36:41.716 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:41.431+0000 7f0b62d3a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:36:41.716 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:41.610+0000 7f0b62d3a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:36:42.011 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:41.912+0000 7f0b62d3a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:36:42.011 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:42.010+0000 7f0b62d3a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:36:42.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:42.219+0000 7f0b62d3a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:36:42.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:42.418+0000 7f0b62d3a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:36:43.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:43.024+0000 7f0b62d3a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:36:43.746 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:43.446+0000 7f0b62d3a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:36:43.746 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:43.543+0000 7f0b62d3a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:36:43.746 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:43.745+0000 7f0b62d3a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:36:44.519 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.264+0000 7f0b62d3a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:36:44.520 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.356+0000 7f0b62d3a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:36:44.829 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.599+0000 7f0b62d3a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:36:44.829 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.733+0000 7f0b62d3a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:36:44.829 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.828+0000 7f0b62d3a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:36:45.110 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:44.944+0000 7f0b62d3a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:36:45.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:45.240+0000 7f0b62d3a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:36:46.106 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:45.835+0000 7f0b62d3a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:36:46.106 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:36:46.049+0000 7f0b62d3a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: Active manager daemon x restarted 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: Activating manager daemon x 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: osdmap e3: 0 total, 0 up, 0 in 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: mgrmap e10: x(active, starting, since 0.00341664s) 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:36:46.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: Manager daemon x is now available 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:46.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:36:47.354 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:36:47.354 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:47 smithi087 ceph-mon[32276]: mgrmap e11: x(active, since 1.00573s) 2024-06-06T07:36:47.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-06-06T07:36:47.678 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-06T07:36:47.678 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.509+0000 7f1b4bcd6700 1 Processor -- start 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.509+0000 7f1b4bcd6700 1 -- start start 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.509+0000 7f1b4bcd6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.509+0000 7f1b4bcd6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4407c730 con 0x7f1b44108a90 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.509+0000 7f1b49a72700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b49a72700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44960/0 (socket says 172.21.15.87:44960) 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b49a72700 1 -- 172.21.15.87:0/2844140018 learned_addr learned my addr 172.21.15.87:0/2844140018 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:47.679 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b49a72700 1 -- 172.21.15.87:0/2844140018 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4410b3c0 con 0x7f1b44108a90 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b49a72700 1 --2- 172.21.15.87:0/2844140018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b9be239dfe8051 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b48a70700 1 -- 172.21.15.87:0/2844140018 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40004160 con 0x7f1b44108a90 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b48a70700 1 -- 172.21.15.87:0/2844140018 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b40004300 con 0x7f1b44108a90 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.510+0000 7f1b48a70700 1 -- 172.21.15.87:0/2844140018 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40011b20 con 0x7f1b44108a90 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/2844140018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 msgr2=0x7f1b4410ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/2844140018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b40009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/2844140018 shutdown_connections 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/2844140018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4410ae80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/2844140018 >> 172.21.15.87:0/2844140018 conn(0x7f1b441024c0 msgr2=0x7f1b44104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:47.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/2844140018 shutdown_connections 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.511+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/2844140018 wait complete. 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b4bcd6700 1 Processor -- start 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b4bcd6700 1 -- start start 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b4bcd6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b4bcd6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410c430 con 0x7f1b44108a90 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b49a72700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b49a72700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44962/0 (socket says 172.21.15.87:44962) 2024-06-06T07:36:47.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.512+0000 7f1b49a72700 1 -- 172.21.15.87:0/736195262 learned_addr learned my addr 172.21.15.87:0/736195262 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b49a72700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4419c410 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b49a72700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b40011b60 tx=0x7f1b40011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40003b60 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4001a910 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40024930 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4419c6a0 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.513+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4419cb00 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1b40022020 con 0x7f1b44108a90 2024-06-06T07:36:47.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b3affd700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b3003bc70 con 0x7f1b300390a0 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b4004d3f0 con 0x7f1b44108a90 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b49271700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.514+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.715+0000 7f1b49271700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:41.715+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:42.116+0000 7f1b49271700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:42.116+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:42.917+0000 7f1b49271700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:42.917+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T07:36:47.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:44.518+0000 7f1b49271700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/1605115594 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:44.518+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:46.053+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f1b4001ab30 con 0x7f1b44108a90 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:46.053+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 msgr2=0x7f1b3003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:46.053+0000 7f1b3affd700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.053+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f1b4004f470 con 0x7f1b44108a90 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.053+0000 7f1b3affd700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 0x7f1b3003ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.053+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b40012ea0 con 0x7f1b3003cb60 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.054+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 0x7f1b3003ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.054+0000 7f1b49271700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 0x7f1b3003ef50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b34003a10 tx=0x7f1b340092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:47.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.055+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f1b40012ea0 con 0x7f1b3003cb60 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.061+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1b44063af0 con 0x7f1b3003cb60 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b3affd700 1 -- 172.21.15.87:0/736195262 <== mgr.14150 v2:172.21.15.87:6800/2460003046 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1b44063af0 con 0x7f1b3003cb60 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 msgr2=0x7f1b3003ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 0x7f1b3003ef50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b34003a10 tx=0x7f1b340092b0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 msgr2=0x7f1b4419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b40011b60 tx=0x7f1b40011c40 comp rx=0 tx=0).stop 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 shutdown_connections 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1b3003cb60 0x7f1b3003ef50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b44108a90 0x7f1b4419bed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.685 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 --2- 172.21.15.87:0/736195262 >> [v2:172.21.15.87:6800/1605115594,v1:172.21.15.87:6801/1605115594] conn(0x7f1b300390a0 0x7f1b3003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:47.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 >> 172.21.15.87:0/736195262 conn(0x7f1b441024c0 msgr2=0x7f1b44102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:47.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 shutdown_connections 2024-06-06T07:36:47.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:47.062+0000 7f1b4bcd6700 1 -- 172.21.15.87:0/736195262 wait complete. 2024-06-06T07:36:47.686 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 9 is available 2024-06-06T07:36:47.686 INFO:teuthology.orchestra.run.smithi087.stdout:Generating a dashboard self-signed certificate... 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:46] ENGINE Bus STARTING 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:46] ENGINE Serving on https://172.21.15.87:7150 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:46] ENGINE Serving on http://172.21.15.87:8765 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: [06/Jun/2024:07:36:46] ENGINE Bus STARTED 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T07:36:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:48 smithi087 ceph-mon[32276]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T07:36:49.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-06T07:36:49.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.372+0000 7f48f6627700 1 Processor -- start 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.372+0000 7f48f6627700 1 -- start start 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48f6627700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48f6627700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0108a60 con 0x7f48f0108070 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36818/0 (socket says 172.21.15.87:36818) 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48effff700 1 -- 172.21.15.87:0/1121683012 learned_addr learned my addr 172.21.15.87:0/1121683012 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48effff700 1 -- 172.21.15.87:0/1121683012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f0109220 con 0x7f48f0108070 2024-06-06T07:36:49.252 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48effff700 1 --2- 172.21.15.87:0/1121683012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48e00098a0 tx=0x7f48e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba73144cc8d5a9f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48eeffd700 1 -- 172.21.15.87:0/1121683012 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e0004160 con 0x7f48f0108070 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48eeffd700 1 -- 172.21.15.87:0/1121683012 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e0004300 con 0x7f48f0108070 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.373+0000 7f48eeffd700 1 -- 172.21.15.87:0/1121683012 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e0011b20 con 0x7f48f0108070 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 -- 172.21.15.87:0/1121683012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 msgr2=0x7f48f0108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 --2- 172.21.15.87:0/1121683012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48e00098a0 tx=0x7f48e0009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:49.253 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 -- 172.21.15.87:0/1121683012 shutdown_connections 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 --2- 172.21.15.87:0/1121683012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f0108490 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 -- 172.21.15.87:0/1121683012 >> 172.21.15.87:0/1121683012 conn(0x7f48f0103610 msgr2=0x7f48f0105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 -- 172.21.15.87:0/1121683012 shutdown_connections 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 -- 172.21.15.87:0/1121683012 wait complete. 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.374+0000 7f48f6627700 1 Processor -- start 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48f6627700 1 -- start start 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48f6627700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:49.254 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48f6627700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c5d0 con 0x7f48f0108070 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36826/0 (socket says 172.21.15.87:36826) 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48effff700 1 -- 172.21.15.87:0/1943319622 learned_addr learned my addr 172.21.15.87:0/1943319622 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48effff700 1 -- 172.21.15.87:0/1943319622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f007e930 con 0x7f48f0108070 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.375+0000 7f48effff700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48e00116a0 tx=0x7f48e0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.376+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e0003a40 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.376+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f007cb20 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.376+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e0003be0 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.376+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e001a720 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.376+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f007cfe0 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f48e001aa20 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f0063af0 con 0x7f48f0108070 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48ed7fa700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 0x7f48d803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:49.256 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48e004d810 con 0x7f48f0108070 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48ef7fe700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 0x7f48d803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.377+0000 7f48ef7fe700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 0x7f48d803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48e4006fd0 tx=0x7f48e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.382+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f48e0011e20 con 0x7f48f0108070 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.582+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f48f007d980 con 0x7f48d8039050 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.762+0000 7f48ed7fa700 1 -- 172.21.15.87:0/1943319622 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f48f007d980 con 0x7f48d8039050 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 msgr2=0x7f48d803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:49.257 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 0x7f48d803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48e4006fd0 tx=0x7f48e4006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 msgr2=0x7f48f007e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48e00116a0 tx=0x7f48e0011ba0 comp rx=0 tx=0).stop 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 shutdown_connections 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48d8039050 0x7f48d803b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 --2- 172.21.15.87:0/1943319622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0108070 0x7f48f007e3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 >> 172.21.15.87:0/1943319622 conn(0x7f48f0103610 msgr2=0x7f48f0104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:49.258 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 shutdown_connections 2024-06-06T07:36:49.259 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:48.765+0000 7f48f6627700 1 -- 172.21.15.87:0/1943319622 wait complete. 2024-06-06T07:36:49.259 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial admin user... 2024-06-06T07:36:49.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:49 smithi087 ceph-mon[32276]: mgrmap e12: x(active, since 2s) 2024-06-06T07:36:49.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:49.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:49.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:50.263 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:50 smithi087 ceph-mon[32276]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:50.263 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:51.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:51 smithi087 ceph-mon[32276]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:36:51.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:51.418 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jZlRt9HxONmFCTb/YanROe0hMmni3iPQHZRh/5lZcyJ38R4zIM6JK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717659410, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-06T07:36:51.418 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.689+0000 7f48a0aa9700 1 Processor -- start 2024-06-06T07:36:51.418 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f48a0aa9700 1 -- start start 2024-06-06T07:36:51.418 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f48a0aa9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:51.418 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f48a0aa9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f489c107850 con 0x7f489c106ef0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f489a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f489a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36834/0 (socket says 172.21.15.87:36834) 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f489a59c700 1 -- 172.21.15.87:0/1271371270 learned_addr learned my addr 172.21.15.87:0/1271371270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.690+0000 7f489a59c700 1 -- 172.21.15.87:0/1271371270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c108090 con 0x7f489c106ef0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f489a59c700 1 --2- 172.21.15.87:0/1271371270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098a0 tx=0x7f488c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a113515df23b0ff0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f489959a700 1 -- 172.21.15.87:0/1271371270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f488c004160 con 0x7f489c106ef0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f489959a700 1 -- 172.21.15.87:0/1271371270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c004300 con 0x7f489c106ef0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f489959a700 1 -- 172.21.15.87:0/1271371270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f488c011b20 con 0x7f489c106ef0 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1271371270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 msgr2=0x7f489c107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.691+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1271371270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098a0 tx=0x7f488c009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.692+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1271371270 shutdown_connections 2024-06-06T07:36:51.419 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.692+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1271371270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c107310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.692+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1271371270 >> 172.21.15.87:0/1271371270 conn(0x7f489c102560 msgr2=0x7f489c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.692+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1271371270 shutdown_connections 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.692+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1271371270 wait complete. 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f48a0aa9700 1 Processor -- start 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f48a0aa9700 1 -- start start 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f48a0aa9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f48a0aa9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f489c07b850 con 0x7f489c106ef0 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f489a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f489a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36848/0 (socket says 172.21.15.87:36848) 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f489a59c700 1 -- 172.21.15.87:0/1259816857 learned_addr learned my addr 172.21.15.87:0/1259816857 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f489a59c700 1 -- 172.21.15.87:0/1259816857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c19a180 con 0x7f489c106ef0 2024-06-06T07:36:51.420 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.693+0000 7f489a59c700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0117e0 tx=0x7f488c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.694+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f488c003bf0 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.694+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c01a910 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.694+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c19a410 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.694+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f488c0249a0 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.694+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c19a870 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f488c022020 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c063af0 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f488b7fe700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 0x7f488403d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f488c04d510 con 0x7f489c106ef0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f4899d9b700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 0x7f488403d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.695+0000 7f4899d9b700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 0x7f488403d740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:51.421 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.700+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f488c011e20 con 0x7f489c106ef0 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:49.928+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f489c19d280 con 0x7f488403b280 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.152+0000 7f488b7fe700 1 -- 172.21.15.87:0/1259816857 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f489c19d280 con 0x7f488403b280 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 msgr2=0x7f488403d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 0x7f488403d740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 msgr2=0x7f489c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0117e0 tx=0x7f488c011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 shutdown_connections 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f488403b280 0x7f488403d740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 --2- 172.21.15.87:0/1259816857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f489c106ef0 0x7f489c199c40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 >> 172.21.15.87:0/1259816857 conn(0x7f489c102560 msgr2=0x7f489c1901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 shutdown_connections 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:50.154+0000 7f48a0aa9700 1 -- 172.21.15.87:0/1259816857 wait complete. 2024-06-06T07:36:51.422 INFO:teuthology.orchestra.run.smithi087.stdout:Fetching dashboard port number... 2024-06-06T07:36:52.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:52 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1496209089' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-06T07:36:52.650 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 8443 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe53496c700 1 Processor -- start 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe53496c700 1 -- start start 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe53496c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe53496c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe530108a10 con 0x7fe530108020 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36860/0 (socket says 172.21.15.87:36860) 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.881+0000 7fe52e59c700 1 -- 172.21.15.87:0/1505503112 learned_addr learned my addr 172.21.15.87:0/1505503112 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe52e59c700 1 -- 172.21.15.87:0/1505503112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe530109220 con 0x7fe530108020 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe52e59c700 1 --2- 172.21.15.87:0/1505503112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098a0 tx=0x7fe520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25a8ea6452fe84d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe52d59a700 1 -- 172.21.15.87:0/1505503112 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520004160 con 0x7fe530108020 2024-06-06T07:36:52.651 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe52d59a700 1 -- 172.21.15.87:0/1505503112 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520004300 con 0x7fe530108020 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe52d59a700 1 -- 172.21.15.87:0/1505503112 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520011b20 con 0x7fe530108020 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe53496c700 1 -- 172.21.15.87:0/1505503112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 msgr2=0x7fe530108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.882+0000 7fe53496c700 1 --2- 172.21.15.87:0/1505503112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098a0 tx=0x7fe520009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 -- 172.21.15.87:0/1505503112 shutdown_connections 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 --2- 172.21.15.87:0/1505503112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe530108440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 -- 172.21.15.87:0/1505503112 >> 172.21.15.87:0/1505503112 conn(0x7fe5301035e0 msgr2=0x7fe530105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 -- 172.21.15.87:0/1505503112 shutdown_connections 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 -- 172.21.15.87:0/1505503112 wait complete. 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 Processor -- start 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.883+0000 7fe53496c700 1 -- start start 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe53496c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe53496c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010c5d0 con 0x7fe530108020 2024-06-06T07:36:52.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36870/0 (socket says 172.21.15.87:36870) 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe52e59c700 1 -- 172.21.15.87:0/1496209089 learned_addr learned my addr 172.21.15.87:0/1496209089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe52e59c700 1 -- 172.21.15.87:0/1496209089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53019c400 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe52e59c700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5200116a0 tx=0x7fe520011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520003a40 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53019c690 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520003be0 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe52001a720 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.884+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53019caf0 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.885+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe52001aa20 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.885+0000 7fe51f7fe700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 0x7fe51803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.885+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe52004d8a0 con 0x7fe530108020 2024-06-06T07:36:52.653 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.886+0000 7fe52dd9b700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 0x7fe51803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.885+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe530063af0 con 0x7fe530108020 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.886+0000 7fe52dd9b700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 0x7fe51803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:51.891+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe520011e20 con 0x7fe530108020 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.088+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe53019f380 con 0x7fe530108020 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.088+0000 7fe51f7fe700 1 -- 172.21.15.87:0/1496209089 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe52004d130 con 0x7fe530108020 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.090+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 msgr2=0x7fe51803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.090+0000 7fe53496c700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 0x7fe51803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.090+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 msgr2=0x7fe53019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.090+0000 7fe53496c700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5200116a0 tx=0x7fe520011ba0 comp rx=0 tx=0).stop 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 shutdown_connections 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe518039050 0x7fe51803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:52.654 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 --2- 172.21.15.87:0/1496209089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe530108020 0x7fe53019bec0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:52.655 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 >> 172.21.15.87:0/1496209089 conn(0x7fe5301035e0 msgr2=0x7fe530104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:52.655 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 shutdown_connections 2024-06-06T07:36:52.655 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:52.091+0000 7fe53496c700 1 -- 172.21.15.87:0/1496209089 wait complete. 2024-06-06T07:36:52.658 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T07:36:52.658 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-06-06T07:36:52.665 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T07:36:52.665 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-06-06T07:36:52.665 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-06-06T07:36:52.665 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph Dashboard is now available at: 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout: URL: https://smithi087.front.sepia.ceph.com:8443/ 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout: User: admin 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout: Password: evq3bzx7ah 2024-06-06T07:36:52.667 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:52.668 INFO:teuthology.orchestra.run.smithi087.stdout:Saving cluster configuration to /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config directory 2024-06-06T07:36:52.668 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling autotune for osd_memory_target 2024-06-06T07:36:53.598 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:53 smithi087 ceph-mon[32276]: mgrmap e13: x(active, since 6s) 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.098+0000 7ff4efd6d700 1 Processor -- start 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4efd6d700 1 -- start start 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4efd6d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4efd6d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e807c6e0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4edb09700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4edb09700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36882/0 (socket says 172.21.15.87:36882) 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4edb09700 1 -- 172.21.15.87:0/13357691 learned_addr learned my addr 172.21.15.87:0/13357691 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:53.825 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4edb09700 1 -- 172.21.15.87:0/13357691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e810b3e0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.099+0000 7ff4edb09700 1 --2- 172.21.15.87:0/13357691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00098a0 tx=0x7ff4e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f20e9a4b746105 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4ecb07700 1 -- 172.21.15.87:0/13357691 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e0004160 con 0x7ff4e8108ab0 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4ecb07700 1 -- 172.21.15.87:0/13357691 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e0004300 con 0x7ff4e8108ab0 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4ecb07700 1 -- 172.21.15.87:0/13357691 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e0011b20 con 0x7ff4e8108ab0 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 -- 172.21.15.87:0/13357691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 msgr2=0x7ff4e810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/13357691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00098a0 tx=0x7ff4e0009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 -- 172.21.15.87:0/13357691 shutdown_connections 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/13357691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e810aea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 -- 172.21.15.87:0/13357691 >> 172.21.15.87:0/13357691 conn(0x7ff4e8102560 msgr2=0x7ff4e8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:53.826 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 -- 172.21.15.87:0/13357691 shutdown_connections 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.100+0000 7ff4efd6d700 1 -- 172.21.15.87:0/13357691 wait complete. 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4efd6d700 1 Processor -- start 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4efd6d700 1 -- start start 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4efd6d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4efd6d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e810c440 con 0x7ff4e8108ab0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4edb09700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4edb09700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36894/0 (socket says 172.21.15.87:36894) 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4edb09700 1 -- 172.21.15.87:0/1409246150 learned_addr learned my addr 172.21.15.87:0/1409246150 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.101+0000 7ff4edb09700 1 -- 172.21.15.87:0/1409246150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e819c360 con 0x7ff4e8108ab0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4edb09700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00117e0 tx=0x7ff4e0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e0003ba0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e001a910 con 0x7ff4e8108ab0 2024-06-06T07:36:53.827 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e819c5f0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e00248f0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.102+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e819ca50 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff4e0022020 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4e6ffd700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 0x7ff4d003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e8063af0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4e004daa0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4ed308700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 0x7ff4d003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.103+0000 7ff4ed308700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 0x7ff4d003b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.108+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4e00212c0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.304+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff4e819f420 con 0x7ff4e8108ab0 2024-06-06T07:36:53.828 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.304+0000 7ff4e6ffd700 1 -- 172.21.15.87:0/1409246150 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff4e00151c0 con 0x7ff4e8108ab0 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 msgr2=0x7ff4d003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 0x7ff4d003b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 msgr2=0x7ff4e819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00117e0 tx=0x7ff4e0011ba0 comp rx=0 tx=0).stop 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 shutdown_connections 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4d0039140 0x7ff4d003b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 --2- 172.21.15.87:0/1409246150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4e8108ab0 0x7ff4e819be20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 >> 172.21.15.87:0/1409246150 conn(0x7ff4e8102560 msgr2=0x7ff4e8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 shutdown_connections 2024-06-06T07:36:53.829 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:53.306+0000 7ff4efd6d700 1 -- 172.21.15.87:0/1409246150 wait complete. 2024-06-06T07:36:55.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.277+0000 7ff8a628e700 1 Processor -- start 2024-06-06T07:36:55.179 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.277+0000 7ff8a628e700 1 -- start start 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.277+0000 7ff8a628e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.277+0000 7ff8a628e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0108a10 con 0x7ff8a0108020 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.277+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36910/0 (socket says 172.21.15.87:36910) 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89ffff700 1 -- 172.21.15.87:0/1154888552 learned_addr learned my addr 172.21.15.87:0/1154888552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89ffff700 1 -- 172.21.15.87:0/1154888552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a0109220 con 0x7ff8a0108020 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89ffff700 1 --2- 172.21.15.87:0/1154888552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098a0 tx=0x7ff890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb87b6a2a7dfa08 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89f7fe700 1 -- 172.21.15.87:0/1154888552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890004160 con 0x7ff8a0108020 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89f7fe700 1 -- 172.21.15.87:0/1154888552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff890004300 con 0x7ff8a0108020 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.278+0000 7ff89f7fe700 1 -- 172.21.15.87:0/1154888552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890011b20 con 0x7ff8a0108020 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- 172.21.15.87:0/1154888552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 msgr2=0x7ff8a0108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:55.180 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1154888552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098a0 tx=0x7ff890009bb0 comp rx=0 tx=0).stop 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- 172.21.15.87:0/1154888552 shutdown_connections 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1154888552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a0108440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- 172.21.15.87:0/1154888552 >> 172.21.15.87:0/1154888552 conn(0x7ff8a01035e0 msgr2=0x7ff8a0105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- 172.21.15.87:0/1154888552 shutdown_connections 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- 172.21.15.87:0/1154888552 wait complete. 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 Processor -- start 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.279+0000 7ff8a628e700 1 -- start start 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff8a628e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff8a628e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a010c5d0 con 0x7ff8a0108020 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36922/0 (socket says 172.21.15.87:36922) 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89ffff700 1 -- 172.21.15.87:0/1310923239 learned_addr learned my addr 172.21.15.87:0/1310923239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:55.181 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89ffff700 1 -- 172.21.15.87:0/1310923239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a019c400 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89ffff700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff890000c00 tx=0x7ff890011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890003b60 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89001a910 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890024930 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a019c690 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.280+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a019caf0 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.281+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff890022020 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.281+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a0063af0 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.281+0000 7ff89dffb700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 0x7ff88c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.281+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff89004d470 con 0x7ff8a0108020 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.282+0000 7ff897fff700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 0x7ff88c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.282+0000 7ff897fff700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 0x7ff88c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff888006fd0 tx=0x7ff888006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:55.182 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.286+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff89001ab70 con 0x7ff8a0108020 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.589+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff8a019f440 con 0x7ff8a0108020 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.592+0000 7ff89dffb700 1 -- 172.21.15.87:0/1310923239 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7ff89002d3f0 con 0x7ff8a0108020 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 msgr2=0x7ff88c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 0x7ff88c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff888006fd0 tx=0x7ff888006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 msgr2=0x7ff8a019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff890000c00 tx=0x7ff890011c10 comp rx=0 tx=0).stop 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 shutdown_connections 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff88c039140 0x7ff88c03b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 --2- 172.21.15.87:0/1310923239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a0108020 0x7ff8a019bec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 >> 172.21.15.87:0/1310923239 conn(0x7ff8a01035e0 msgr2=0x7ff8a0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 shutdown_connections 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-06-06T07:36:54.594+0000 7ff8a628e700 1 -- 172.21.15.87:0/1310923239 wait complete. 2024-06-06T07:36:55.183 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout:Or, if you are only running a single cluster on this host: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: ceph telemetry on 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout:For more information see: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.184 INFO:teuthology.orchestra.run.smithi087.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-06T07:36:55.185 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:36:55.185 INFO:teuthology.orchestra.run.smithi087.stdout:Bootstrap complete. 2024-06-06T07:36:55.217 INFO:tasks.cephadm:Fetching config... 2024-06-06T07:36:55.217 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:36:55.217 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-06T07:36:55.234 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-06T07:36:55.234 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:36:55.234 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-06T07:36:55.289 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-06T07:36:55.289 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:36:55.289 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-06T07:36:55.354 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-06T07:36:55.354 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:36:55.354 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-06T07:36:55.410 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-06T07:36:55.410 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T07:36:55.502 INFO:teuthology.orchestra.run.smithi087.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:55.514 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T07:36:55.552 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:55.564 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T07:36:55.605 INFO:teuthology.orchestra.run.smithi195.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClL3WaZZsWomQpL0U2VYRAsgtjYgP7Lk9UqLYQsnA6eyaz1ckJrc8kK5/ioi3p+iDVyXDjCeSL+8fS1NCq1cRH0ud1stOiWfUWe5iMnaM16qp23JG6xcC9km8311TjFiBico3UCV8j0yPb4P3nR4y51qQK9hsmwgdZx2v8l3HAQy8qLV7KqfBvjEFj4BvBBf0PNuxh1n6eidcUcu1Hrqxv+NNYIodbhnXKC19SbW2Bon4w1dClivTFo7I533Tsq4+AqtHqwAr74DT3PQlODxLa1hrF9KJnKaFyiixS0AtD2430gfPvKz+3r0EUajyk1rCbXJEooa0NSDkZzrlFqjEGQlo1hhrsjUJTAtmI1Hlb/H/kS9c8tVa2pZQ+kqtgpv83hFvpRScx5q2P8mI0Q5dPPRRc9Q7BbSRScJetX+viUGotY8mr+o1CM2WZM13GqiiGrAOAGZyHbYAeo7iWWcVLx2vSj88uGeXw1P7wZRZxR2HhobpzDmewZNFxohewdIk= ceph-3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:36:55.616 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-06T07:36:55.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:55 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1310923239' entity='client.admin' 2024-06-06T07:36:55.861 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 -- 172.21.15.87:0/3513149995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 msgr2=0x7fc220100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3513149995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc220100600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc214009910 tx=0x7fc214009c20 comp rx=0 tx=0).stop 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 -- 172.21.15.87:0/3513149995 shutdown_connections 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3513149995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc220100600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 -- 172.21.15.87:0/3513149995 >> 172.21.15.87:0/3513149995 conn(0x7fc2200fbb40 msgr2=0x7fc2200fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 -- 172.21.15.87:0/3513149995 shutdown_connections 2024-06-06T07:36:57.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.943+0000 7fc227ac9700 1 -- 172.21.15.87:0/3513149995 wait complete. 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.944+0000 7fc227ac9700 1 Processor -- start 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.944+0000 7fc227ac9700 1 -- start start 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.944+0000 7fc227ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.944+0000 7fc227ac9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc220107fb0 con 0x7fc2201001e0 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc225865700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc225865700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47476/0 (socket says 172.21.15.87:47476) 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc225865700 1 -- 172.21.15.87:0/3392585712 learned_addr learned my addr 172.21.15.87:0/3392585712 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc225865700 1 -- 172.21.15.87:0/3392585712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc220197b30 con 0x7fc2201001e0 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc225865700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc214006010 tx=0x7fc214005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc214027070 con 0x7fc2201001e0 2024-06-06T07:36:57.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc21400be40 con 0x7fc2201001e0 2024-06-06T07:36:57.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21400f460 con 0x7fc2201001e0 2024-06-06T07:36:57.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc220197dc0 con 0x7fc2201001e0 2024-06-06T07:36:57.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.945+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22019aab0 con 0x7fc2201001e0 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.946+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc21402a070 con 0x7fc2201001e0 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.946+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc220108730 con 0x7fc2201001e0 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.946+0000 7fc212ffd700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 0x7fc20c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.946+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc21401e030 con 0x7fc2201001e0 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.946+0000 7fc225064700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 0x7fc20c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:36:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.947+0000 7fc225064700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 0x7fc20c03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc21c006fd0 tx=0x7fc21c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:36:57.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:57.951+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc214013070 con 0x7fc2201001e0 2024-06-06T07:36:58.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.146+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc22002ddb0 con 0x7fc2201001e0 2024-06-06T07:36:58.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.148+0000 7fc212ffd700 1 -- 172.21.15.87:0/3392585712 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc214031020 con 0x7fc2201001e0 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.150+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 msgr2=0x7fc20c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.150+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 0x7fc20c03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc21c006fd0 tx=0x7fc21c006e40 comp rx=0 tx=0).stop 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.150+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 msgr2=0x7fc2201975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.150+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc214006010 tx=0x7fc214005050 comp rx=0 tx=0).stop 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 shutdown_connections 2024-06-06T07:36:58.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc20c0390f0 0x7fc20c03b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:58.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 --2- 172.21.15.87:0/3392585712 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2201001e0 0x7fc2201975f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:36:58.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 >> 172.21.15.87:0/3392585712 conn(0x7fc2200fbb40 msgr2=0x7fc2200fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:36:58.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 shutdown_connections 2024-06-06T07:36:58.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:36:58.151+0000 7fc227ac9700 1 -- 172.21.15.87:0/3392585712 wait complete. 2024-06-06T07:36:58.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3392585712' entity='client.admin' 2024-06-06T07:36:58.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:36:58.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:36:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:36:58.763 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-06T07:36:58.763 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-06T07:36:58.994 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:00.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 -- 172.21.15.87:0/2643211394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 msgr2=0x7f59bc103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:00.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 --2- 172.21.15.87:0/2643211394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc103980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009910 tx=0x7f59a4009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:00.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 -- 172.21.15.87:0/2643211394 shutdown_connections 2024-06-06T07:37:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 --2- 172.21.15.87:0/2643211394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc103980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 -- 172.21.15.87:0/2643211394 >> 172.21.15.87:0/2643211394 conn(0x7f59bc0fd4e0 msgr2=0x7f59bc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 -- 172.21.15.87:0/2643211394 shutdown_connections 2024-06-06T07:37:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.483+0000 7f59c107e700 1 -- 172.21.15.87:0/2643211394 wait complete. 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59c107e700 1 Processor -- start 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59c107e700 1 -- start start 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59c107e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59c107e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc108030 con 0x7f59bc103560 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59bad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59bad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47496/0 (socket says 172.21.15.87:47496) 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59bad9d700 1 -- 172.21.15.87:0/1559367235 learned_addr learned my addr 172.21.15.87:0/1559367235 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.484+0000 7f59bad9d700 1 -- 172.21.15.87:0/1559367235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc197c40 con 0x7f59bc103560 2024-06-06T07:37:00.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59bad9d700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59a4004fa0 tx=0x7f59a4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a4027070 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc197ed0 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a400be40 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a400f460 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.485+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc19abc0 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.486+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f59a402a070 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.486+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc1087b0 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.486+0000 7f59b3fff700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 0x7f59a803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.486+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59a401e030 con 0x7f59bc103560 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.486+0000 7f59ba59c700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 0x7f59a803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.487+0000 7f59ba59c700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 0x7f59a803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:00.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.491+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f59a4013070 con 0x7f59bc103560 2024-06-06T07:37:00.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.687+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f59bc0628d0 con 0x7f59a80390f0 2024-06-06T07:37:00.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.690+0000 7f59b3fff700 1 -- 172.21.15.87:0/1559367235 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59bc0628d0 con 0x7f59a80390f0 2024-06-06T07:37:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 msgr2=0x7f59a803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:00.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 0x7f59a803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 msgr2=0x7f59bc197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59a4004fa0 tx=0x7f59a4005080 comp rx=0 tx=0).stop 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 shutdown_connections 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f59a80390f0 0x7f59a803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 --2- 172.21.15.87:0/1559367235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59bc103560 0x7f59bc197700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 >> 172.21.15.87:0/1559367235 conn(0x7f59bc0fd4e0 msgr2=0x7f59bc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 shutdown_connections 2024-06-06T07:37:00.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:00.693+0000 7f59c107e700 1 -- 172.21.15.87:0/1559367235 wait complete. 2024-06-06T07:37:01.545 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-06-06T07:37:01.545 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:37:01.545 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-06-06T07:37:01.564 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:37:01.564 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:01.621 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-06-06T07:37:01.622 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch host add smithi181 2024-06-06T07:37:01.855 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:37:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:01.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:01.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:01.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:01.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:02.969 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:02 smithi087 ceph-mon[32276]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:37:03.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 -- 172.21.15.87:0/2530583582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 msgr2=0x7f31c0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:03.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 --2- 172.21.15.87:0/2530583582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c0106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00c210 tx=0x7f31bc00c520 comp rx=0 tx=0).stop 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 -- 172.21.15.87:0/2530583582 shutdown_connections 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 --2- 172.21.15.87:0/2530583582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c0106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 -- 172.21.15.87:0/2530583582 >> 172.21.15.87:0/2530583582 conn(0x7f31c0078320 msgr2=0x7f31c0078730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 -- 172.21.15.87:0/2530583582 shutdown_connections 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.346+0000 7f31c802a700 1 -- 172.21.15.87:0/2530583582 wait complete. 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c802a700 1 Processor -- start 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c802a700 1 -- start start 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c802a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c802a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010a2c0 con 0x7f31c0106590 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c5dc6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c5dc6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47518/0 (socket says 172.21.15.87:47518) 2024-06-06T07:37:03.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c5dc6700 1 -- 172.21.15.87:0/629987815 learned_addr learned my addr 172.21.15.87:0/629987815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c5dc6700 1 -- 172.21.15.87:0/629987815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01974c0 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c5dc6700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31bc016040 tx=0x7f31bc008d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc023070 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc003b30 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.347+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c0195b70 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.348+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc018070 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.348+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c0196030 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.348+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f31bc026070 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.348+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c0101c70 con 0x7f31c0106590 2024-06-06T07:37:03.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.349+0000 7f31b6ffd700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 0x7f31ac03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:03.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.349+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f31bc010050 con 0x7f31c0106590 2024-06-06T07:37:03.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.349+0000 7f31c55c5700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 0x7f31ac03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:03.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.349+0000 7f31c55c5700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 0x7f31ac03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006fd0 tx=0x7f31b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:03.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.353+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f31bc01c360 con 0x7f31c0106590 2024-06-06T07:37:03.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:03.555+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) -- 0x7f31c0196b70 con 0x7f31ac039150 2024-06-06T07:37:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:04 smithi087 ceph-mon[32276]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:05 smithi087 ceph-mon[32276]: Deploying cephadm binary to smithi181 2024-06-06T07:37:07.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:07 smithi087 ceph-mon[32276]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:07.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.676+0000 7f31b6ffd700 1 -- 172.21.15.87:0/629987815 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f31c0196b70 con 0x7f31ac039150 2024-06-06T07:37:07.677 INFO:teuthology.orchestra.run.smithi087.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-06-06T07:37:07.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 msgr2=0x7f31ac03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:07.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 0x7f31ac03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006fd0 tx=0x7f31b0006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:07.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 msgr2=0x7f31c01955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:07.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31bc016040 tx=0x7f31bc008d60 comp rx=0 tx=0).stop 2024-06-06T07:37:07.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 shutdown_connections 2024-06-06T07:37:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31ac039150 0x7f31ac03b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 --2- 172.21.15.87:0/629987815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0106590 0x7f31c01955d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 >> 172.21.15.87:0/629987815 conn(0x7f31c0078320 msgr2=0x7f31c00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 shutdown_connections 2024-06-06T07:37:07.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:07.678+0000 7f31c802a700 1 -- 172.21.15.87:0/629987815 wait complete. 2024-06-06T07:37:08.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T07:37:08.523 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:08.948 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:08.948 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:08 smithi087 ceph-mon[32276]: Added host smithi181 2024-06-06T07:37:08.948 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:08.948 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:08 smithi087 ceph-mon[32276]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:08.948 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 -- 172.21.15.87:0/12410611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 msgr2=0x7fd990074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 --2- 172.21.15.87:0/12410611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd990074c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd984009910 tx=0x7fd984009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 -- 172.21.15.87:0/12410611 shutdown_connections 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 --2- 172.21.15.87:0/12410611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd990074c10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 -- 172.21.15.87:0/12410611 >> 172.21.15.87:0/12410611 conn(0x7fd9900fd9d0 msgr2=0x7fd9900ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:09.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 -- 172.21.15.87:0/12410611 shutdown_connections 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.946+0000 7fd996ed3700 1 -- 172.21.15.87:0/12410611 wait complete. 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd996ed3700 1 Processor -- start 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd996ed3700 1 -- start start 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd996ed3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd996ed3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd990112b40 con 0x7fd990074830 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd994c6f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd994c6f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41230/0 (socket says 172.21.15.87:41230) 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.947+0000 7fd994c6f700 1 -- 172.21.15.87:0/586758881 learned_addr learned my addr 172.21.15.87:0/586758881 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd994c6f700 1 -- 172.21.15.87:0/586758881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99019e5b0 con 0x7fd990074830 2024-06-06T07:37:09.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd994c6f700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd984006010 tx=0x7fd984005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd984026070 con 0x7fd990074830 2024-06-06T07:37:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99019e840 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98400bc30 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd984003ab0 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.948+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99019eca0 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.949+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd984029070 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.949+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd990075f70 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.949+0000 7fd98dffb700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 0x7fd98003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.949+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd98401e070 con 0x7fd990074830 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.949+0000 7fd98ffff700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 0x7fd98003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.950+0000 7fd98ffff700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 0x7fd98003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006fd0 tx=0x7fd97c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:09.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:09.954+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd984013070 con 0x7fd990074830 2024-06-06T07:37:10.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:10.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.156+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd99019f3d0 con 0x7fd9800390f0 2024-06-06T07:37:10.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.157+0000 7fd98dffb700 1 -- 172.21.15.87:0/586758881 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fd99019f3d0 con 0x7fd9800390f0 2024-06-06T07:37:10.158 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:37:10.158 INFO:teuthology.orchestra.run.smithi087.stdout:[{"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-06-06T07:37:10.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 msgr2=0x7fd98003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:10.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 0x7fd98003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006fd0 tx=0x7fd97c006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:10.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 msgr2=0x7fd99019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:10.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd984006010 tx=0x7fd984005050 comp rx=0 tx=0).stop 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 shutdown_connections 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd9800390f0 0x7fd98003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 --2- 172.21.15.87:0/586758881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd990074830 0x7fd99019e070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 >> 172.21.15.87:0/586758881 conn(0x7fd9900fd9d0 msgr2=0x7fd9900ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 shutdown_connections 2024-06-06T07:37:10.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:10.160+0000 7fd996ed3700 1 -- 172.21.15.87:0/586758881 wait complete. 2024-06-06T07:37:10.693 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi195 2024-06-06T07:37:10.693 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:37:10.693 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.conf 2024-06-06T07:37:10.711 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:37:10.711 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:10.768 INFO:tasks.cephadm:Adding host smithi195 to orchestrator... 2024-06-06T07:37:10.768 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch host add smithi195 2024-06-06T07:37:11.004 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:11.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:10 smithi087 ceph-mon[32276]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:11.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:10 smithi087 ceph-mon[32276]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:37:11.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.492+0000 7fb001b45700 1 -- 172.21.15.87:0/2604672619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 msgr2=0x7faffc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.492+0000 7fb001b45700 1 --2- 172.21.15.87:0/2604672619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc108290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fafec00b0a0 tx=0x7fafec00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.492+0000 7fb001b45700 1 -- 172.21.15.87:0/2604672619 shutdown_connections 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 --2- 172.21.15.87:0/2604672619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc108290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 -- 172.21.15.87:0/2604672619 >> 172.21.15.87:0/2604672619 conn(0x7faffc078320 msgr2=0x7faffc078730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 -- 172.21.15.87:0/2604672619 shutdown_connections 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 -- 172.21.15.87:0/2604672619 wait complete. 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 Processor -- start 2024-06-06T07:37:12.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.493+0000 7fb001b45700 1 -- start start 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7fb001b45700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7fb001b45700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faffc10c4d0 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faffb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faffb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41248/0 (socket says 172.21.15.87:41248) 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faffb7fe700 1 -- 172.21.15.87:0/3011611471 learned_addr learned my addr 172.21.15.87:0/3011611471 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faffb7fe700 1 -- 172.21.15.87:0/3011611471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc103650 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faffb7fe700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec015040 tx=0x7fafec003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafec017040 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc101d00 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafec022070 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafec01d5d0 con 0x7faffc107eb0 2024-06-06T07:37:12.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.494+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc1021c0 con 0x7faffc107eb0 2024-06-06T07:37:12.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.495+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fafec01d8d0 con 0x7faffc107eb0 2024-06-06T07:37:12.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.495+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc06a070 con 0x7faffc107eb0 2024-06-06T07:37:12.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.495+0000 7faff8ff9700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 0x7fafe403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:12.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.495+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fafec00f050 con 0x7faffc107eb0 2024-06-06T07:37:12.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.495+0000 7faffaffd700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 0x7fafe403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:12.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.496+0000 7faffaffd700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 0x7fafe403b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:12.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.500+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fafec007e20 con 0x7faffc107eb0 2024-06-06T07:37:12.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:12.701+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}) -- 0x7faffc102d00 con 0x7fafe40390f0 2024-06-06T07:37:13.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:13 smithi087 ceph-mon[32276]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:15 smithi087 ceph-mon[32276]: Deploying cephadm binary to smithi195 2024-06-06T07:37:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:15 smithi087 ceph-mon[32276]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:37:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:15 smithi087 ceph-mon[32276]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:15 smithi087 ceph-mon[32276]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:15 smithi087 ceph-mon[32276]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.889+0000 7faff8ff9700 1 -- 172.21.15.87:0/3011611471 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7faffc102d00 con 0x7fafe40390f0 2024-06-06T07:37:16.890 INFO:teuthology.orchestra.run.smithi087.stdout:Added host 'smithi195' with addr '172.21.15.195' 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 msgr2=0x7fafe403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 0x7fafe403b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 msgr2=0x7faffc101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec015040 tx=0x7fafec003d50 comp rx=0 tx=0).stop 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 shutdown_connections 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fafe40390f0 0x7fafe403b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 --2- 172.21.15.87:0/3011611471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faffc107eb0 0x7faffc101760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 >> 172.21.15.87:0/3011611471 conn(0x7faffc078320 msgr2=0x7faffc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 shutdown_connections 2024-06-06T07:37:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:16.891+0000 7fb001b45700 1 -- 172.21.15.87:0/3011611471 wait complete. 2024-06-06T07:37:17.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:17.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:17.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:17.211 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:17.211 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:17.211 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:17.521 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T07:37:17.753 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:18.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:18 smithi087 ceph-mon[32276]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:37:18.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:18 smithi087 ceph-mon[32276]: Added host smithi195 2024-06-06T07:37:18.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:18.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:19.196 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:19 smithi087 ceph-mon[32276]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 -- 172.21.15.87:0/4289676906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 msgr2=0x7f9b301013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 --2- 172.21.15.87:0/4289676906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b301013e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009910 tx=0x7f9b24009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 -- 172.21.15.87:0/4289676906 shutdown_connections 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 --2- 172.21.15.87:0/4289676906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b301013e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 -- 172.21.15.87:0/4289676906 >> 172.21.15.87:0/4289676906 conn(0x7f9b300fc8b0 msgr2=0x7f9b300fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 -- 172.21.15.87:0/4289676906 shutdown_connections 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.222+0000 7f9b38698700 1 -- 172.21.15.87:0/4289676906 wait complete. 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b38698700 1 Processor -- start 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b38698700 1 -- start start 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b38698700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b38698700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010b3d0 con 0x7f9b30101000 2024-06-06T07:37:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b36434700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b36434700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47330/0 (socket says 172.21.15.87:47330) 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.223+0000 7f9b36434700 1 -- 172.21.15.87:0/1392217900 learned_addr learned my addr 172.21.15.87:0/1392217900 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b36434700 1 -- 172.21.15.87:0/1392217900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b30198dc0 con 0x7f9b30101000 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b36434700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005850 tx=0x7f9b24005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24026070 con 0x7f9b30101000 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2400bbd0 con 0x7f9b30101000 2024-06-06T07:37:19.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301990b0 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24003970 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.224+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b30199570 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9b24029070 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b301024e0 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b237fe700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 0x7f9b1c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b2401e070 con 0x7f9b30101000 2024-06-06T07:37:19.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b35c33700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 0x7f9b1c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.225+0000 7f9b35c33700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 0x7f9b1c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c006fd0 tx=0x7f9b2c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:19.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.230+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9b24013070 con 0x7f9b30101000 2024-06-06T07:37:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.431+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b3006a970 con 0x7f9b1c0390f0 2024-06-06T07:37:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.432+0000 7f9b237fe700 1 -- 172.21.15.87:0/1392217900 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f9b3006a970 con 0x7f9b1c0390f0 2024-06-06T07:37:19.433 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:37:19.433 INFO:teuthology.orchestra.run.smithi087.stdout:[{"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}, {"addr": "172.21.15.195", "hostname": "smithi195", "labels": [], "status": ""}] 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 msgr2=0x7f9b1c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 0x7f9b1c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c006fd0 tx=0x7f9b2c006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 msgr2=0x7f9b30198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005850 tx=0x7f9b24005030 comp rx=0 tx=0).stop 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 shutdown_connections 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9b1c0390f0 0x7f9b1c03b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 --2- 172.21.15.87:0/1392217900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30101000 0x7f9b30198880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 >> 172.21.15.87:0/1392217900 conn(0x7f9b300fc8b0 msgr2=0x7f9b30103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 shutdown_connections 2024-06-06T07:37:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:19.434+0000 7f9b38698700 1 -- 172.21.15.87:0/1392217900 wait complete. 2024-06-06T07:37:19.968 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-06T07:37:19.968 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-06T07:37:20.201 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:20 smithi087 ceph-mon[32276]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:37:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:20 smithi087 ceph-mon[32276]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 -- 172.21.15.87:0/2964596617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 msgr2=0x7f8af41024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/2964596617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af41024e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009910 tx=0x7f8ae4009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 -- 172.21.15.87:0/2964596617 shutdown_connections 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/2964596617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af41024e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 -- 172.21.15.87:0/2964596617 >> 172.21.15.87:0/2964596617 conn(0x7f8af40fd9b0 msgr2=0x7f8af40ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:21.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 -- 172.21.15.87:0/2964596617 shutdown_connections 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.673+0000 7f8afa4fc700 1 -- 172.21.15.87:0/2964596617 wait complete. 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8afa4fc700 1 Processor -- start 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8afa4fc700 1 -- start start 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8afa4fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8afa4fc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8af410c4d0 con 0x7f8af4102100 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47348/0 (socket says 172.21.15.87:47348) 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.674+0000 7f8af3fff700 1 -- 172.21.15.87:0/1320741609 learned_addr learned my addr 172.21.15.87:0/1320741609 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:21.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8af3fff700 1 -- 172.21.15.87:0/1320741609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af419dbc0 con 0x7f8af4102100 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8af3fff700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4005850 tx=0x7f8ae4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4026070 con 0x7f8af4102100 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af419deb0 con 0x7f8af4102100 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae400bbd0 con 0x7f8af4102100 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4003970 con 0x7f8af4102100 2024-06-06T07:37:21.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.675+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af4197800 con 0x7f8af4102100 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8ae4029070 con 0x7f8af4102100 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af41035e0 con 0x7f8af4102100 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8af17fa700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 0x7f8adc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ae401e070 con 0x7f8af4102100 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8af37fe700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 0x7f8adc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:21.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.676+0000 7f8af37fe700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 0x7f8adc03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006fd0 tx=0x7f8ae8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:21.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.681+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ae4013070 con 0x7f8af4102100 2024-06-06T07:37:21.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.883+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8af4198d30 con 0x7f8af4102100 2024-06-06T07:37:21.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.934+0000 7f8af17fa700 1 -- 172.21.15.87:0/1320741609 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8ae4030090 con 0x7f8af4102100 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 msgr2=0x7f8adc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 0x7f8adc03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006fd0 tx=0x7f8ae8006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 msgr2=0x7f8af419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4005850 tx=0x7f8ae4005030 comp rx=0 tx=0).stop 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 shutdown_connections 2024-06-06T07:37:21.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8adc0390f0 0x7f8adc03b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 --2- 172.21.15.87:0/1320741609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8af4102100 0x7f8af419d680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.936+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 >> 172.21.15.87:0/1320741609 conn(0x7f8af40fd9b0 msgr2=0x7f8af4104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.937+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 shutdown_connections 2024-06-06T07:37:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:21.937+0000 7f8afa4fc700 1 -- 172.21.15.87:0/1320741609 wait complete. 2024-06-06T07:37:21.938 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-06-06T07:37:22.345 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:21 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1320741609' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T07:37:22.540 INFO:tasks.cephadm:Adding mon.a on smithi087 2024-06-06T07:37:22.540 INFO:tasks.cephadm:Adding mon.b on smithi181 2024-06-06T07:37:22.540 INFO:tasks.cephadm:Adding mon.c on smithi195 2024-06-06T07:37:22.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c' 2024-06-06T07:37:22.812 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-06T07:37:22.861 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1320741609' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:23.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:37:23.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:23.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:24.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.481+0000 7fc8512e9700 1 -- 172.21.15.195:0/2166813201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 msgr2=0x7fc84c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.481+0000 7fc8512e9700 1 --2- 172.21.15.195:0/2166813201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c102510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009910 tx=0x7fc83c009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.481+0000 7fc8512e9700 1 -- 172.21.15.195:0/2166813201 shutdown_connections 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.481+0000 7fc8512e9700 1 --2- 172.21.15.195:0/2166813201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c102510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.481+0000 7fc8512e9700 1 -- 172.21.15.195:0/2166813201 >> 172.21.15.195:0/2166813201 conn(0x7fc84c0fd9e0 msgr2=0x7fc84c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 -- 172.21.15.195:0/2166813201 shutdown_connections 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 -- 172.21.15.195:0/2166813201 wait complete. 2024-06-06T07:37:24.483 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 Processor -- start 2024-06-06T07:37:24.484 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 -- start start 2024-06-06T07:37:24.484 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:24.484 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.482+0000 7fc8512e9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c10c500 con 0x7fc84c102130 2024-06-06T07:37:24.485 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc84affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:24.485 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc84affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.195:33824/0 (socket says 172.21.15.195:33824) 2024-06-06T07:37:24.485 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc84affd700 1 -- 172.21.15.195:0/691496340 learned_addr learned my addr 172.21.15.195:0/691496340 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:37:24.485 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc84affd700 1 -- 172.21.15.195:0/691496340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c19dc00 con 0x7fc84c102130 2024-06-06T07:37:24.485 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc84affd700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc83c005850 tx=0x7fc83c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc83c026070 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c19def0 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83c00bbd0 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc83c003970 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.483+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c197840 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.485+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc83c029070 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.484+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c103610 con 0x7fc84c102130 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.485+0000 7fc848ff9700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 0x7fc83803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:24.486 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.485+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc83c01e070 con 0x7fc84c102130 2024-06-06T07:37:24.487 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.485+0000 7fc8427fc700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 0x7fc83803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:24.487 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.485+0000 7fc8427fc700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 0x7fc83803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc834006fd0 tx=0x7fc834006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:24.491 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.490+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc83c013070 con 0x7fc84c102130 2024-06-06T07:37:24.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.698+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}) -- 0x7fc84c198e30 con 0x7fc8380390f0 2024-06-06T07:37:24.705 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.705+0000 7fc848ff9700 1 -- 172.21.15.195:0/691496340 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc84c198e30 con 0x7fc8380390f0 2024-06-06T07:37:24.705 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mon update... 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 msgr2=0x7fc83803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 0x7fc83803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc834006fd0 tx=0x7fc834006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 msgr2=0x7fc84c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc83c005850 tx=0x7fc83c005030 comp rx=0 tx=0).stop 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 shutdown_connections 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc8380390f0 0x7fc83803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 --2- 172.21.15.195:0/691496340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102130 0x7fc84c19d6c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 >> 172.21.15.195:0/691496340 conn(0x7fc84c0fd9e0 msgr2=0x7fc84c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 shutdown_connections 2024-06-06T07:37:24.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:24.707+0000 7fc8512e9700 1 -- 172.21.15.195:0/691496340 wait complete. 2024-06-06T07:37:25.269 DEBUG:teuthology.orchestra.run.smithi181:mon.b> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.b.service 2024-06-06T07:37:25.272 DEBUG:teuthology.orchestra.run.smithi195:mon.c> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.c.service 2024-06-06T07:37:25.275 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T07:37:25.275 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:26.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:25 smithi087 ceph-mon[32276]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:26.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:25 smithi087 ceph-mon[32276]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c;count:3 2024-06-06T07:37:26.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:25 smithi087 ceph-mon[32276]: Deploying daemon mon.c on smithi195 2024-06-06T07:37:26.394 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:27.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:26 smithi087 ceph-mon[32276]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:28.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:28 smithi195 systemd[1]: Starting Ceph mon.c for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:37:28.910 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- 172.21.15.195:0/1070302852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 msgr2=0x7f21c81024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 --2- 172.21.15.195:0/1070302852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c81024e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009910 tx=0x7f21bc009c20 comp rx=0 tx=0).stop 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- 172.21.15.195:0/1070302852 shutdown_connections 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 --2- 172.21.15.195:0/1070302852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c81024e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- 172.21.15.195:0/1070302852 >> 172.21.15.195:0/1070302852 conn(0x7f21c80fd9d0 msgr2=0x7f21c80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- 172.21.15.195:0/1070302852 shutdown_connections 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- 172.21.15.195:0/1070302852 wait complete. 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 Processor -- start 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- start start 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.908+0000 7f21cee5f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21c810c4d0 con 0x7f21c8102100 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21ccbfb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:28.911 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21ccbfb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.195:44412/0 (socket says 172.21.15.195:44412) 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21ccbfb700 1 -- 172.21.15.195:0/3909520440 learned_addr learned my addr 172.21.15.195:0/3909520440 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21ccbfb700 1 -- 172.21.15.195:0/3909520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c8076680 con 0x7f21c8102100 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21ccbfb700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005850 tx=0x7f21bc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc026070 con 0x7f21c8102100 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c8074d30 con 0x7f21c8102100 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21bc00bc30 con 0x7f21c8102100 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.909+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21bc003a20 con 0x7f21c8102100 2024-06-06T07:37:28.912 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.910+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c80751f0 con 0x7f21c8102100 2024-06-06T07:37:28.913 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.910+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f21bc029070 con 0x7f21c8102100 2024-06-06T07:37:28.913 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.910+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c81035e0 con 0x7f21c8102100 2024-06-06T07:37:28.914 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.910+0000 7f21c5ffb700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 0x7f21b803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:28.914 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.911+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f21bc01e070 con 0x7f21c8102100 2024-06-06T07:37:28.914 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.911+0000 7f21c7fff700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 0x7f21b803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:28.914 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.911+0000 7f21c7fff700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 0x7f21b803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:28.916 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:28.916+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21bc013070 con 0x7f21c8102100 2024-06-06T07:37:29.217 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.216+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f21c8075cc0 con 0x7f21c8102100 2024-06-06T07:37:29.217 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 podman[37008]: 2024-06-06 07:37:29.116168492 +0000 UTC m=+0.113500406 container create a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:37:29.217 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 podman[37008]: 2024-06-06 07:37:29.02331633 +0000 UTC m=+0.020648236 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:37:29.218 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.216+0000 7f21c5ffb700 1 -- 172.21.15.195:0/3909520440 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f21bc030020 con 0x7f21c8102100 2024-06-06T07:37:29.218 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:37:29.218 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":1,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","modified":"2024-06-06T07:35:46.713783Z","created":"2024-06-06T07:35:46.713783Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 msgr2=0x7f21b803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 0x7f21b803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4006e40 comp rx=0 tx=0).stop 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 msgr2=0x7f21c8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005850 tx=0x7f21bc005030 comp rx=0 tx=0).stop 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 shutdown_connections 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f21b80390f0 0x7f21b803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 --2- 172.21.15.195:0/3909520440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c8102100 0x7f21c8074790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 >> 172.21.15.195:0/3909520440 conn(0x7f21c80fd9d0 msgr2=0x7f21c8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 shutdown_connections 2024-06-06T07:37:29.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:29.219+0000 7f21cee5f700 1 -- 172.21.15.195:0/3909520440 wait complete. 2024-06-06T07:37:29.221 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 1 2024-06-06T07:37:29.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 podman[37008]: 2024-06-06 07:37:29.35202366 +0000 UTC m=+0.349355556 container init a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:37:29.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 podman[37008]: 2024-06-06 07:37:29.35536118 +0000 UTC m=+0.352693081 container start a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T07:37:29.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pidfile_write: ignore empty --pid-file 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: load: jerasure load: lrc 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: RocksDB version: 7.9.2 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Git sha 0 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: DB SUMMARY 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: DB Session ID: RRNOYZ88EJGWNX1TP1AT 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: CURRENT file: CURRENT 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T07:37:29.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.error_if_exists: 0 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.create_if_missing: 0 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.paranoid_checks: 1 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.env: 0x55de70a8ec60 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.info_log: 0x55de726dbb20 2024-06-06T07:37:29.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.statistics: (nil) 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.use_fsync: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_log_file_size: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_fallocate: 1 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T07:37:29.514 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.use_direct_reads: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.db_log_dir: 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.wal_dir: 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.write_buffer_manager: 0x55de726ac1e0 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T07:37:29.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.unordered_write: 0 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.row_cache: None 2024-06-06T07:37:29.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.wal_filter: None 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.two_write_queues: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.wal_compression: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.atomic_flush: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T07:37:29.518 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.log_readahead_size: 0 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_background_jobs: 2 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_background_compactions: -1 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_subcompactions: 1 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T07:37:29.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_open_files: -1 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_background_flushes: -1 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Compression algorithms supported: 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kZSTD supported: 0 2024-06-06T07:37:29.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kXpressCompression supported: 0 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kZlibCompression supported: 1 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kSnappyCompression supported: 1 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kLZ4Compression supported: 1 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: kBZip2Compression supported: 0 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.merge_operator: 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_filter: None 2024-06-06T07:37:29.521 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55de726dbc40) 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks: 1 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T07:37:29.522 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_top_level_index_and_filter: 1 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_type: 0 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_index_type: 0 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_shortening: 1 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: checksum: 4 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: no_block_cache: 0 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache: 0x55de726dd090 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_name: BinnedLRUCache 2024-06-06T07:37:29.523 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_options: 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: capacity : 536870912 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_shard_bits : 4 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: strict_capacity_limit : 0 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: high_pri_pool_ratio: 0.000 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_compressed: (nil) 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: persistent_cache: (nil) 2024-06-06T07:37:29.524 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size: 4096 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size_deviation: 10 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_restart_interval: 16 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_block_restart_interval: 1 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: metadata_block_size: 4096 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: partition_filters: 0 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: use_delta_encoding: 1 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: filter_policy: bloomfilter 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: whole_key_filtering: 1 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: verify_compression: 0 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: read_amp_bytes_per_bit: 0 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: format_version: 5 2024-06-06T07:37:29.525 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_index_compression: 1 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_align: 0 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout: max_auto_readahead_size: 262144 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout: prepopulate_block_cache: 0 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout: initial_auto_readahead_size: 8192 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression: NoCompression 2024-06-06T07:37:29.526 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.num_levels: 7 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T07:37:29.527 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T07:37:29.528 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T07:37:29.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T07:37:29.530 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T07:37:29.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T07:37:29.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T07:37:29.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T07:37:29.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T07:37:29.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T07:37:29.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T07:37:29.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.table_properties_collectors: 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.inplace_update_support: 0 2024-06-06T07:37:29.534 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.bloom_locality: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.max_successive_merges: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T07:37:29.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.ttl: 2592000 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enable_blob_files: false 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.min_blob_size: 0 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T07:37:29.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 95deb791-fc6a-4b16-a54e-a41f7dfe44e6 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659449385217, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659449385618, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717659449, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "95deb791-fc6a-4b16-a54e-a41f7dfe44e6", "db_session_id": "RRNOYZ88EJGWNX1TP1AT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717659449385697, "job": 1, "event": "recovery_finished"} 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55de727cc000 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: DB pointer 0x55de727b6000 2024-06-06T07:37:29.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** DB Stats ** 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T07:37:29.538 INFO:journalctl@ceph.mon.c.smithi195.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T07:37:29.539 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache BinnedLRUCache@0x55de726dd090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: using public_addr v2:172.21.15.195:0/0 -> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: starting mon.c rank -1 at public addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] at bind addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(???) e0 preinit fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:29.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).mds e1 new map 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: e1 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: legacy client fscid: -1 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout: No filesystems configured 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-06T07:37:29.541 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.542 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Deploying cephadm binary to smithi195 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.543 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Added host smithi195 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1320741609' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1320741609' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:29.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c;count:3 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: Deploying daemon mon.c on smithi195 2024-06-06T07:37:29.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:29.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 ceph-mon[37040]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T07:37:29.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 bash[37008]: a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f 2024-06-06T07:37:29.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:29 smithi195 systemd[1]: Started Ceph mon.c for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:37:29.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:29 smithi087 ceph-mon[32276]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:31.023 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T07:37:31.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T07:37:31.260 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:37:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:37:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: mon.a calling monitor election 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: mon.c calling monitor election 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: monmap epoch 2 2024-06-06T07:37:34.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: last_changed 2024-06-06T07:37:29.409955+0000 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: min_mon_release 19 (squid) 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: election_strategy: 1 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: fsmap 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: mgrmap e13: x(active, since 48s) 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: overall HEALTH_OK 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:34.764 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:34.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:37:34.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: mon.a calling monitor election 2024-06-06T07:37:34.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: mon.c calling monitor election 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:34.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: monmap epoch 2 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: last_changed 2024-06-06T07:37:29.409955+0000 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: min_mon_release 19 (squid) 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: election_strategy: 1 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: fsmap 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: mgrmap e13: x(active, since 48s) 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: overall HEALTH_OK 2024-06-06T07:37:34.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:34.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:34.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:35.566 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 -- 172.21.15.195:0/1266738025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 msgr2=0x7fc4bc005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 --2- 172.21.15.195:0/1266738025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4bc005610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005fa0 tx=0x7fc4c400ffc0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 -- 172.21.15.195:0/1266738025 shutdown_connections 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 --2- 172.21.15.195:0/1266738025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4bc005610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 -- 172.21.15.195:0/1266738025 >> 172.21.15.195:0/1266738025 conn(0x7fc4d40fda50 msgr2=0x7fc4d40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 -- 172.21.15.195:0/1266738025 shutdown_connections 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.565+0000 7fc4da553700 1 -- 172.21.15.195:0/1266738025 wait complete. 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.566+0000 7fc4da553700 1 Processor -- start 2024-06-06T07:37:35.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.566+0000 7fc4da553700 1 -- start start 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4da553700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4da553700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4da553700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d410a010 con 0x7fc4d41021e0 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4da553700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4109e90 con 0x7fc4d40763d0 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:35.570 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59230/0 (socket says 172.21.15.195:59230) 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 -- 172.21.15.195:0/440591732 learned_addr learned my addr 172.21.15.195:0/440591732 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d3fff700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 -- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 msgr2=0x7fc4d4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 -- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 msgr2=0x7fc4d4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d37fe700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d3fff700 1 -- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 msgr2=0x7fc4d4074cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d3fff700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.567+0000 7fc4d3fff700 1 -- 172.21.15.195:0/440591732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4075210 con 0x7fc4d41021e0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4d3fff700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4d4074790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000c00 tx=0x7fc4c4004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc4c40053b0 con 0x7fc4d41021e0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4005550 con 0x7fc4d41021e0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d40754a0 con 0x7fc4d41021e0 2024-06-06T07:37:35.571 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc4c4011670 con 0x7fc4d41021e0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.568+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d41a8690 con 0x7fc4d41021e0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.569+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc4c4011810 con 0x7fc4d41021e0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.569+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d41036c0 con 0x7fc4d41021e0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.569+0000 7fc4d17fa700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 0x7fc4c003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.569+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc4c404e770 con 0x7fc4d41021e0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.570+0000 7fc4d37fe700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 0x7fc4c003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:35.572 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.570+0000 7fc4d37fe700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 0x7fc4c003b3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4d419bc00 tx=0x7fc4c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:35.575 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.575+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4c4022070 con 0x7fc4d41021e0 2024-06-06T07:37:35.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:35 smithi195 ceph-mon[37040]: Deploying daemon mon.b on smithi181 2024-06-06T07:37:35.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:35 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:35.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.850+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc4d4075d60 con 0x7fc4d41021e0 2024-06-06T07:37:35.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.850+0000 7fc4d17fa700 1 -- 172.21.15.195:0/440591732 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc4c4019bf0 con 0x7fc4d41021e0 2024-06-06T07:37:35.851 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:37:35.851 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","modified":"2024-06-06T07:37:29.409955Z","created":"2024-06-06T07:35:46.713783Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T07:37:35.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 msgr2=0x7fc4c003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:35.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 0x7fc4c003b3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4d419bc00 tx=0x7fc4c8008040 comp rx=0 tx=0).stop 2024-06-06T07:37:35.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 msgr2=0x7fc4d4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4d4074790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000c00 tx=0x7fc4c4004d60 comp rx=0 tx=0).stop 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 shutdown_connections 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc4c0038f00 0x7fc4c003b3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d41021e0 0x7fc4d4074790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 --2- 172.21.15.195:0/440591732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d40763d0 0x7fc4d4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 >> 172.21.15.195:0/440591732 conn(0x7fc4d40fda50 msgr2=0x7fc4d4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 shutdown_connections 2024-06-06T07:37:35.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:35.853+0000 7fc4da553700 1 -- 172.21.15.195:0/440591732 wait complete. 2024-06-06T07:37:35.855 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-06-06T07:37:35.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:35 smithi087 ceph-mon[32276]: Deploying daemon mon.b on smithi181 2024-06-06T07:37:35.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:35 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:36.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:36 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/440591732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T07:37:36.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:36 smithi195 ceph-mon[37040]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:36 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/440591732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T07:37:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:36 smithi087 ceph-mon[32276]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:37.429 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T07:37:37.430 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T07:37:37.669 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:37:38.503 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:38 smithi181 ceph-mon[36790]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T07:37:38.780 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:38 smithi181 bash[36770]: 39452bd0d8b23086d5921c0f7fca37e0eadb711d8160693d0afcb27385c71065 2024-06-06T07:37:38.781 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:38 smithi181 systemd[1]: Started Ceph mon.b for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:37:43.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:37:43.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: mon.a calling monitor election 2024-06-06T07:37:43.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: mon.c calling monitor election 2024-06-06T07:37:43.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: mon.b calling monitor election 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: monmap epoch 3 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:43.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: last_changed 2024-06-06T07:37:38.495366+0000 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: min_mon_release 19 (squid) 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: election_strategy: 1 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: fsmap 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: mgrmap e13: x(active, since 57s) 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: overall HEALTH_OK 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:43.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: mon.a calling monitor election 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: mon.c calling monitor election 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: mon.b calling monitor election 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: monmap epoch 3 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: last_changed 2024-06-06T07:37:38.495366+0000 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: min_mon_release 19 (squid) 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: election_strategy: 1 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: fsmap 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:44.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: mgrmap e13: x(active, since 57s) 2024-06-06T07:37:44.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: overall HEALTH_OK 2024-06-06T07:37:44.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: mon.a calling monitor election 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: mon.c calling monitor election 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: mon.b calling monitor election 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: monmap epoch 3 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: last_changed 2024-06-06T07:37:38.495366+0000 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: created 2024-06-06T07:35:46.713783+0000 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: min_mon_release 19 (squid) 2024-06-06T07:37:44.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: election_strategy: 1 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: 0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: fsmap 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: mgrmap e13: x(active, since 57s) 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: overall HEALTH_OK 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:44.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:44 smithi181 ceph-mon[36790]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:44.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:44.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:44.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:44 smithi087 ceph-mon[32276]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:44.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:45.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:45.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:44 smithi195 ceph-mon[37040]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:45.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:37:45.074 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 -- 172.21.15.195:0/4073494966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b7c0062a0 msgr2=0x7f7b7c008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:45.074 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 --2- 172.21.15.195:0/4073494966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b7c0062a0 0x7f7b7c008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b90101f60 tx=0x7f7b8000fe90 comp rx=0 tx=0).stop 2024-06-06T07:37:45.074 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 -- 172.21.15.195:0/4073494966 shutdown_connections 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 --2- 172.21.15.195:0/4073494966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b7c0062a0 0x7f7b7c008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 -- 172.21.15.195:0/4073494966 >> 172.21.15.195:0/4073494966 conn(0x7f7b900fc9d0 msgr2=0x7f7b900fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.068+0000 7f7b973f7700 1 -- 172.21.15.195:0/4073494966 shutdown_connections 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 -- 172.21.15.195:0/4073494966 wait complete. 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 Processor -- start 2024-06-06T07:37:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 -- start start 2024-06-06T07:37:45.076 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:45.076 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 0x7f7b90199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:45.076 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b901a1bb0 0x7f7b901a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:45.076 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90108f10 con 0x7f7b90199080 2024-06-06T07:37:45.076 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90108d90 con 0x7f7b901a1bb0 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b973f7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90109090 con 0x7f7b7c0062a0 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.069+0000 7f7b95193700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b95193700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33982/0 (socket says 172.21.15.195:33982) 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b95193700 1 -- 172.21.15.195:0/2575763502 learned_addr learned my addr 172.21.15.195:0/2575763502 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 0x7f7b90199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:45.077 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b95994700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b901a1bb0 0x7f7b901a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:45.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 -- 172.21.15.195:0/2575763502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 msgr2=0x7f7b90198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:45.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 -- 172.21.15.195:0/2575763502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b901a1bb0 msgr2=0x7f7b901a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:45.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b901a1bb0 0x7f7b901a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 -- 172.21.15.195:0/2575763502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901a47e0 con 0x7f7b90199080 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b95193700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b94992700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 0x7f7b90199500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b550 tx=0x7f7b8c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c004230 con 0x7f7b90199080 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c0043d0 con 0x7f7b90199080 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b901a4ad0 con 0x7f7b90199080 2024-06-06T07:37:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c011b20 con 0x7f7b90199080 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.070+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901a5010 con 0x7f7b90199080 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.071+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7b8c011d00 con 0x7f7b90199080 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.071+0000 7f7b867fc700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 0x7f7b7403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.071+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b8c04cd10 con 0x7f7b90199080 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.071+0000 7f7b95193700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 0x7f7b7403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.071+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b90063af0 con 0x7f7b90199080 2024-06-06T07:37:45.081 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.072+0000 7f7b95193700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 0x7f7b7403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b8001f040 tx=0x7f7b80029040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:45.081 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.077+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b8c020500 con 0x7f7b90199080 2024-06-06T07:37:45.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.373+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7b9002dca0 con 0x7f7b90199080 2024-06-06T07:37:45.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.373+0000 7f7b867fc700 1 -- 172.21.15.195:0/2575763502 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f7b8c0202a0 con 0x7f7b90199080 2024-06-06T07:37:45.374 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-06T07:37:45.374 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":3,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","modified":"2024-06-06T07:37:38.495366Z","created":"2024-06-06T07:35:46.713783Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-06T07:37:45.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 msgr2=0x7f7b7403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 0x7f7b7403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b8001f040 tx=0x7f7b80029040 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 msgr2=0x7f7b90199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 0x7f7b90199500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b550 tx=0x7f7b8c00b860 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 shutdown_connections 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b74039580 0x7f7b7403ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b901a1bb0 0x7f7b901a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b7c0062a0 0x7f7b90198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 --2- 172.21.15.195:0/2575763502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b90199080 0x7f7b90199500 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 >> 172.21.15.195:0/2575763502 conn(0x7f7b900fc9d0 msgr2=0x7f7b900fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 shutdown_connections 2024-06-06T07:37:45.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:45.376+0000 7f7b973f7700 1 -- 172.21.15.195:0/2575763502 wait complete. 2024-06-06T07:37:45.378 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 3 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:37:45.944 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/2575763502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:45.946 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:45.946 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:45.946 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:45.972 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-06T07:37:45.973 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-06T07:37:46.174 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.174 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/2575763502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.175 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:46.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:46.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/2575763502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:46.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:47.141 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: Reconfiguring daemon mon.a on smithi087 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: Reconfiguring daemon mon.a on smithi087 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: Reconfiguring daemon mon.a on smithi087 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.297 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: Reconfiguring mon.b (monmap changed)... 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: Reconfiguring daemon mon.b on smithi181 2024-06-06T07:37:48.298 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:48 smithi087 ceph-mon[32276]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: Reconfiguring mon.b (monmap changed)... 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: Reconfiguring daemon mon.b on smithi181 2024-06-06T07:37:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:48 smithi181 ceph-mon[36790]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: Reconfiguring mon.b (monmap changed)... 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: Reconfiguring daemon mon.b on smithi181 2024-06-06T07:37:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:48 smithi195 ceph-mon[37040]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 -- 172.21.15.87:0/2266506034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00641030f0 msgr2=0x7f0064107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 --2- 172.21.15.87:0/2266506034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00641030f0 0x7f0064107110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00580099c0 tx=0x7f0058009cd0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 -- 172.21.15.87:0/2266506034 shutdown_connections 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 --2- 172.21.15.87:0/2266506034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0064107840 0x7f006410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 --2- 172.21.15.87:0/2266506034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 0x7f0064102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 --2- 172.21.15.87:0/2266506034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00641030f0 0x7f0064107110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 -- 172.21.15.87:0/2266506034 >> 172.21.15.87:0/2266506034 conn(0x7f00640795f0 msgr2=0x7f0064079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.513+0000 7f0069c14700 1 -- 172.21.15.87:0/2266506034 shutdown_connections 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 -- 172.21.15.87:0/2266506034 wait complete. 2024-06-06T07:37:48.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 Processor -- start 2024-06-06T07:37:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 -- start start 2024-06-06T07:37:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 0x7f0064074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:48.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00641030f0 0x7f0064074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00641128a0 con 0x7f0064107840 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0064112720 con 0x7f00641030f0 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.514+0000 7f0069c14700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0064112a20 con 0x7f0064102740 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41714/0 (socket says 172.21.15.87:41714) 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 -- 172.21.15.87:0/3744088580 learned_addr learned my addr 172.21.15.87:0/3744088580 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f00637fe700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 0x7f0064074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0062ffd700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00641030f0 0x7f0064074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 -- 172.21.15.87:0/3744088580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 msgr2=0x7f0064074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 0x7f0064074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 -- 172.21.15.87:0/3744088580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00641030f0 msgr2=0x7f0064074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:48.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00641030f0 0x7f0064074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 -- 172.21.15.87:0/3744088580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00641a7270 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0063fff700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b970 tx=0x7f005c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c00d860 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c00e430 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00641a7560 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c017660 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.515+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641a7a70 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.516+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f005c0203f0 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.516+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0064069180 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.516+0000 7f0060ff9700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 0x7f004c03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.516+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f005c04be70 con 0x7f0064107840 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.516+0000 7f00637fe700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 0x7f004c03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:48.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.517+0000 7f00637fe700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 0x7f004c03b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0054009a10 tx=0x7f0054009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:48.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.521+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f005c013030 con 0x7f0064107840 2024-06-06T07:37:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.719+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f006402de60 con 0x7f0064107840 2024-06-06T07:37:48.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.719+0000 7f0060ff9700 1 -- 172.21.15.87:0/3744088580 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f005c016360 con 0x7f0064107840 2024-06-06T07:37:48.720 INFO:teuthology.orchestra.run.smithi087.stdout:# minimal ceph.conf for 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:48.721 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-06-06T07:37:48.721 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:37:48.721 INFO:teuthology.orchestra.run.smithi087.stdout: mon_host = [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-06T07:37:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 msgr2=0x7f004c03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 0x7f004c03b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0054009a10 tx=0x7f0054009190 comp rx=0 tx=0).stop 2024-06-06T07:37:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 msgr2=0x7f0064075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:48.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b970 tx=0x7f005c00bc80 comp rx=0 tx=0).stop 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 shutdown_connections 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f004c039330 0x7f004c03b7f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0064107840 0x7f0064075210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0064102740 0x7f0064074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 --2- 172.21.15.87:0/3744088580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00641030f0 0x7f0064074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 >> 172.21.15.87:0/3744088580 conn(0x7f00640795f0 msgr2=0x7f00641007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 shutdown_connections 2024-06-06T07:37:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:48.722+0000 7f0069c14700 1 -- 172.21.15.87:0/3744088580 wait complete. 2024-06-06T07:37:49.305 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-06T07:37:49.306 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:37:49.306 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T07:37:49.336 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:37:49.336 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:49.402 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:37:49.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T07:37:49.430 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:37:49.430 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:49.495 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:37:49.495 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T07:37:49.521 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:37:49.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:37:49.585 INFO:tasks.cephadm:Adding mgr.x on smithi087 2024-06-06T07:37:49.585 INFO:tasks.cephadm:Adding mgr.y on smithi181 2024-06-06T07:37:49.586 INFO:tasks.cephadm:Adding mgr.z on smithi195 2024-06-06T07:37:49.586 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi087=x;smithi181=y;smithi195=z' 2024-06-06T07:37:49.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: Reconfiguring mon.c (monmap changed)... 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: Reconfiguring daemon mon.c on smithi195 2024-06-06T07:37:49.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:49 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3744088580' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: Reconfiguring mon.c (monmap changed)... 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: Reconfiguring daemon mon.c on smithi195 2024-06-06T07:37:49.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:49 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3744088580' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: Reconfiguring mon.c (monmap changed)... 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: Reconfiguring daemon mon.c on smithi195 2024-06-06T07:37:49.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:49 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3744088580' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:50.032 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:37:51.270 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.270 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.270 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:51.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:51.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:51.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:51.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:51.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 -- 172.21.15.195:0/790257302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 msgr2=0x7ff348102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/790257302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348102200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009990 tx=0x7ff33c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 -- 172.21.15.195:0/790257302 shutdown_connections 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/790257302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348102740 0x7ff348108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/790257302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348100ba0 0x7ff348100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/790257302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348102200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 -- 172.21.15.195:0/790257302 >> 172.21.15.195:0/790257302 conn(0x7ff3480fc140 msgr2=0x7ff3480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:51.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 -- 172.21.15.195:0/790257302 shutdown_connections 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.442+0000 7ff34e2f9700 1 -- 172.21.15.195:0/790257302 wait complete. 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 Processor -- start 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 -- start start 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 0x7ff348198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348102740 0x7ff34819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff34810e1b0 con 0x7ff348102740 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff34810e030 con 0x7ff348100ba0 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34e2f9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff34810e330 con 0x7ff348101da0 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff3477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff3477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:53794/0 (socket says 172.21.15.195:53794) 2024-06-06T07:37:51.446 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff3477fe700 1 -- 172.21.15.195:0/1586205413 learned_addr learned my addr 172.21.15.195:0/1586205413 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.443+0000 7ff34c896700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348102740 0x7ff34819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff347fff700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 0x7ff348198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 -- 172.21.15.195:0/1586205413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 msgr2=0x7ff348198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 0x7ff348198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 -- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348102740 msgr2=0x7ff34819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348102740 0x7ff34819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 -- 172.21.15.195:0/1586205413 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3481a03b0 con 0x7ff348101da0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff347fff700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 0x7ff348198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3477fe700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff33c004790 tx=0x7ff33c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c027070 con 0x7ff348101da0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3481a0640 con 0x7ff348101da0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c004150 con 0x7ff348101da0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00f460 con 0x7ff348101da0 2024-06-06T07:37:51.447 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.444+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a0b70 con 0x7ff348101da0 2024-06-06T07:37:51.448 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.445+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff33c02a070 con 0x7ff348101da0 2024-06-06T07:37:51.448 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.445+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff348069180 con 0x7ff348101da0 2024-06-06T07:37:51.448 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.445+0000 7ff3457fa700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 0x7ff33003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:51.449 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.445+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff33c04d270 con 0x7ff348101da0 2024-06-06T07:37:51.449 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.445+0000 7ff347fff700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 0x7ff33003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:51.449 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.446+0000 7ff347fff700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 0x7ff33003b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff3380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:51.451 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.450+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff33c013070 con 0x7ff348101da0 2024-06-06T07:37:51.659 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.658+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi181=y;smithi195=z", "target": ["mon-mgr", ""]}) -- 0x7ff348062960 con 0x7ff330039330 2024-06-06T07:37:51.664 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.664+0000 7ff3457fa700 1 -- 172.21.15.195:0/1586205413 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff348062960 con 0x7ff330039330 2024-06-06T07:37:51.664 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mgr update... 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 msgr2=0x7ff33003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 0x7ff33003b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff3380058e0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 msgr2=0x7ff348198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff33c004790 tx=0x7ff33c009580 comp rx=0 tx=0).stop 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 shutdown_connections 2024-06-06T07:37:51.667 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff330039330 0x7ff33003b7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff348100ba0 0x7ff348198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff348102740 0x7ff34819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 --2- 172.21.15.195:0/1586205413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff348101da0 0x7ff348198c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 >> 172.21.15.195:0/1586205413 conn(0x7ff3480fc140 msgr2=0x7ff348105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 shutdown_connections 2024-06-06T07:37:51.668 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:37:51.666+0000 7ff34e2f9700 1 -- 172.21.15.195:0/1586205413 wait complete. 2024-06-06T07:37:52.287 DEBUG:teuthology.orchestra.run.smithi181:mgr.y> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-06T07:37:52.289 DEBUG:teuthology.orchestra.run.smithi195:mgr.z> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-06T07:37:52.291 INFO:tasks.cephadm:Deploying OSDs... 2024-06-06T07:37:52.292 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:37:52.292 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T07:37:52.311 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T07:37:52.311 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-06-06T07:37:52.369 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T07:37:52.369 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:52.369 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-06-06 07:36:56.180795741 +0000 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-06-06 07:32:11.760110655 +0000 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-06-06 07:32:11.760110655 +0000 2024-06-06T07:37:52.370 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-06-06 07:32:11.760110655 +0000 2024-06-06T07:37:52.370 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T07:37:52.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-06-06T07:37:52.437 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-06-06T07:37:52.437 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00029164 s, 1.8 MB/s 2024-06-06T07:37:52.438 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T07:37:52.497 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-06-06T07:37:52.557 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T07:37:52.557 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-06-06 07:36:56.212795033 +0000 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-06-06 07:32:12.110102791 +0000 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-06-06 07:32:12.110102791 +0000 2024-06-06T07:37:52.558 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-06-06 07:32:12.110102791 +0000 2024-06-06T07:37:52.558 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T07:37:52.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-06-06T07:37:52.627 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-06-06T07:37:52.627 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000201118 s, 2.5 MB/s 2024-06-06T07:37:52.628 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T07:37:52.687 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-06-06 07:36:56.243794347 +0000 2024-06-06T07:37:52.744 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-06-06 07:32:12.448095195 +0000 2024-06-06T07:37:52.745 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-06-06 07:32:12.448095195 +0000 2024-06-06T07:37:52.745 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-06-06 07:32:12.448095195 +0000 2024-06-06T07:37:52.745 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi181=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: Saving service mgr spec with placement smithi087=x;smithi181=y;smithi195=z;count:3 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:52.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:52.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:52.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:52.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: Deploying daemon mgr.z on smithi195 2024-06-06T07:37:52.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:52 smithi195 ceph-mon[37040]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi181=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: Saving service mgr spec with placement smithi087=x;smithi181=y;smithi195=z;count:3 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:52.807 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:52.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:52.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:52.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:52.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: Deploying daemon mgr.z on smithi195 2024-06-06T07:37:52.808 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:52 smithi087 ceph-mon[32276]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:52.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-06-06T07:37:52.809 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-06-06T07:37:52.809 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000166384 s, 3.1 MB/s 2024-06-06T07:37:52.810 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T07:37:52.867 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-06-06 07:36:56.275793639 +0000 2024-06-06T07:37:52.923 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-06-06 07:32:12.794087420 +0000 2024-06-06T07:37:52.924 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-06-06 07:32:12.794087420 +0000 2024-06-06T07:37:52.924 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-06-06 07:32:12.794087420 +0000 2024-06-06T07:37:52.924 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T07:37:52.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-06-06T07:37:52.987 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-06-06T07:37:52.987 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000214242 s, 2.4 MB/s 2024-06-06T07:37:52.988 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T07:37:53.045 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-06-06T07:37:53.045 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi181=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: Saving service mgr spec with placement smithi087=x;smithi181=y;smithi195=z;count:3 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:53.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: Deploying daemon mgr.z on smithi195 2024-06-06T07:37:53.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:52 smithi181 ceph-mon[36790]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:53.064 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T07:37:53.064 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-06-06 07:37:37.943945341 +0000 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-06-06 07:32:25.762926125 +0000 2024-06-06T07:37:53.122 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-06-06 07:32:25.763926106 +0000 2024-06-06T07:37:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-06-06 07:32:25.762926125 +0000 2024-06-06T07:37:53.123 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T07:37:53.188 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-06-06T07:37:53.188 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-06-06T07:37:53.188 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000216673 s, 2.4 MB/s 2024-06-06T07:37:53.189 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T07:37:53.247 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-06-06T07:37:53.303 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T07:37:53.303 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:53.303 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-06-06 07:37:37.944945322 +0000 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-06-06 07:32:26.080920071 +0000 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-06-06 07:32:26.080920071 +0000 2024-06-06T07:37:53.304 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-06-06 07:32:26.080920071 +0000 2024-06-06T07:37:53.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T07:37:53.367 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-06-06T07:37:53.367 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-06-06T07:37:53.367 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000280188 s, 1.8 MB/s 2024-06-06T07:37:53.369 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T07:37:53.426 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-06-06 07:37:37.944945322 +0000 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-06-06 07:32:26.399913998 +0000 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-06-06 07:32:26.399913998 +0000 2024-06-06T07:37:53.482 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-06-06 07:32:26.399913998 +0000 2024-06-06T07:37:53.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T07:37:53.545 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-06-06T07:37:53.545 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-06-06T07:37:53.545 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000170238 s, 3.0 MB/s 2024-06-06T07:37:53.546 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T07:37:53.602 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-06-06 07:37:37.944945322 +0000 2024-06-06T07:37:53.657 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-06-06 07:32:26.727907754 +0000 2024-06-06T07:37:53.658 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-06-06 07:32:26.727907754 +0000 2024-06-06T07:37:53.658 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-06-06 07:32:26.727907754 +0000 2024-06-06T07:37:53.658 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T07:37:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-06-06T07:37:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-06-06T07:37:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000211092 s, 2.4 MB/s 2024-06-06T07:37:53.721 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T07:37:53.777 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-06T07:37:53.777 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T07:37:53.793 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T07:37:53.794 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:53.848 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-06 07:37:53.684260850 +0000 2024-06-06T07:37:53.849 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-06 07:32:26.381718982 +0000 2024-06-06T07:37:53.849 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-06 07:32:26.381718982 +0000 2024-06-06T07:37:53.849 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-06-06 07:32:26.381718982 +0000 2024-06-06T07:37:53.849 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T07:37:53.910 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:53 smithi195 systemd[1]: Starting Ceph mgr.z for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:37:53.913 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-06T07:37:53.913 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-06T07:37:53.913 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.6241e-05 s, 5.9 MB/s 2024-06-06T07:37:53.914 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T07:37:53.970 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-06 07:37:53.684260850 +0000 2024-06-06T07:37:54.025 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-06 07:32:26.718712853 +0000 2024-06-06T07:37:54.026 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-06 07:32:26.718712853 +0000 2024-06-06T07:37:54.026 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-06-06 07:32:26.718712853 +0000 2024-06-06T07:37:54.026 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T07:37:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-06T07:37:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-06T07:37:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.0150811 s, 33.9 kB/s 2024-06-06T07:37:54.105 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T07:37:54.160 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-06 07:37:53.685260832 +0000 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-06 07:32:27.031707160 +0000 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-06 07:32:27.031707160 +0000 2024-06-06T07:37:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-06-06 07:32:27.031707160 +0000 2024-06-06T07:37:54.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T07:37:54.222 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:53 smithi195 podman[38678]: 2024-06-06 07:37:53.860019948 +0000 UTC m=+0.017343861 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:37:54.222 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:53 smithi195 podman[38678]: 2024-06-06 07:37:53.978393896 +0000 UTC m=+0.135717797 container create f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:37:54.222 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 podman[38678]: 2024-06-06 07:37:54.222912998 +0000 UTC m=+0.380236899 container init f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T07:37:54.244 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-06T07:37:54.245 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-06T07:37:54.245 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.0754e-05 s, 5.6 MB/s 2024-06-06T07:37:54.246 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T07:37:54.302 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-06-06T07:37:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T07:37:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T07:37:54.361 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-06T07:37:54.361 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:37:54.361 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T07:37:54.362 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-06 07:37:53.685260832 +0000 2024-06-06T07:37:54.362 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-06 07:32:27.345701449 +0000 2024-06-06T07:37:54.362 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-06 07:32:27.345701449 +0000 2024-06-06T07:37:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-06-06 07:32:27.345701449 +0000 2024-06-06T07:37:54.362 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T07:37:54.423 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-06T07:37:54.423 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-06T07:37:54.424 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.4069e-05 s, 5.4 MB/s 2024-06-06T07:37:54.425 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T07:37:54.481 INFO:tasks.cephadm:Deploying osd.0 on smithi087 with /dev/vg_nvme/lv_4... 2024-06-06T07:37:54.481 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T07:37:54.511 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 podman[38678]: 2024-06-06 07:37:54.225448651 +0000 UTC m=+0.382772550 container start f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:37:54.511 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 ceph-mgr[38819]: -- 172.21.15.195:0/34032797 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56469083c5a0 con 0x564690810c00 2024-06-06T07:37:54.511 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 bash[38678]: f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b 2024-06-06T07:37:54.511 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 systemd[1]: Started Ceph mgr.z for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:37:54.511 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:54.387+0000 7f980a82b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:37:54.718 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:54.994 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:54.582+0000 7f980a82b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:37:54.994 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:54.684+0000 7f980a82b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:37:54.994 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:54.993+0000 7f980a82b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:37:55.260 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:55.078+0000 7f980a82b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:55.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:55.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:55.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:55.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T07:37:55.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:55.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:56.212 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:56 smithi181 ceph-mon[36790]: Deploying daemon mgr.y on smithi181 2024-06-06T07:37:56.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:56 smithi087 ceph-mon[32276]: Deploying daemon mgr.y on smithi181 2024-06-06T07:37:56.434 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:56 smithi195 ceph-mon[37040]: Deploying daemon mgr.y on smithi181 2024-06-06T07:37:56.434 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:56.275+0000 7f980a82b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:37:56.448 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T07:37:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T07:37:56.449 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-06-06T07:37:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-06-06T07:37:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124463 s, 842 MB/s 2024-06-06T07:37:56.449 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-06-06T07:37:56.760 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:56.453+0000 7f980a82b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:37:56.760 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:56.635+0000 7f980a82b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:37:56.993 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:56.992+0000 7f124d6d6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:37:57.097 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi087:vg_nvme/lv_4 2024-06-06T07:37:57.251 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:57.251 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:56.938+0000 7f980a82b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:57.036+0000 7f980a82b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:37:57.252 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:57.250+0000 7f980a82b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:57.105+0000 7f124d6d6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.283 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:37:57.454 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:57.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:57.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:37:57.510 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:57.455+0000 7f980a82b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:37:57.543 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:57.440+0000 7f124d6d6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:37:57.543 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:57.542+0000 7f124d6d6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:37:58.510 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:58.115+0000 7f980a82b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:37:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- 172.21.15.87:0/785276230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64104120 msgr2=0x7f0b6410a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- 172.21.15.87:0/785276230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64104120 0x7f0b6410a9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00b0a0 tx=0x7f0b5c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- 172.21.15.87:0/785276230 shutdown_connections 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- 172.21.15.87:0/785276230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 0x7f0b64100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- 172.21.15.87:0/785276230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64104120 0x7f0b6410a9d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- 172.21.15.87:0/785276230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64103780 0x7f0b64103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- 172.21.15.87:0/785276230 >> 172.21.15.87:0/785276230 conn(0x7f0b640fc120 msgr2=0x7f0b640fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- 172.21.15.87:0/785276230 shutdown_connections 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- 172.21.15.87:0/785276230 wait complete. 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 Processor -- start 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 -- start start 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 0x7f0b64198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:58.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.836+0000 7f0b6b141700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64104120 0x7f0b6419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b6b141700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410e0c0 con 0x7f0b64103780 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b6b141700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410df40 con 0x7f0b641007a0 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b6b141700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6410e240 con 0x7f0b64104120 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b68edd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 0x7f0b64198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52138/0 (socket says 172.21.15.87:52138) 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 -- 172.21.15.87:0/2106258658 learned_addr learned my addr 172.21.15.87:0/2106258658 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b696de700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64104120 0x7f0b6419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 -- 172.21.15.87:0/2106258658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64104120 msgr2=0x7f0b6419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:58.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64104120 0x7f0b6419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 -- 172.21.15.87:0/2106258658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 msgr2=0x7f0b64198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 0x7f0b64198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 -- 172.21.15.87:0/2106258658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b641a02d0 con 0x7f0b64103780 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b63fff700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d9b0 tx=0x7f0b5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50017070 con 0x7f0b64103780 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641a05c0 con 0x7f0b64103780 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b50005020 con 0x7f0b64103780 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.837+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50009a90 con 0x7f0b64103780 2024-06-06T07:37:58.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.838+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a0ab0 con 0x7f0b64103780 2024-06-06T07:37:58.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.838+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0b50009c30 con 0x7f0b64103780 2024-06-06T07:37:58.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.838+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b64069180 con 0x7f0b64103780 2024-06-06T07:37:58.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.839+0000 7f0b61ffb700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 0x7f0b5403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:37:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.839+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0b5004c310 con 0x7f0b64103780 2024-06-06T07:37:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.839+0000 7f0b68edd700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 0x7f0b5403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:37:58.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.839+0000 7f0b68edd700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 0x7f0b5403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b64069f70 tx=0x7f0b5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:37:58.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:58.844+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b50020020 con 0x7f0b64103780 2024-06-06T07:37:58.857 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:58.555+0000 7f980a82b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:37:58.857 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:58.653+0000 7f980a82b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:37:58.857 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:58.856+0000 7f980a82b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:37:59.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:37:59.059+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0b640628d0 con 0x7f0b54039330 2024-06-06T07:37:59.126 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:58.815+0000 7f124d6d6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:37:59.127 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:58.984+0000 7f124d6d6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:37:59.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:59.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:37:59.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:37:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:59.406 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:59.406 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.406 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:37:59.406 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:37:59.406 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.440 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:59.155+0000 7f124d6d6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:37:59.440 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:59.439+0000 7f124d6d6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:37:59.442 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:37:59.442 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:37:59.736 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:59.531+0000 7f124d6d6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:37:59.736 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:59.735+0000 7f124d6d6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:37:59.743 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:59.404+0000 7f980a82b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:37:59.744 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:59.496+0000 7f980a82b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:38:00.010 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:59.743+0000 7f980a82b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:38:00.010 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:59.879+0000 7f980a82b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:38:00.011 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:37:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:37:59.974+0000 7f980a82b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:38:00.057 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:37:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:37:59.927+0000 7f124d6d6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: Reconfiguring daemon mgr.x on smithi087 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:00.186 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.399 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:38:00.092+0000 7f980a82b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:38:00.399+0000 7f980a82b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: Reconfiguring daemon mgr.x on smithi087 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:00.400 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.532 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: Reconfiguring daemon mgr.x on smithi087 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:00.533 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:00.807 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:00.532+0000 7f124d6d6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:38:01.202 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:01 smithi087 ceph-mon[32276]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:01.202 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:01.238 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-mon[36790]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:01.239 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:01.239 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:00.945+0000 7f124d6d6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:38:01.239 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:01.039+0000 7f124d6d6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:38:01.239 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:01.238+0000 7f124d6d6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:38:01.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:01 smithi195 ceph-mon[37040]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:01.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:01 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:01.261 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:38:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:38:00.997+0000 7f980a82b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:38:01.261 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:38:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:38:01.216+0000 7f980a82b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:38:02.057 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:01.779+0000 7f124d6d6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:38:02.057 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:01.869+0000 7f124d6d6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:38:02.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:02.136+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f0b5001b400 con 0x7f0b64103780 2024-06-06T07:38:02.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: Standby manager daemon z started 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]: dispatch 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]': finished 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T07:38:02.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:02 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: Standby manager daemon z started 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]: dispatch 2024-06-06T07:38:02.444 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]': finished 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:02.107+0000 7f124d6d6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:02.238+0000 7f124d6d6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:02.330+0000 7f124d6d6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:38:02.445 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:02.443+0000 7f124d6d6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:38:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: Standby manager daemon z started 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/66942478' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]: dispatch 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2162883787' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cb21c7c-fbf5-4e45-9644-1e793e30f06d"}]': finished 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T07:38:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:02 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:03.056 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:02.743+0000 7f124d6d6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:38:03.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:03 smithi195 ceph-mon[37040]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:03.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:03 smithi195 ceph-mon[37040]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T07:38:03.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:38:03.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/645040870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-mon[36790]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-mon[36790]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/645040870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:03.332+0000 7f124d6d6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:38:03.544 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:38:03 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:38:03.543+0000 7f124d6d6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:38:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:03 smithi087 ceph-mon[32276]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:03 smithi087 ceph-mon[32276]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T07:38:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:38:03.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/645040870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:04.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:04.145+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0b500051d0 con 0x7f0b64103780 2024-06-06T07:38:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:04 smithi195 ceph-mon[37040]: Standby manager daemon y started 2024-06-06T07:38:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:04 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:38:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:04 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:04 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:38:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:04 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:04.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:04 smithi181 ceph-mon[36790]: Standby manager daemon y started 2024-06-06T07:38:04.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:04 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:38:04.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:04 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:04.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:04 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:38:04.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:04 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:04 smithi087 ceph-mon[32276]: Standby manager daemon y started 2024-06-06T07:38:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:04 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:38:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:04 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:38:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:04 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:38:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:04 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/2527731378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:38:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:05 smithi195 ceph-mon[37040]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:05 smithi195 ceph-mon[37040]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-06-06T07:38:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:05 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:38:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:05 smithi181 ceph-mon[36790]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:05 smithi181 ceph-mon[36790]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-06-06T07:38:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:05 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:38:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:05 smithi087 ceph-mon[32276]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:05 smithi087 ceph-mon[32276]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-06-06T07:38:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:05 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:38:07.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:07 smithi087 ceph-mon[32276]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:07.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:07 smithi195 ceph-mon[37040]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:07.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:07 smithi181 ceph-mon[36790]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:09 smithi195 ceph-mon[37040]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:09.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:09 smithi181 ceph-mon[36790]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:09.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:09 smithi087 ceph-mon[32276]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T07:38:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:10.517 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T07:38:10.517 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:10.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T07:38:10.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:11.321 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:11 smithi087 ceph-mon[32276]: Deploying daemon osd.0 on smithi087 2024-06-06T07:38:11.321 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:11 smithi087 ceph-mon[32276]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:11 smithi195 ceph-mon[37040]: Deploying daemon osd.0 on smithi087 2024-06-06T07:38:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:11 smithi195 ceph-mon[37040]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:11 smithi181 ceph-mon[36790]: Deploying daemon osd.0 on smithi087 2024-06-06T07:38:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:11 smithi181 ceph-mon[36790]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:13.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:13 smithi087 ceph-mon[32276]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:13 smithi195 ceph-mon[37040]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:13 smithi181 ceph-mon[36790]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:15.264 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:15 smithi087 ceph-mon[32276]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:15.264 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:15.264 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:15.264 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:15.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:15 smithi195 ceph-mon[37040]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:15.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:15.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:15.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:15 smithi181 ceph-mon[36790]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:17.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:17 smithi087 ceph-mon[32276]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:17 smithi195 ceph-mon[37040]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:17 smithi181 ceph-mon[36790]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:18.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.264+0000 7f0b61ffb700 1 -- 172.21.15.87:0/2106258658 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b640628d0 con 0x7f0b54039330 2024-06-06T07:38:18.265 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 0 on host 'smithi087' 2024-06-06T07:38:18.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 msgr2=0x7f0b5403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:18.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 0x7f0b5403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b64069f70 tx=0x7f0b5800b410 comp rx=0 tx=0).stop 2024-06-06T07:38:18.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 msgr2=0x7f0b64198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:18.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d9b0 tx=0x7f0b5000dcc0 comp rx=0 tx=0).stop 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 shutdown_connections 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0b54039330 0x7f0b5403b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b641007a0 0x7f0b64198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b64104120 0x7f0b6419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 --2- 172.21.15.87:0/2106258658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103780 0x7f0b64198bb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 >> 172.21.15.87:0/2106258658 conn(0x7f0b640fc120 msgr2=0x7f0b64106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 shutdown_connections 2024-06-06T07:38:18.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:38:18.266+0000 7f0b6b141700 1 -- 172.21.15.87:0/2106258658 wait complete. 2024-06-06T07:38:18.470 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.470 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:18.471 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:18.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:18.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:18.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:18.924 DEBUG:teuthology.orchestra.run.smithi087:osd.0> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.0.service 2024-06-06T07:38:18.928 INFO:tasks.cephadm:Deploying osd.1 on smithi181 with /dev/vg_nvme/lv_4... 2024-06-06T07:38:18.929 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T07:38:19.175 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:38:19.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:19 smithi181 ceph-mon[36790]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T07:38:19.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:19 smithi087 ceph-mon[32276]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T07:38:19.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:19 smithi195 ceph-mon[37040]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T07:38:20.472 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:20 smithi181 ceph-mon[36790]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T07:38:20.473 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:20 smithi181 ceph-mon[36790]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T07:38:20.473 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:20 smithi181 ceph-mon[36790]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:38:20.473 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:20.473 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:20 smithi181 ceph-mon[36790]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-mon[32276]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T07:38:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-mon[32276]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T07:38:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-mon[32276]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:38:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-mon[32276]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:20.611 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0[44748]: 2024-06-06T07:38:20.385+0000 7efcbc339700 -1 osd.0 0 waiting for initial osdmap 2024-06-06T07:38:20.611 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:38:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0[44748]: 2024-06-06T07:38:20.388+0000 7efcb812c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:38:20.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:20 smithi195 ceph-mon[37040]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T07:38:20.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:20 smithi195 ceph-mon[37040]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T07:38:20.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:20 smithi195 ceph-mon[37040]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:38:20.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:20.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:20 smithi195 ceph-mon[37040]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T07:38:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T07:38:21.049 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-06-06T07:38:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-06-06T07:38:21.049 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124136 s, 845 MB/s 2024-06-06T07:38:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-06-06T07:38:21.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:21 smithi181 ceph-mon[36790]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:38:21.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:21 smithi181 ceph-mon[36790]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T07:38:21.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:21 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.674 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:21 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-06-06T07:38:21.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:21 smithi195 ceph-mon[37040]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:38:21.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:21 smithi195 ceph-mon[37040]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T07:38:21.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:21 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:21 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:21 smithi087 ceph-mon[32276]: from='osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:38:21.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:21 smithi087 ceph-mon[32276]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T07:38:21.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:21 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:21 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:21.941 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:38:22.739 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:38:22.739 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:38:22.739 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] boot 2024-06-06T07:38:22.739 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi087 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 17088M 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.740 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:22 smithi181 ceph-mon[36790]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] boot 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi087 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 17088M 2024-06-06T07:38:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:22.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:22.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:22 smithi195 ceph-mon[37040]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:22.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:38:22.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:38:22.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: osd.0 [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] boot 2024-06-06T07:38:22.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi087 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 17088M 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:22.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:22 smithi087 ceph-mon[32276]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T07:38:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 -- 172.21.15.181:0/3797842885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001024e0 msgr2=0x7f3600102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 --2- 172.21.15.181:0/3797842885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001024e0 0x7f3600102900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35f00099c0 tx=0x7f35f0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 -- 172.21.15.181:0/3797842885 shutdown_connections 2024-06-06T07:38:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 --2- 172.21.15.181:0/3797842885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001024e0 0x7f3600102900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 --2- 172.21.15.181:0/3797842885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3600104080 0x7f360010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 --2- 172.21.15.181:0/3797842885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36001036e0 0x7f3600103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 -- 172.21.15.181:0/3797842885 >> 172.21.15.181:0/3797842885 conn(0x7f36000fda80 msgr2=0x7f36000ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:38:23.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 -- 172.21.15.181:0/3797842885 shutdown_connections 2024-06-06T07:38:23.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.529+0000 7f3604f49700 1 -- 172.21.15.181:0/3797842885 wait complete. 2024-06-06T07:38:23.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 Processor -- start 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 -- start start 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36001024e0 0x7f3600198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 0x7f360019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f360010e1a0 con 0x7f3600104080 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f360010e020 con 0x7f36001036e0 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f3604f49700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f360010e320 con 0x7f36001024e0 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f35fdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f35fdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57966/0 (socket says 172.21.15.181:57966) 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.530+0000 7f35fdd9b700 1 -- 172.21.15.181:0/976849480 learned_addr learned my addr 172.21.15.181:0/976849480 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-06-06T07:38:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fe59c700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36001024e0 0x7f3600198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fed9d700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 0x7f360019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 -- 172.21.15.181:0/976849480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36001024e0 msgr2=0x7f3600198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36001024e0 0x7f3600198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 -- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 msgr2=0x7f360019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 0x7f360019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 -- 172.21.15.181:0/976849480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36001a04a0 con 0x7f36001036e0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fed9d700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 0x7f360019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35fdd9b700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b530 tx=0x7f35e800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8015020 con 0x7f36001036e0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36001a0790 con 0x7f36001036e0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8011d10 con 0x7f36001036e0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e800f2e0 con 0x7f36001036e0 2024-06-06T07:38:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.531+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36001a0d50 con 0x7f36001036e0 2024-06-06T07:38:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.532+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3600069180 con 0x7f36001036e0 2024-06-06T07:38:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.533+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f35e8018020 con 0x7f36001036e0 2024-06-06T07:38:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.534+0000 7f35f77fe700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 0x7f35ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.534+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f35e80ccd90 con 0x7f36001036e0 2024-06-06T07:38:23.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.534+0000 7f35fe59c700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 0x7f35ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:23.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.534+0000 7f35fe59c700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 0x7f35ec0a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b580 tx=0x7f35f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:38:23.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.538+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35e80d0030 con 0x7f36001036e0 2024-06-06T07:38:23.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:23 smithi181 ceph-mon[36790]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T07:38:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:23.743+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f36000628d0 con 0x7f35ec0a2b90 2024-06-06T07:38:23.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:23 smithi195 ceph-mon[37040]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T07:38:23.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:23 smithi087 ceph-mon[32276]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T07:38:24.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:24 smithi181 ceph-mon[36790]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:24.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:24.636 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:24.637 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:24.637 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:24 smithi181 ceph-mon[36790]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:24.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:24 smithi195 ceph-mon[37040]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:24.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:24.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:24.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:24.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:24 smithi195 ceph-mon[37040]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:24.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:24 smithi087 ceph-mon[32276]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:24.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:24.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:24.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:24.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:24 smithi087 ceph-mon[32276]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/3066548765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]': finished 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:27.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:27 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/4134996254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/3066548765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]': finished 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T07:38:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:27.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:27 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/4134996254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/3066548765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]: dispatch 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe682-6bd8-4452-8e0c-cb40af20028b"}]': finished 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:27 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/4134996254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:29.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:29 smithi195 ceph-mon[37040]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:29.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:29 smithi181 ceph-mon[36790]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:29.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:29 smithi087 ceph-mon[32276]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:31.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:31 smithi181 ceph-mon[36790]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:31 smithi195 ceph-mon[37040]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:31 smithi087 ceph-mon[32276]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:33.429 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:33 smithi181 ceph-mon[36790]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:33 smithi195 ceph-mon[37040]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:33 smithi087 ceph-mon[32276]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:34.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T07:38:34.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T07:38:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:34.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T07:38:34.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:35.462 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:35 smithi181 ceph-mon[36790]: Deploying daemon osd.1 on smithi181 2024-06-06T07:38:35.462 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:35 smithi181 ceph-mon[36790]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:35.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:35 smithi195 ceph-mon[37040]: Deploying daemon osd.1 on smithi181 2024-06-06T07:38:35.513 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:35 smithi195 ceph-mon[37040]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:35.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:35 smithi087 ceph-mon[32276]: Deploying daemon osd.1 on smithi181 2024-06-06T07:38:35.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:35 smithi087 ceph-mon[32276]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:37.237 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:37 smithi181 ceph-mon[36790]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:37 smithi195 ceph-mon[37040]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:37.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:37 smithi087 ceph-mon[32276]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:39.421 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:39 smithi181 ceph-mon[36790]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:39.421 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.421 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.421 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:39.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:39 smithi195 ceph-mon[37040]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:39.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:39.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:39 smithi087 ceph-mon[32276]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:39.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:39.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:41.290 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:41 smithi181 ceph-mon[36790]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:41 smithi195 ceph-mon[37040]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:41 smithi087 ceph-mon[32276]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:42.557 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 1 on host 'smithi181' 2024-06-06T07:38:42.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.556+0000 7f35f77fe700 1 -- 172.21.15.181:0/976849480 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f36000628d0 con 0x7f35ec0a2b90 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.558+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 msgr2=0x7f35ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.558+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 0x7f35ec0a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b580 tx=0x7f35f0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 msgr2=0x7f3600198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b530 tx=0x7f35e800b840 comp rx=0 tx=0).stop 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 shutdown_connections 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f35ec0a2b90 0x7f35ec0a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3600104080 0x7f360019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:42.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36001024e0 0x7f3600198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:42.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 --2- 172.21.15.181:0/976849480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36001036e0 0x7f3600198d80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:42.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 >> 172.21.15.181:0/976849480 conn(0x7f36000fda80 msgr2=0x7f3600106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:38:42.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 shutdown_connections 2024-06-06T07:38:42.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:38:42.559+0000 7f3604f49700 1 -- 172.21.15.181:0/976849480 wait complete. 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:43.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:43.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:43.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:43.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:43.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:38:43.248 DEBUG:teuthology.orchestra.run.smithi181:osd.1> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.1.service 2024-06-06T07:38:43.250 INFO:tasks.cephadm:Deploying osd.2 on smithi195 with /dev/vg_nvme/lv_4... 2024-06-06T07:38:43.250 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T07:38:43.491 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:38:43.747 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:43 smithi181 ceph-mon[36790]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:43.748 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:43 smithi181 ceph-mon[36790]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:43.917 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:43 smithi195 ceph-mon[37040]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:43.917 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:43 smithi195 ceph-mon[37040]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:44.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:43 smithi087 ceph-mon[32276]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:44.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:43 smithi087 ceph-mon[32276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T07:38:44.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T07:38:44.718 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:44.718 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T07:38:44.718 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:44.719 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:44.719 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:44 smithi195 ceph-mon[37040]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:44.790 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-mon[36790]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:44.790 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1[41406]: 2024-06-06T07:38:44.653+0000 7f4e255d9700 -1 osd.1 0 waiting for initial osdmap 2024-06-06T07:38:44.791 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:38:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1[41406]: 2024-06-06T07:38:44.657+0000 7f4e213cc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: from='osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:38:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:44 smithi087 ceph-mon[32276]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:45.319 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T07:38:45.320 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T07:38:45.320 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-06-06T07:38:45.320 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-06-06T07:38:45.320 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121689 s, 862 MB/s 2024-06-06T07:38:45.320 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-06-06T07:38:45.871 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_4 2024-06-06T07:38:45.893 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:45 smithi195 ceph-mon[37040]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:38:45.894 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:45 smithi195 ceph-mon[37040]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T07:38:45.894 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.894 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.894 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.961 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:45 smithi181 ceph-mon[36790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:38:45.961 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:45 smithi181 ceph-mon[36790]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T07:38:45.961 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.962 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:45.962 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:46.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:45 smithi087 ceph-mon[32276]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:38:46.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:45 smithi087 ceph-mon[32276]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T07:38:46.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:46.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:46.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:46.111 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:38:46.908 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:38:46.908 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] boot 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:46.909 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] boot 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:47.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:47.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: osd.1 [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] boot 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:38:47.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:38:47.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:38:47.616 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- 172.21.15.195:0/3393867032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 msgr2=0x7fa210074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- 172.21.15.195:0/3393867032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa210074920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009990 tx=0x7fa20c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- 172.21.15.195:0/3393867032 shutdown_connections 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- 172.21.15.195:0/3393867032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 0x7fa210110ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- 172.21.15.195:0/3393867032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa210074920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- 172.21.15.195:0/3393867032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa210075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- 172.21.15.195:0/3393867032 >> 172.21.15.195:0/3393867032 conn(0x7fa2100fda90 msgr2=0x7fa2100ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- 172.21.15.195:0/3393867032 shutdown_connections 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- 172.21.15.195:0/3393867032 wait complete. 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 Processor -- start 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- start start 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa21019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:47.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 0x7fa2101a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa210114750 con 0x7fa21010ead0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101145d0 con 0x7fa210074ef0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.614+0000 7fa217d9f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101148d0 con 0x7fa2100764c0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:54578/0 (socket says 172.21.15.195:54578) 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa215b3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa21019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 -- 172.21.15.195:0/4128405249 learned_addr learned my addr 172.21.15.195:0/4128405249 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21633c700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 0x7fa2101a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 -- 172.21.15.195:0/4128405249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 msgr2=0x7fa21019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa21019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 -- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 msgr2=0x7fa2101a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 0x7fa2101a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 -- 172.21.15.195:0/4128405249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2101a6a30 con 0x7fa2100764c0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa215b3b700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa21019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa21533a700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa20000b530 tx=0x7fa20000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200015020 con 0x7fa2100764c0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa200011d10 con 0x7fa2100764c0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200010d90 con 0x7fa2100764c0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2101a6d20 con 0x7fa2100764c0 2024-06-06T07:38:47.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.615+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2101a71b0 con 0x7fa2100764c0 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.616+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21002dca0 con 0x7fa2100764c0 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.617+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa200018020 con 0x7fa2100764c0 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.618+0000 7fa206ffd700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 0x7fa1fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.618+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fa2000cd430 con 0x7fa2100764c0 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.618+0000 7fa215b3b700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 0x7fa1fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:38:47.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.619+0000 7fa215b3b700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 0x7fa1fc0a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa20c003890 tx=0x7fa20c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:38:47.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.622+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa20001f020 con 0x7fa2100764c0 2024-06-06T07:38:47.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:38:47.825+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa2100628d0 con 0x7fa1fc0a2b90 2024-06-06T07:38:47.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:47 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi181 2024-06-06T07:38:47.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:47 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 17088M 2024-06-06T07:38:47.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:47 smithi195 ceph-mon[37040]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T07:38:48.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:47 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi181 2024-06-06T07:38:48.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:47 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 17088M 2024-06-06T07:38:48.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:47 smithi181 ceph-mon[36790]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T07:38:48.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:47 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi181 2024-06-06T07:38:48.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:47 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 17088M 2024-06-06T07:38:48.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:47 smithi087 ceph-mon[32276]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T07:38:48.723 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:48 smithi195 ceph-mon[37040]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:48.723 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:48.723 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:48.723 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:48.723 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:48 smithi195 ceph-mon[37040]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:48 smithi181 ceph-mon[36790]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:48 smithi181 ceph-mon[36790]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:49.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:48 smithi087 ceph-mon[32276]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:38:49.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:38:49.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:38:49.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:49.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:48 smithi087 ceph-mon[32276]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/2300311155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]': finished 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T07:38:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:38:51.512 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:51 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/3431223072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/2300311155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]': finished 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:38:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:51 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/3431223072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/2300311155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]: dispatch 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cabc61d6-3e07-4e07-96bf-474a45e3c752"}]': finished 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:38:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:51 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/3431223072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:38:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:53 smithi195 ceph-mon[37040]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:53 smithi181 ceph-mon[36790]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:53 smithi087 ceph-mon[32276]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:55.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:55 smithi195 ceph-mon[37040]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:55.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:55 smithi181 ceph-mon[36790]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:55 smithi087 ceph-mon[32276]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:57 smithi195 ceph-mon[37040]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:57 smithi181 ceph-mon[36790]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:57 smithi087 ceph-mon[32276]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:58.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T07:38:58.417 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:58.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T07:38:58.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:58.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T07:38:58.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:38:59.356 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:59 smithi195 ceph-mon[37040]: Deploying daemon osd.2 on smithi195 2024-06-06T07:38:59.356 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:38:59 smithi195 ceph-mon[37040]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:59 smithi181 ceph-mon[36790]: Deploying daemon osd.2 on smithi195 2024-06-06T07:38:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:38:59 smithi181 ceph-mon[36790]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:38:59.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:59 smithi087 ceph-mon[32276]: Deploying daemon osd.2 on smithi195 2024-06-06T07:38:59.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:38:59 smithi087 ceph-mon[32276]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:01.231 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:01 smithi195 ceph-mon[37040]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:01 smithi181 ceph-mon[36790]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:01.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:01 smithi087 ceph-mon[32276]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:03.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:03 smithi195 ceph-mon[37040]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:03.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:03 smithi181 ceph-mon[36790]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:03 smithi087 ceph-mon[32276]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:05.411 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:05 smithi195 ceph-mon[37040]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:05 smithi181 ceph-mon[36790]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:05 smithi087 ceph-mon[32276]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:05.743 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 2 on host 'smithi195' 2024-06-06T07:39:05.743 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.741+0000 7fa206ffd700 1 -- 172.21.15.195:0/4128405249 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2100628d0 con 0x7fa1fc0a2b90 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 msgr2=0x7fa1fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 0x7fa1fc0a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa20c003890 tx=0x7fa20c005fd0 comp rx=0 tx=0).stop 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 msgr2=0x7fa21019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa20000b530 tx=0x7fa20000b840 comp rx=0 tx=0).stop 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 shutdown_connections 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa1fc0a2b90 0x7fa1fc0a5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:05.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa210074ef0 0x7fa21019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:05.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa21010ead0 0x7fa2101a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:05.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 --2- 172.21.15.195:0/4128405249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2100764c0 0x7fa21019f310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:05.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.743+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 >> 172.21.15.195:0/4128405249 conn(0x7fa2100fda90 msgr2=0x7fa210101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:39:05.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.744+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 shutdown_connections 2024-06-06T07:39:05.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:39:05.744+0000 7fa217d9f700 1 -- 172.21.15.195:0/4128405249 wait complete. 2024-06-06T07:39:06.334 DEBUG:teuthology.orchestra.run.smithi195:osd.2> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.2.service 2024-06-06T07:39:06.377 INFO:tasks.cephadm:Deploying osd.3 on smithi087 with /dev/vg_nvme/lv_3... 2024-06-06T07:39:06.377 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T07:39:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:39:07.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:07.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-mon[37040]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:07.011 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T07:39:06.586+0000 7f7ebec767c0 -1 osd.2 0 log_to_monitors true 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:07.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:06 smithi181 ceph-mon[36790]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:07.068 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:06 smithi087 ceph-mon[32276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T07:39:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T07:39:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T07:39:08.580 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-06-06T07:39:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-06-06T07:39:08.580 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129521 s, 810 MB/s 2024-06-06T07:39:08.581 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.639 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:08 smithi087 ceph-mon[32276]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:08.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T07:39:08.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T07:39:08.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:08.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:08 smithi195 ceph-mon[37040]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: from='osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:39:08.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:08 smithi181 ceph-mon[36790]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:09.245 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T07:39:09.091+0000 7f7eb4adb700 -1 osd.2 0 waiting for initial osdmap 2024-06-06T07:39:09.246 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T07:39:09.096+0000 7f7eac0c5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:39:09.297 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi087:vg_nvme/lv_3 2024-06-06T07:39:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-mon[37040]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:39:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-mon[37040]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T07:39:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:09 smithi195 ceph-mon[37040]: from='osd.2 ' entity='osd.2' 2024-06-06T07:39:09.535 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:39:09.593 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:09 smithi087 ceph-mon[32276]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:39:09.593 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:09 smithi087 ceph-mon[32276]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T07:39:09.593 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.593 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.594 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:09 smithi087 ceph-mon[32276]: from='osd.2 ' entity='osd.2' 2024-06-06T07:39:09.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:09 smithi181 ceph-mon[36790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:39:09.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:09 smithi181 ceph-mon[36790]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T07:39:09.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:09.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:09 smithi181 ceph-mon[36790]: from='osd.2 ' entity='osd.2' 2024-06-06T07:39:10.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:39:10.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:39:10.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi195 2024-06-06T07:39:10.702 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 17088M 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] boot 2024-06-06T07:39:10.703 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T07:39:10.704 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.704 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:10 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi195 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 17088M 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] boot 2024-06-06T07:39:10.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T07:39:10.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:10 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi195 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 17088M 2024-06-06T07:39:10.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: osd.2 [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] boot 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:39:10.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:10 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:11.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.227+0000 7fad01af6700 1 -- 172.21.15.87:0/423203922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 msgr2=0x7facfc10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.227+0000 7fad01af6700 1 --2- 172.21.15.87:0/423203922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc10a650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facf400b0a0 tx=0x7facf400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 -- 172.21.15.87:0/423203922 shutdown_connections 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 --2- 172.21.15.87:0/423203922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 0x7facfc105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 --2- 172.21.15.87:0/423203922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc10a650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 --2- 172.21.15.87:0/423203922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 0x7facfc107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 -- 172.21.15.87:0/423203922 >> 172.21.15.87:0/423203922 conn(0x7facfc0fc990 msgr2=0x7facfc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 -- 172.21.15.87:0/423203922 shutdown_connections 2024-06-06T07:39:11.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 -- 172.21.15.87:0/423203922 wait complete. 2024-06-06T07:39:11.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 Processor -- start 2024-06-06T07:39:11.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 -- start start 2024-06-06T07:39:11.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 0x7facfc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.228+0000 7fad01af6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 0x7facfc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10e0c0 con 0x7facfc1081e0 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10df40 con 0x7facfc102f80 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10e240 con 0x7facfc1058b0 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58792/0 (socket says 172.21.15.87:58792) 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 -- 172.21.15.87:0/2702688071 learned_addr learned my addr 172.21.15.87:0/2702688071 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfb7fe700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 0x7facfc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:11.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facf3fff700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 0x7facfc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 -- 172.21.15.87:0/2702688071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 msgr2=0x7facfc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 0x7facfc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 -- 172.21.15.87:0/2702688071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 msgr2=0x7facfc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 0x7facfc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 -- 172.21.15.87:0/2702688071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc1a0340 con 0x7facfc1081e0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facfbfff700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7facf4004d00 tx=0x7facf4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4021070 con 0x7facfc1081e0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf4007b80 con 0x7facfc1081e0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facfc1a05d0 con 0x7facfc1081e0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4005280 con 0x7facfc1081e0 2024-06-06T07:39:11.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.229+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facfc1a0a80 con 0x7facfc1081e0 2024-06-06T07:39:11.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.230+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facfc069180 con 0x7facfc1081e0 2024-06-06T07:39:11.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.231+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7facf4024070 con 0x7facfc1081e0 2024-06-06T07:39:11.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.232+0000 7facf97fa700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 0x7face80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:11.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.232+0000 7facfb7fe700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 0x7face80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:11.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.232+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7facf400f020 con 0x7facfc1081e0 2024-06-06T07:39:11.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.233+0000 7facfb7fe700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 0x7face80a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facec009c60 tx=0x7facec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:39:11.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.236+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7facf402b020 con 0x7facfc1081e0 2024-06-06T07:39:11.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:11.443+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7facfc0628d0 con 0x7face80a2b90 2024-06-06T07:39:12.323 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:12.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T07:39:12.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:12.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:12.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:12.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:12.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T07:39:12.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:12.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:12.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:12.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:12.610 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:12.610 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:12.610 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:12.610 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49234]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:13.032 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49240]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:13.032 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:13.032 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49240]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:13.032 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 07:39:12 smithi087 sudo[49240]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:13.216 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:12 smithi181 sudo[45220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:13.216 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:12 smithi181 sudo[45220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:13.217 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:12 smithi181 sudo[45220]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:13.217 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:12 smithi181 sudo[45220]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T07:39:13.217 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:13 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:13.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:13.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:13.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:39:13.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:13.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T07:39:13.316 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:13 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T07:39:13.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:13 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:13.557 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:13 smithi181 sudo[45223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:13.557 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:13 smithi181 sudo[45223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:13.557 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:13 smithi181 sudo[45223]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:13.557 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 07:39:13 smithi181 sudo[45223]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:13.737 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46228]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:13.737 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46228]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:13.737 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46228]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:13.737 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46228]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:14.010 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:14.011 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:14.011 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:14.011 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 07:39:13 smithi195 sudo[46231]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:14.275 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:14.275 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:14.275 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49305]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:14.275 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49305]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 ceph-mon[37040]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:14.532 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:14.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:14.533 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46234]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:14 smithi181 ceph-mon[36790]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T07:39:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:14 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:14 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:14 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 sudo[49308]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 ceph-mon[32276]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:14.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:14 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:15.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:15.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46237]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:15.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:14 smithi195 sudo[46237]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:15.262 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T07:39:15.262 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:15.262 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45226]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:15.262 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45226]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:15.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:15.272+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7facf4099860 con 0x7facfc1081e0 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45229]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45229]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45229]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 sudo[45229]: pam_unix(sudo:session): session closed for user root 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:15.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]': finished 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:15 smithi181 ceph-mon[36790]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]': finished 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.612 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:15 smithi087 ceph-mon[32276]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T07:39:15.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:15.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3590210430' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1da26cda-ee26-4b26-b7c8-fed45cdac09a"}]': finished 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:39:15.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:39:15.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T07:39:15.763 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:15 smithi195 ceph-mon[37040]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T07:39:16.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:16 smithi181 ceph-mon[36790]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:39:16.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:16 smithi181 ceph-mon[36790]: Cluster is now healthy 2024-06-06T07:39:16.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:16 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/964726905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:16.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:16 smithi181 ceph-mon[36790]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:16.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:16 smithi087 ceph-mon[32276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:39:16.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:16 smithi087 ceph-mon[32276]: Cluster is now healthy 2024-06-06T07:39:16.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:16 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/964726905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:16.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:16 smithi087 ceph-mon[32276]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:16 smithi195 ceph-mon[37040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:39:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:16 smithi195 ceph-mon[37040]: Cluster is now healthy 2024-06-06T07:39:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:16 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/964726905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:16 smithi195 ceph-mon[37040]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:19.219 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:19 smithi087 ceph-mon[32276]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:19 smithi195 ceph-mon[37040]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:19.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:19 smithi181 ceph-mon[36790]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:21 smithi195 ceph-mon[37040]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:21.542 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:21 smithi087 ceph-mon[32276]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:21 smithi181 ceph-mon[36790]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:23 smithi087 ceph-mon[32276]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T07:39:23.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:23 smithi195 ceph-mon[37040]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:23 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T07:39:23.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:23 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:23 smithi181 ceph-mon[36790]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:23.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:23 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T07:39:23.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:23 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:24.349 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:24 smithi087 ceph-mon[32276]: Deploying daemon osd.3 on smithi087 2024-06-06T07:39:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:24 smithi195 ceph-mon[37040]: Deploying daemon osd.3 on smithi087 2024-06-06T07:39:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:24 smithi181 ceph-mon[36790]: Deploying daemon osd.3 on smithi087 2024-06-06T07:39:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:25 smithi087 ceph-mon[32276]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:25.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:25 smithi181 ceph-mon[36790]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:25 smithi195 ceph-mon[37040]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:27.509 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:27 smithi087 ceph-mon[32276]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:27 smithi195 ceph-mon[37040]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:27 smithi181 ceph-mon[36790]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:29.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:29.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:28 smithi181 ceph-mon[36790]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:29.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:29.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:28 smithi087 ceph-mon[32276]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:29.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:28 smithi195 ceph-mon[37040]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:31.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.497+0000 7facf97fa700 1 -- 172.21.15.87:0/2702688071 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7facfc0628d0 con 0x7face80a2b90 2024-06-06T07:39:31.497 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 3 on host 'smithi087' 2024-06-06T07:39:31.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 msgr2=0x7face80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 0x7face80a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facec009c60 tx=0x7facec009260 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 msgr2=0x7facfc19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7facf4004d00 tx=0x7facf4004d30 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 shutdown_connections 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7face80a2b90 0x7face80a5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc102f80 0x7facfc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facfc1081e0 0x7facfc19fca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 --2- 172.21.15.87:0/2702688071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facfc1058b0 0x7facfc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 >> 172.21.15.87:0/2702688071 conn(0x7facfc0fc990 msgr2=0x7facfc1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 shutdown_connections 2024-06-06T07:39:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:39:31.499+0000 7fad01af6700 1 -- 172.21.15.87:0/2702688071 wait complete. 2024-06-06T07:39:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:31.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.184 DEBUG:teuthology.orchestra.run.smithi087:osd.3> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.3.service 2024-06-06T07:39:32.186 INFO:tasks.cephadm:Deploying osd.4 on smithi181 with /dev/vg_nvme/lv_3... 2024-06-06T07:39:32.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T07:39:32.432 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:39:32.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.554 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.554 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:32.554 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:32 smithi181 ceph-mon[36790]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T07:39:32.554 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:32 smithi181 ceph-mon[36790]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:32.720 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.720 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.720 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:32.720 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:32 smithi087 ceph-mon[32276]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T07:39:32.720 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:32 smithi087 ceph-mon[32276]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:32 smithi195 ceph-mon[37040]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T07:39:32.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:32 smithi195 ceph-mon[37040]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:33 smithi087 ceph-mon[32276]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T07:39:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:33 smithi087 ceph-mon[32276]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T07:39:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:33 smithi087 ceph-mon[32276]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:39:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:33.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:33 smithi195 ceph-mon[37040]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T07:39:33.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:33 smithi195 ceph-mon[37040]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T07:39:33.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:33 smithi195 ceph-mon[37040]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:39:33.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:33.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:33 smithi181 ceph-mon[36790]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T07:39:33.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:33 smithi181 ceph-mon[36790]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T07:39:33.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:33 smithi181 ceph-mon[36790]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:39:33.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.471 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T07:39:34.175+0000 7f055bdf2700 -1 osd.3 0 waiting for initial osdmap 2024-06-06T07:39:34.471 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T07:39:34.181+0000 7f0553bdd700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:39:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T07:39:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T07:39:34.560 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-06-06T07:39:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-06-06T07:39:34.561 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114886 s, 913 MB/s 2024-06-06T07:39:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-06-06T07:39:34.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] boot 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T07:39:34.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] boot 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T07:39:34.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: from='osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594]' entity='osd.3' 2024-06-06T07:39:34.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: osd.3 [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] boot 2024-06-06T07:39:34.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T07:39:34.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:39:35.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-06-06T07:39:35.481 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:39:35.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:35 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:39:35.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:35 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:39:35.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:35 smithi181 ceph-mon[36790]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T07:39:35.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:35 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:39:35.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:35 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:39:35.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:35 smithi195 ceph-mon[37040]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T07:39:35.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:35 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:39:35.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:35 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:39:35.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:35 smithi087 ceph-mon[32276]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T07:39:36.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi087 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 8544M 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:36.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:36 smithi181 ceph-mon[36790]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi087 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:36.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 8544M 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:36.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:36 smithi087 ceph-mon[32276]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T07:39:37.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi087 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 8544M 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:37.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:36 smithi195 ceph-mon[37040]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T07:39:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/777430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 msgr2=0x7f3c7810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/777430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c7810a930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b0d0 tx=0x7f3c7000b3e0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/777430896 shutdown_connections 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/777430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c7810a930 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/777430896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781024e0 0x7f3c78102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/777430896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781036e0 0x7f3c78103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/777430896 >> 172.21.15.181:0/777430896 conn(0x7f3c780fda80 msgr2=0x7f3c780ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/777430896 shutdown_connections 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/777430896 wait complete. 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 Processor -- start 2024-06-06T07:39:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.121+0000 7f3c7e9e7700 1 -- start start 2024-06-06T07:39:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781024e0 0x7f3c78076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781036e0 0x7f3c78074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810e0b0 con 0x7f3c781024e0 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810df30 con 0x7f3c78104080 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810e230 con 0x7f3c781036e0 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46136/0 (socket says 172.21.15.181:46136) 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 -- 172.21.15.181:0/2363091334 learned_addr learned my addr 172.21.15.181:0/2363091334 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c777fe700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781036e0 0x7f3c78074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c77fff700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781024e0 0x7f3c78076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 -- 172.21.15.181:0/2363091334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781036e0 msgr2=0x7f3c78074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781036e0 0x7f3c78074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 -- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781024e0 msgr2=0x7f3c78076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781024e0 0x7f3c78076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 -- 172.21.15.181:0/2363091334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c78075670 con 0x7f3c78104080 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7cf84700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c700062a0 tx=0x7f3c70007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70003c00 con 0x7f3c78104080 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c781a2730 con 0x7f3c78104080 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.122+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70003da0 con 0x7f3c78104080 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.123+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c700055d0 con 0x7f3c78104080 2024-06-06T07:39:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.123+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a2c10 con 0x7f3c78104080 2024-06-06T07:39:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.124+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c78069180 con 0x7f3c78104080 2024-06-06T07:39:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.125+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3c70005770 con 0x7f3c78104080 2024-06-06T07:39:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.125+0000 7f3c757fa700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 0x7f3c600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:39:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.125+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f3c7000f020 con 0x7f3c78104080 2024-06-06T07:39:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.125+0000 7f3c77fff700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 0x7f3c600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:39:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.126+0000 7f3c77fff700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 0x7f3c600a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c68005fd0 tx=0x7f3c68005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:39:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.130+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c7002b020 con 0x7f3c78104080 2024-06-06T07:39:37.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:37.335+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3c780628d0 con 0x7f3c600a2c20 2024-06-06T07:39:37.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:37 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:37.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:37 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:37.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:37 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:37.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:37 smithi087 ceph-mon[32276]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T07:39:37.947 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:37 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:37.947 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:37 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:37.948 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:37 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:37.948 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:37 smithi181 ceph-mon[36790]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T07:39:38.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:37 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:39:38.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:37 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:39:38.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:37 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:38.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:37 smithi195 ceph-mon[37040]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T07:39:38.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:38 smithi087 ceph-mon[32276]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:38.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:38 smithi087 ceph-mon[32276]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:39:38.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:38 smithi087 ceph-mon[32276]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T07:39:38.979 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:38 smithi181 ceph-mon[36790]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:38.979 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:38 smithi181 ceph-mon[36790]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:39:38.979 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:38 smithi181 ceph-mon[36790]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T07:39:39.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:38 smithi195 ceph-mon[37040]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:39:39.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:38 smithi195 ceph-mon[37040]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 396 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:39:39.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:38 smithi195 ceph-mon[37040]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T07:39:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/3066534104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]': finished 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:41.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:41 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/3198072139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/3066534104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]': finished 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:41.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:41 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/3198072139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/3066534104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]: dispatch 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3da9aeb5-12e1-46c8-8369-e1ee920ca615"}]': finished 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/3198072139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:39:43.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:43 smithi087 ceph-mon[32276]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-06T07:39:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:43 smithi195 ceph-mon[37040]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-06T07:39:43.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:43 smithi181 ceph-mon[36790]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-06T07:39:45.393 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:45 smithi181 ceph-mon[36790]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:45 smithi195 ceph-mon[37040]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:45.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:45 smithi087 ceph-mon[32276]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:47.432 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:47 smithi181 ceph-mon[36790]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:47 smithi195 ceph-mon[37040]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:47.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:47 smithi087 ceph-mon[32276]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:48.368 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T07:39:48.368 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T07:39:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:48.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T07:39:48.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:39:49.466 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:49 smithi181 ceph-mon[36790]: Deploying daemon osd.4 on smithi181 2024-06-06T07:39:49.467 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:49 smithi181 ceph-mon[36790]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:49.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:49 smithi195 ceph-mon[37040]: Deploying daemon osd.4 on smithi181 2024-06-06T07:39:49.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:49 smithi195 ceph-mon[37040]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:49 smithi087 ceph-mon[32276]: Deploying daemon osd.4 on smithi181 2024-06-06T07:39:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:49 smithi087 ceph-mon[32276]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:51.492 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:51 smithi181 ceph-mon[36790]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:51 smithi195 ceph-mon[37040]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:51.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:51 smithi087 ceph-mon[32276]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:53.426 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:53 smithi181 ceph-mon[36790]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:53.426 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.426 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.426 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:53 smithi195 ceph-mon[37040]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:53 smithi087 ceph-mon[32276]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:39:55.272 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:55 smithi181 ceph-mon[36790]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:55 smithi195 ceph-mon[37040]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:55 smithi087 ceph-mon[32276]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:56.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.499+0000 7f3c757fa700 1 -- 172.21.15.181:0/2363091334 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c780628d0 con 0x7f3c600a2c20 2024-06-06T07:39:56.500 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 4 on host 'smithi181' 2024-06-06T07:39:56.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 msgr2=0x7f3c600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:56.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 0x7f3c600a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c68005fd0 tx=0x7f3c68005e50 comp rx=0 tx=0).stop 2024-06-06T07:39:56.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 msgr2=0x7f3c78074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c700062a0 tx=0x7f3c70007e40 comp rx=0 tx=0).stop 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 shutdown_connections 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c600a2c20 0x7f3c600a50e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c78104080 0x7f3c78074dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c781024e0 0x7f3c78076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 --2- 172.21.15.181:0/2363091334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c781036e0 0x7f3c78074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 >> 172.21.15.181:0/2363091334 conn(0x7f3c780fda80 msgr2=0x7f3c78106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 shutdown_connections 2024-06-06T07:39:56.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:39:56.502+0000 7f3c7e9e7700 1 -- 172.21.15.181:0/2363091334 wait complete. 2024-06-06T07:39:57.232 DEBUG:teuthology.orchestra.run.smithi181:osd.4> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.4.service 2024-06-06T07:39:57.235 INFO:tasks.cephadm:Deploying osd.5 on smithi195 with /dev/vg_nvme/lv_3... 2024-06-06T07:39:57.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T07:39:57.423 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:57.423 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.424 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.424 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.424 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.424 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:57.424 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:57 smithi181 ceph-mon[36790]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:57.479 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:57 smithi195 ceph-mon[37040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:57.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:57 smithi087 ceph-mon[32276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.700 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-mon[36790]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:58.700 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T07:39:58.573+0000 7faee3571700 -1 osd.4 0 waiting for initial osdmap 2024-06-06T07:39:58.700 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 07:39:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T07:39:58.578+0000 7faedf364700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:39:58.714 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T07:39:58.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T07:39:58.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:58.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.715 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:58 smithi195 ceph-mon[37040]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: from='osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:39:58.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:58 smithi087 ceph-mon[32276]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00876702 s, 1.2 GB/s 2024-06-06T07:39:59.262 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-06-06T07:39:59.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:59 smithi181 ceph-mon[36790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:39:59.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:59 smithi181 ceph-mon[36790]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T07:39:59.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:39:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:59 smithi087 ceph-mon[32276]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:39:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:59 smithi087 ceph-mon[32276]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T07:39:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:39:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.873 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:59 smithi195 ceph-mon[37040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:39:59.874 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:59 smithi195 ceph-mon[37040]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T07:39:59.874 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.874 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:39:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:39:59.902 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_3 2024-06-06T07:40:00.138 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] boot 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:40:00.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: overall HEALTH_OK 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi181 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 8544M 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:00.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:00.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:00 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] boot 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: overall HEALTH_OK 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi181 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 8544M 2024-06-06T07:40:01.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:01.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:00 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: osd.4 [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] boot 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: overall HEALTH_OK 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:01.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi181 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 8544M 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:01.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:01.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:00 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.713+0000 7f07f5307700 1 -- 172.21.15.195:0/1778903665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f01025f0 msgr2=0x7f07f0102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.713+0000 7f07f5307700 1 --2- 172.21.15.195:0/1778903665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f01025f0 0x7f07f0102a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009990 tx=0x7f07e4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- 172.21.15.195:0/1778903665 shutdown_connections 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- 172.21.15.195:0/1778903665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01013f0 0x7f07f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- 172.21.15.195:0/1778903665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f0102f90 0x7f07f0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- 172.21.15.195:0/1778903665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f01025f0 0x7f07f0102a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- 172.21.15.195:0/1778903665 >> 172.21.15.195:0/1778903665 conn(0x7f07f00fc9b0 msgr2=0x7f07f00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- 172.21.15.195:0/1778903665 shutdown_connections 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- 172.21.15.195:0/1778903665 wait complete. 2024-06-06T07:40:01.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 Processor -- start 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- start start 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 0x7f07f006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01025f0 0x7f07f019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 0x7f07f019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07f010cfc0 con 0x7f07f01013f0 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f07f010ce40 con 0x7f07f01025f0 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.714+0000 7f07f5307700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07f010d140 con 0x7f07f0102f90 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07eeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 0x7f07f006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07eeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 0x7f07f006b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.195:37452/0 (socket says 172.21.15.195:37452) 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07eeffd700 1 -- 172.21.15.195:0/934240876 learned_addr learned my addr 172.21.15.195:0/934240876 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 0x7f07f019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:01.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ee7fc700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01025f0 0x7f07f019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 -- 172.21.15.195:0/934240876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01025f0 msgr2=0x7f07f019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01025f0 0x7f07f019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 -- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 msgr2=0x7f07f006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 0x7f07f006b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 -- 172.21.15.195:0/934240876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f01a0370 con 0x7f07f0102f90 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07ef7fe700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 0x7f07f019faa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f07e8007f10 tx=0x7f07e800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8004db0 con 0x7f07f0102f90 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f01a0660 con 0x7f07f0102f90 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e8009e20 con 0x7f07f0102f90 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015400 con 0x7f07f0102f90 2024-06-06T07:40:01.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.715+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f01a0b20 con 0x7f07f0102f90 2024-06-06T07:40:01.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.716+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f0069180 con 0x7f07f0102f90 2024-06-06T07:40:01.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.717+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f07e800db20 con 0x7f07f0102f90 2024-06-06T07:40:01.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.718+0000 7f07d7fff700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 0x7f07d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:01.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.718+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f07e80cd930 con 0x7f07f0102f90 2024-06-06T07:40:01.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.718+0000 7f07eeffd700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 0x7f07d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:01.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.718+0000 7f07eeffd700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 0x7f07d80a50e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bd70 tx=0x7f07e000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:01.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.723+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f07e8010070 con 0x7f07f0102f90 2024-06-06T07:40:01.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:01 smithi195 ceph-mon[37040]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T07:40:01.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:01 smithi087 ceph-mon[32276]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T07:40:01.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:01.928+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f07f00628d0 con 0x7f07d80a2c20 2024-06-06T07:40:02.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:01 smithi181 ceph-mon[36790]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T07:40:02.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:02 smithi087 ceph-mon[32276]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:02.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:02 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:02.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:02 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:02.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:02 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:02.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:02 smithi087 ceph-mon[32276]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:02.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:02 smithi195 ceph-mon[37040]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:02.871 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:02 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:02.871 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:02 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:02.871 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:02 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:02.871 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:02 smithi195 ceph-mon[37040]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:03.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:02 smithi181 ceph-mon[36790]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:03.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:02 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:03.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:02 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:03.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:02 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:03.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:02 smithi181 ceph-mon[36790]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/3972962522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]': finished 2024-06-06T07:40:05.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T07:40:05.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:05 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/3972962522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]': finished 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T07:40:05.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:05 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/3972962522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]: dispatch 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fd8d26c-d2e3-4542-9624-1d5c7cbeb684"}]': finished 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T07:40:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:05 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:06 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/3053515137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:06 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/3053515137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:06 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/3053515137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:07 smithi195 ceph-mon[37040]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:07 smithi181 ceph-mon[36790]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:07 smithi087 ceph-mon[32276]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:09.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:09 smithi195 ceph-mon[37040]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:09 smithi087 ceph-mon[32276]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:09 smithi181 ceph-mon[36790]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:11 smithi195 ceph-mon[37040]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:11 smithi181 ceph-mon[36790]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:11.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:11 smithi087 ceph-mon[32276]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:12.431 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T07:40:12.431 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T07:40:12.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T07:40:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:13.419 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:13 smithi195 ceph-mon[37040]: Deploying daemon osd.5 on smithi195 2024-06-06T07:40:13.420 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:13 smithi195 ceph-mon[37040]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:13 smithi087 ceph-mon[32276]: Deploying daemon osd.5 on smithi195 2024-06-06T07:40:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:13 smithi087 ceph-mon[32276]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:13 smithi181 ceph-mon[36790]: Deploying daemon osd.5 on smithi195 2024-06-06T07:40:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:13 smithi181 ceph-mon[36790]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:15 smithi195 ceph-mon[37040]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:15.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:15 smithi181 ceph-mon[36790]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:15.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:15 smithi087 ceph-mon[32276]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:17.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:17 smithi195 ceph-mon[37040]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:17 smithi181 ceph-mon[36790]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:17 smithi087 ceph-mon[32276]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:18.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:18.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:18 smithi181 ceph-mon[36790]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:18.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:18.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:18 smithi087 ceph-mon[32276]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:18.651 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.652 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:18.652 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:18.652 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:18 smithi195 ceph-mon[37040]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:20.844 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 5 on host 'smithi195' 2024-06-06T07:40:20.844 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.843+0000 7f07d7fff700 1 -- 172.21.15.195:0/934240876 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f07f00628d0 con 0x7f07d80a2c20 2024-06-06T07:40:20.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 msgr2=0x7f07d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 0x7f07d80a50e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bd70 tx=0x7f07e000b4a0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 msgr2=0x7f07f019faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 0x7f07f019faa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f07e8007f10 tx=0x7f07e800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 shutdown_connections 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07d80a2c20 0x7f07d80a50e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f0102f90 0x7f07f019faa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07f01013f0 0x7f07f006b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 --2- 172.21.15.195:0/934240876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f01025f0 0x7f07f019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 >> 172.21.15.195:0/934240876 conn(0x7f07f00fc9b0 msgr2=0x7f07f0105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 shutdown_connections 2024-06-06T07:40:20.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:40:20.846+0000 7f07f5307700 1 -- 172.21.15.195:0/934240876 wait complete. 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:21.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:21.489 DEBUG:teuthology.orchestra.run.smithi195:osd.5> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.5.service 2024-06-06T07:40:21.491 INFO:tasks.cephadm:Deploying osd.6 on smithi087 with /dev/vg_nvme/lv_2... 2024-06-06T07:40:21.491 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T07:40:21.736 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:40:22.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:21 smithi195 ceph-mon[37040]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:22.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:21 smithi195 ceph-mon[37040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:22.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:21 smithi087 ceph-mon[32276]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:22.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:21 smithi087 ceph-mon[32276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:21 smithi181 ceph-mon[36790]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:22.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:21 smithi181 ceph-mon[36790]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:22 smithi087 ceph-mon[32276]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:23.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T07:40:23.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T07:40:23.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:23.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:22 smithi195 ceph-mon[37040]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:23.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T07:40:23.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: from='osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T07:40:23.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:23.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:40:23.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:22 smithi181 ceph-mon[36790]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123936 s, 846 MB/s 2024-06-06T07:40:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-06-06T07:40:23.888 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T07:40:23.540+0000 7f543083c700 -1 osd.5 0 waiting for initial osdmap 2024-06-06T07:40:23.889 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T07:40:23.548+0000 7f5427e26700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:40:24.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-mon[37040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:40:24.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-mon[37040]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T07:40:24.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:23 smithi195 ceph-mon[37040]: from='osd.5 ' entity='osd.5' 2024-06-06T07:40:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:23 smithi181 ceph-mon[36790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:40:24.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:23 smithi181 ceph-mon[36790]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T07:40:24.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:23 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:23 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:23 smithi181 ceph-mon[36790]: from='osd.5 ' entity='osd.5' 2024-06-06T07:40:24.321 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:23 smithi087 ceph-mon[32276]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:40:24.322 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:23 smithi087 ceph-mon[32276]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T07:40:24.322 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.322 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:23 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:24.322 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:23 smithi087 ceph-mon[32276]: from='osd.5 ' entity='osd.5' 2024-06-06T07:40:24.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi087:vg_nvme/lv_2 2024-06-06T07:40:24.703 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:40:25.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] boot 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.112 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:24 smithi087 ceph-mon[32276]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] boot 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:25.208 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.209 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:24 smithi181 ceph-mon[36790]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T07:40:25.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:40:25.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: osd.5 [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] boot 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:25.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:24 smithi195 ceph-mon[37040]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T07:40:26.177 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:25 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi195 2024-06-06T07:40:26.178 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:25 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 8544M 2024-06-06T07:40:26.178 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:25 smithi087 ceph-mon[32276]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T07:40:26.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:25 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi195 2024-06-06T07:40:26.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:25 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 8544M 2024-06-06T07:40:26.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:25 smithi195 ceph-mon[37040]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T07:40:26.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 -- 172.21.15.87:0/2005900453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0101670 msgr2=0x7f17e0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:26.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/2005900453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0101670 0x7f17e0101af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009990 tx=0x7f17c8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 -- 172.21.15.87:0/2005900453 shutdown_connections 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/2005900453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0102030 0x7f17e01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/2005900453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/2005900453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0101670 0x7f17e0101af0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 -- 172.21.15.87:0/2005900453 >> 172.21.15.87:0/2005900453 conn(0x7f17e00fc140 msgr2=0x7f17e00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 -- 172.21.15.87:0/2005900453 shutdown_connections 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.281+0000 7f17e4eb8700 1 -- 172.21.15.87:0/2005900453 wait complete. 2024-06-06T07:40:26.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 Processor -- start 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 -- start start 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 0x7f17e0105410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010d060 con 0x7f17e0102030 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010cee0 con 0x7f17e0101670 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17e4eb8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010d1e0 con 0x7f17e0102aa0 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.282+0000 7f17ddd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58050/0 (socket says 172.21.15.87:58050) 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 -- 172.21.15.87:0/3894272919 learned_addr learned my addr 172.21.15.87:0/3894272919 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:40:26.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17de59c700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 0x7f17e0105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ded9d700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 -- 172.21.15.87:0/3894272919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 msgr2=0x7f17e0103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 -- 172.21.15.87:0/3894272919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 msgr2=0x7f17e0105410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 0x7f17e0105410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 -- 172.21.15.87:0/3894272919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e0104850 con 0x7f17e0102030 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17de59c700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 0x7f17e0105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ded9d700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17ddd9b700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005910 tx=0x7f17c800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8026070 con 0x7f17e0102030 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c800f460 con 0x7f17e0102030 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a28a0 con 0x7f17e0102030 2024-06-06T07:40:26.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8021670 con 0x7f17e0102030 2024-06-06T07:40:26.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.283+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a2cf0 con 0x7f17e0102030 2024-06-06T07:40:26.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.284+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e0069180 con 0x7f17e0102030 2024-06-06T07:40:26.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.285+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f17c8029070 con 0x7f17e0102030 2024-06-06T07:40:26.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.286+0000 7f17d77fe700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 0x7f17cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:26.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.286+0000 7f17de59c700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 0x7f17cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:26.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.286+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f17c80cf110 con 0x7f17e0102030 2024-06-06T07:40:26.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.286+0000 7f17de59c700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 0x7f17cc0a5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009de0 tx=0x7f17d0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:26.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.290+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17c805d850 con 0x7f17e0102030 2024-06-06T07:40:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:25 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi195 2024-06-06T07:40:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:25 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 8544M 2024-06-06T07:40:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:25 smithi181 ceph-mon[36790]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T07:40:26.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:26.497+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f17e00fe290 con 0x7f17cc0a2ca0 2024-06-06T07:40:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:26 smithi195 ceph-mon[37040]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:26 smithi195 ceph-mon[37040]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T07:40:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:26 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:26 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:26 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:27.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:26 smithi181 ceph-mon[36790]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:27.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:26 smithi181 ceph-mon[36790]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T07:40:27.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:26 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:27.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:26 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:27.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:26 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:27.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:26 smithi087 ceph-mon[32276]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:27.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:26 smithi087 ceph-mon[32276]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T07:40:27.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:26 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:27.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:26 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:27.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:26 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:28.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:27 smithi195 ceph-mon[37040]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:28.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:27 smithi195 ceph-mon[37040]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T07:40:28.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:27 smithi087 ceph-mon[32276]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:28.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:27 smithi087 ceph-mon[32276]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T07:40:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:27 smithi181 ceph-mon[36790]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:27 smithi181 ceph-mon[36790]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T07:40:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:28 smithi195 ceph-mon[37040]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:40:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:28 smithi181 ceph-mon[36790]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:40:29.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:28 smithi087 ceph-mon[32276]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T07:40:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]: dispatch 2024-06-06T07:40:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]': finished 2024-06-06T07:40:30.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:29 smithi195 ceph-mon[37040]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T07:40:30.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:30.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1429009843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:30.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]: dispatch 2024-06-06T07:40:30.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]': finished 2024-06-06T07:40:30.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:29 smithi181 ceph-mon[36790]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T07:40:30.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:29 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:30.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1429009843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]: dispatch 2024-06-06T07:40:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1244780284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12b3728b-ea86-47f0-8f77-13631e3eded7"}]': finished 2024-06-06T07:40:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:29 smithi087 ceph-mon[32276]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T07:40:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:29 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1429009843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:31.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:30 smithi195 ceph-mon[37040]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:40:31.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:30 smithi181 ceph-mon[36790]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:40:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:30 smithi087 ceph-mon[32276]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:40:33.276 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:33 smithi087 ceph-mon[32276]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:40:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:33 smithi195 ceph-mon[37040]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:40:33.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:33 smithi181 ceph-mon[36790]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:40:35.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:35 smithi087 ceph-mon[32276]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:40:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:35 smithi195 ceph-mon[37040]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:40:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:35 smithi181 ceph-mon[36790]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:40:37.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:37 smithi087 ceph-mon[32276]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:37.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:37 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T07:40:37.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:37 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:37 smithi195 ceph-mon[37040]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:37 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T07:40:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:37 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:37 smithi181 ceph-mon[36790]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:37 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T07:40:37.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:37 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:38.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:38 smithi195 ceph-mon[37040]: Deploying daemon osd.6 on smithi087 2024-06-06T07:40:38.495 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:38 smithi087 ceph-mon[32276]: Deploying daemon osd.6 on smithi087 2024-06-06T07:40:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:38 smithi181 ceph-mon[36790]: Deploying daemon osd.6 on smithi087 2024-06-06T07:40:39.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:39 smithi087 ceph-mon[32276]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:39 smithi195 ceph-mon[37040]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:39 smithi181 ceph-mon[36790]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:41 smithi195 ceph-mon[37040]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:41 smithi181 ceph-mon[36790]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:41.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:41 smithi087 ceph-mon[32276]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:43.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:43 smithi087 ceph-mon[32276]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:43.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:43.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:43.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:43 smithi195 ceph-mon[37040]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:43 smithi181 ceph-mon[36790]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:43.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:45 smithi087 ceph-mon[32276]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:45 smithi195 ceph-mon[37040]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:45 smithi181 ceph-mon[36790]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:46.155 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 6 on host 'smithi087' 2024-06-06T07:40:46.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.155+0000 7f17d77fe700 1 -- 172.21.15.87:0/3894272919 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f17e00fe290 con 0x7f17cc0a2ca0 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 msgr2=0x7f17cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 0x7f17cc0a5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009de0 tx=0x7f17d0009190 comp rx=0 tx=0).stop 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 msgr2=0x7f17e0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005910 tx=0x7f17c800bbb0 comp rx=0 tx=0).stop 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 shutdown_connections 2024-06-06T07:40:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f17cc0a2ca0 0x7f17cc0a5160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e0101670 0x7f17e0105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17e0102aa0 0x7f17e0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 --2- 172.21.15.87:0/3894272919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17e0102030 0x7f17e0103a60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 >> 172.21.15.87:0/3894272919 conn(0x7f17e00fc140 msgr2=0x7f17e00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 shutdown_connections 2024-06-06T07:40:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:40:46.157+0000 7f17e4eb8700 1 -- 172.21.15.87:0/3894272919 wait complete. 2024-06-06T07:40:46.851 DEBUG:teuthology.orchestra.run.smithi087:osd.6> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.6.service 2024-06-06T07:40:46.853 INFO:tasks.cephadm:Deploying osd.7 on smithi181 with /dev/vg_nvme/lv_2... 2024-06-06T07:40:46.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T07:40:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:40:47.223 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:47.223 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.223 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.223 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T07:40:47.224 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.224 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T07:40:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-mon[32276]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T07:40:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-mon[32276]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T07:40:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-mon[32276]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:40:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:48 smithi195 ceph-mon[37040]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T07:40:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:48 smithi195 ceph-mon[37040]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T07:40:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:48 smithi195 ceph-mon[37040]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:40:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:48.521 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:48 smithi181 ceph-mon[36790]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T07:40:48.521 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:48 smithi181 ceph-mon[36790]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T07:40:48.521 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:48 smithi181 ceph-mon[36790]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:40:48.521 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.096 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T07:40:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T07:40:49.097 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-06-06T07:40:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-06-06T07:40:49.097 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123495 s, 849 MB/s 2024-06-06T07:40:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-06-06T07:40:49.150 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T07:40:48.886+0000 7f1b5f3c4700 -1 osd.6 0 waiting for initial osdmap 2024-06-06T07:40:49.151 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 07:40:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T07:40:48.894+0000 7f1b569ae700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:40:49.416 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:49.416 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:40:49.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T07:40:49.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' 2024-06-06T07:40:49.417 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:49 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' 2024-06-06T07:40:49.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:49 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: from='osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418]' entity='osd.6' 2024-06-06T07:40:49.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:49 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:49.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-06-06T07:40:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:40:50.398 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:50 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:40:50.398 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:50 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:40:50.398 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:50 smithi181 ceph-mon[36790]: osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] boot 2024-06-06T07:40:50.398 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:50 smithi181 ceph-mon[36790]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T07:40:50.398 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:50 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:40:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:50 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:40:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:50 smithi195 ceph-mon[37040]: osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] boot 2024-06-06T07:40:50.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:50 smithi195 ceph-mon[37040]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T07:40:50.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:50.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:50 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:40:50.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:50 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:40:50.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:50 smithi087 ceph-mon[32276]: osd.6 [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] boot 2024-06-06T07:40:50.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:50 smithi087 ceph-mon[32276]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T07:40:50.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:40:51.484 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.485 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:51.486 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.486 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.486 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:40:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:40:51.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 -- 172.21.15.181:0/3633716780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b501024f0 msgr2=0x7f6b50102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/3633716780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b501024f0 0x7f6b50102910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b440099c0 tx=0x7f6b44009cd0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 -- 172.21.15.181:0/3633716780 shutdown_connections 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/3633716780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b501024f0 0x7f6b50102910 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/3633716780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b50104090 0x7f6b5010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/3633716780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501036f0 0x7f6b50103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 -- 172.21.15.181:0/3633716780 >> 172.21.15.181:0/3633716780 conn(0x7f6b500fda90 msgr2=0x7f6b500ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 -- 172.21.15.181:0/3633716780 shutdown_connections 2024-06-06T07:40:51.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.717+0000 7f6b56d3a700 1 -- 172.21.15.181:0/3633716780 wait complete. 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 Processor -- start 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 -- start start 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501024f0 0x7f6b50076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b501036f0 0x7f6b50074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5010e0d0 con 0x7f6b501024f0 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5010df50 con 0x7f6b50104090 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b56d3a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5010e250 con 0x7f6b501036f0 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46142/0 (socket says 172.21.15.181:46142) 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 -- 172.21.15.181:0/60476184 learned_addr learned my addr 172.21.15.181:0/60476184 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b54ad6700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501024f0 0x7f6b50076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b4ffff700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b501036f0 0x7f6b50074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:51.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 -- 172.21.15.181:0/60476184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b501036f0 msgr2=0x7f6b50074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b501036f0 0x7f6b50074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 -- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501024f0 msgr2=0x7f6b50076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501024f0 0x7f6b50076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.718+0000 7f6b552d7700 1 -- 172.21.15.181:0/60476184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b50075670 con 0x7f6b50104090 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b552d7700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800ada0 tx=0x7f6b4800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48017020 con 0x7f6b50104090 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4800a0d0 con 0x7f6b50104090 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b501a28f0 con 0x7f6b50104090 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48005480 con 0x7f6b50104090 2024-06-06T07:40:51.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.719+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b501a2db0 con 0x7f6b50104090 2024-06-06T07:40:51.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.720+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b50069180 con 0x7f6b50104090 2024-06-06T07:40:51.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.721+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b48005620 con 0x7f6b50104090 2024-06-06T07:40:51.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.721+0000 7f6b4dffb700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 0x7f6b400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:40:51.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.721+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f6b480cdac0 con 0x7f6b50104090 2024-06-06T07:40:51.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.722+0000 7f6b54ad6700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 0x7f6b400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:40:51.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.722+0000 7f6b54ad6700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 0x7f6b400a50e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b580 tx=0x7f6b44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:40:51.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.726+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b48021020 con 0x7f6b50104090 2024-06-06T07:40:51.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:40:51.935+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6b50062990 con 0x7f6b400a2c20 2024-06-06T07:40:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi087 2024-06-06T07:40:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 5696M 2024-06-06T07:40:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T07:40:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:52.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:52.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:52 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi087 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 5696M 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:52.546 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:52 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi087 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 5696M 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:40:52.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:52 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:40:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:53 smithi195 ceph-mon[37040]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:53 smithi195 ceph-mon[37040]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:53 smithi181 ceph-mon[36790]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:53 smithi181 ceph-mon[36790]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:53 smithi087 ceph-mon[32276]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:40:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:53 smithi087 ceph-mon[32276]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/1651641956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]': finished 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T07:40:55.489 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:40:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/1651641956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]': finished 2024-06-06T07:40:55.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T07:40:55.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/1651641956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]: dispatch 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2403cabf-c28a-4c74-b355-6c695506c49f"}]': finished 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T07:40:55.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:40:56.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:56 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/3724463837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:56 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/3724463837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:56.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:56 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/3724463837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:40:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:57 smithi195 ceph-mon[37040]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:57 smithi181 ceph-mon[36790]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:57.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:57 smithi087 ceph-mon[32276]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:40:59 smithi181 ceph-mon[36790]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:40:59 smithi195 ceph-mon[37040]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:40:59.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:40:59 smithi087 ceph-mon[32276]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:00.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:00 smithi181 ceph-mon[36790]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:00 smithi087 ceph-mon[32276]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:00 smithi195 ceph-mon[37040]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:03 smithi195 ceph-mon[37040]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T07:41:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:03.516 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:03 smithi181 ceph-mon[36790]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:03.516 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T07:41:03.517 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:03 smithi087 ceph-mon[32276]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T07:41:03.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:04 smithi195 ceph-mon[37040]: Deploying daemon osd.7 on smithi181 2024-06-06T07:41:04.511 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:04 smithi181 ceph-mon[36790]: Deploying daemon osd.7 on smithi181 2024-06-06T07:41:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:04 smithi087 ceph-mon[32276]: Deploying daemon osd.7 on smithi181 2024-06-06T07:41:05.281 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:05 smithi181 ceph-mon[36790]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:05 smithi195 ceph-mon[37040]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:05 smithi087 ceph-mon[32276]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:07.191 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:07 smithi181 ceph-mon[36790]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:07 smithi195 ceph-mon[37040]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:07 smithi087 ceph-mon[32276]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:09.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:09.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:08 smithi181 ceph-mon[36790]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:09.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:09.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:08 smithi087 ceph-mon[32276]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:09.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:09.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:09.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:08 smithi195 ceph-mon[37040]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:11.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.748+0000 7f6b4dffb700 1 -- 172.21.15.181:0/60476184 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b50062990 con 0x7f6b400a2c20 2024-06-06T07:41:11.749 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 7 on host 'smithi181' 2024-06-06T07:41:11.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 msgr2=0x7f6b400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:11.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 0x7f6b400a50e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b580 tx=0x7f6b44005fb0 comp rx=0 tx=0).stop 2024-06-06T07:41:11.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 msgr2=0x7f6b50074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800ada0 tx=0x7f6b4800c330 comp rx=0 tx=0).stop 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 shutdown_connections 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6b400a2c20 0x7f6b400a50e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b50104090 0x7f6b50074dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b501024f0 0x7f6b50076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 --2- 172.21.15.181:0/60476184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b501036f0 0x7f6b50074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 >> 172.21.15.181:0/60476184 conn(0x7f6b500fda90 msgr2=0x7f6b50106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 shutdown_connections 2024-06-06T07:41:11.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:41:11.750+0000 7f6b56d3a700 1 -- 172.21.15.181:0/60476184 wait complete. 2024-06-06T07:41:12.408 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:12 smithi181 ceph-mon[36790]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.408 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:12 smithi181 ceph-mon[36790]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.408 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.408 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.408 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:12.440 DEBUG:teuthology.orchestra.run.smithi181:osd.7> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.7.service 2024-06-06T07:41:12.443 INFO:tasks.cephadm:Deploying osd.8 on smithi195 with /dev/vg_nvme/lv_2... 2024-06-06T07:41:12.443 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T07:41:12.469 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:12 smithi195 ceph-mon[37040]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.469 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:12 smithi195 ceph-mon[37040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.469 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:12 smithi087 ceph-mon[32276]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:12 smithi087 ceph-mon[32276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T07:41:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:12.685 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:41:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T07:41:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T07:41:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:13 smithi195 ceph-mon[37040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:13 smithi087 ceph-mon[32276]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: from='osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:13.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-mon[36790]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:41:14.176 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T07:41:13.881+0000 7f08a1e61700 -1 osd.7 0 waiting for initial osdmap 2024-06-06T07:41:14.176 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 07:41:13 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T07:41:13.891+0000 7f0899c4c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:41:14.241 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:14 smithi195 ceph-mon[37040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:41:14.242 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:14 smithi195 ceph-mon[37040]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T07:41:14.242 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:14 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.242 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:14 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.242 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:14 smithi195 ceph-mon[37040]: from='osd.7 ' entity='osd.7' 2024-06-06T07:41:14.528 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:14 smithi087 ceph-mon[32276]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:41:14.529 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:14 smithi087 ceph-mon[32276]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T07:41:14.529 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.529 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:14 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.529 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:14 smithi087 ceph-mon[32276]: from='osd.7 ' entity='osd.7' 2024-06-06T07:41:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:14 smithi181 ceph-mon[36790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:41:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:14 smithi181 ceph-mon[36790]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T07:41:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:14 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:14 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:14.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:14 smithi181 ceph-mon[36790]: from='osd.7 ' entity='osd.7' 2024-06-06T07:41:14.774 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T07:41:14.775 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T07:41:14.775 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-06-06T07:41:14.775 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-06-06T07:41:14.775 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847871 s, 1.2 GB/s 2024-06-06T07:41:14.775 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-06-06T07:41:15.395 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:41:15.395 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:41:15.396 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:15.396 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] boot 2024-06-06T07:41:15.396 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T07:41:15.396 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:15.396 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:15 smithi195 ceph-mon[37040]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] boot 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:15.399 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:15 smithi181 ceph-mon[36790]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T07:41:15.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_2 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: osd.7 [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] boot 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:41:15.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:15 smithi087 ceph-mon[32276]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T07:41:15.712 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi181 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 5696M 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:16.658 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.658 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:16 smithi195 ceph-mon[37040]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi181 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 5696M 2024-06-06T07:41:16.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:16.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:16.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.808 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:16 smithi181 ceph-mon[36790]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:16.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T07:41:16.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi181 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 5696M 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:16.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:16.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:16.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:16 smithi087 ceph-mon[32276]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:17.285 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 -- 172.21.15.195:0/472325212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 msgr2=0x7ff09410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 --2- 172.21.15.195:0/472325212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff09410aac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff084009990 tx=0x7ff084009ca0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 -- 172.21.15.195:0/472325212 shutdown_connections 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 --2- 172.21.15.195:0/472325212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094103ad0 0x7ff094103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 --2- 172.21.15.195:0/472325212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff09410aac0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 --2- 172.21.15.195:0/472325212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094100550 0x7ff0941009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 -- 172.21.15.195:0/472325212 >> 172.21.15.195:0/472325212 conn(0x7ff0940fc0f0 msgr2=0x7ff0940fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 -- 172.21.15.195:0/472325212 shutdown_connections 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.283+0000 7ff09af7b700 1 -- 172.21.15.195:0/472325212 wait complete. 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 Processor -- start 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 -- start start 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094100550 0x7ff0941031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:17.286 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094103ad0 0x7ff094101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e180 con 0x7ff094103ad0 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e000 con 0x7ff094100550 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff09af7b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e300 con 0x7ff094100ef0 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff093fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff093fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:48508/0 (socket says 172.21.15.195:48508) 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff093fff700 1 -- 172.21.15.195:0/1036991359 learned_addr learned my addr 172.21.15.195:0/1036991359 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff099518700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094103ad0 0x7ff094101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.284+0000 7ff098d17700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094100550 0x7ff0941031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 -- 172.21.15.195:0/1036991359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094100550 msgr2=0x7ff0941031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094100550 0x7ff0941031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.287 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 -- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094103ad0 msgr2=0x7ff094101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094103ad0 0x7ff094101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 -- 172.21.15.195:0/1036991359 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094102610 con 0x7ff094100ef0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff093fff700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b530 tx=0x7ff08800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088015020 con 0x7ff094100ef0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0941a2960 con 0x7ff094100ef0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088011d10 con 0x7ff094100ef0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08800f2e0 con 0x7ff094100ef0 2024-06-06T07:41:17.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.285+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941a2ed0 con 0x7ff094100ef0 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.286+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff094069180 con 0x7ff094100ef0 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.287+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff088018020 con 0x7ff094100ef0 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.288+0000 7ff091ffb700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 0x7ff07c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.288+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7ff0880ce110 con 0x7ff094100ef0 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.288+0000 7ff098d17700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 0x7ff07c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:17.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.288+0000 7ff098d17700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 0x7ff07c0a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff094069e50 tx=0x7ff08c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:41:17.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.291+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff08801f020 con 0x7ff094100ef0 2024-06-06T07:41:17.496 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:17.496+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff0940628d0 con 0x7ff07c0a2c20 2024-06-06T07:41:17.496 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:17 smithi195 ceph-mon[37040]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T07:41:17.496 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:17 smithi195 ceph-mon[37040]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T07:41:17.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:17 smithi181 ceph-mon[36790]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T07:41:17.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:17 smithi181 ceph-mon[36790]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T07:41:17.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:17 smithi087 ceph-mon[32276]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T07:41:17.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:17 smithi087 ceph-mon[32276]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T07:41:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:18 smithi195 ceph-mon[37040]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:18 smithi195 ceph-mon[37040]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:18 smithi181 ceph-mon[36790]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:18 smithi181 ceph-mon[36790]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:18.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:18 smithi087 ceph-mon[32276]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:18.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:18.864 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:18.864 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:18.864 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:18 smithi087 ceph-mon[32276]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:20.432 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:20 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/4269372641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.433 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:20 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.433 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:20 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]': finished 2024-06-06T07:41:20.433 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:20 smithi195 ceph-mon[37040]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T07:41:20.433 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:20 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:20 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/4269372641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:20 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:20 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]': finished 2024-06-06T07:41:20.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:20 smithi181 ceph-mon[36790]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T07:41:20.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:20 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:20 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/4269372641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:20 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]: dispatch 2024-06-06T07:41:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:20 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720aa4b-ec90-4411-b392-93bdbe855dac"}]': finished 2024-06-06T07:41:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:20 smithi087 ceph-mon[32276]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T07:41:20.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:20 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:21 smithi195 ceph-mon[37040]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:21 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/3570764031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:21 smithi181 ceph-mon[36790]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:21 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/3570764031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:21.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:21 smithi087 ceph-mon[32276]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:21.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:21 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/3570764031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:23 smithi195 ceph-mon[37040]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:23 smithi181 ceph-mon[36790]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:23 smithi087 ceph-mon[32276]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:25.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:25 smithi195 ceph-mon[37040]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:25 smithi181 ceph-mon[36790]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:25.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:25 smithi087 ceph-mon[32276]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:27 smithi195 ceph-mon[37040]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:27 smithi181 ceph-mon[36790]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:27.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:27 smithi087 ceph-mon[32276]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:28.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T07:41:28.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:28.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T07:41:28.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:28.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T07:41:28.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:29.332 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:29 smithi195 ceph-mon[37040]: Deploying daemon osd.8 on smithi195 2024-06-06T07:41:29.332 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:29 smithi195 ceph-mon[37040]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:29 smithi181 ceph-mon[36790]: Deploying daemon osd.8 on smithi195 2024-06-06T07:41:29.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:29 smithi181 ceph-mon[36790]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:29.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:29 smithi087 ceph-mon[32276]: Deploying daemon osd.8 on smithi195 2024-06-06T07:41:29.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:29 smithi087 ceph-mon[32276]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:31 smithi195 ceph-mon[37040]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:31 smithi181 ceph-mon[36790]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:31 smithi087 ceph-mon[32276]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:33.504 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:33 smithi195 ceph-mon[37040]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:33.505 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.505 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.505 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:33 smithi181 ceph-mon[36790]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:33.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:33 smithi087 ceph-mon[32276]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:33.928 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.928 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:33.929 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:35.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:35 smithi195 ceph-mon[37040]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:35 smithi181 ceph-mon[36790]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:35.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:35 smithi087 ceph-mon[32276]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:36.564 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.564+0000 7ff091ffb700 1 -- 172.21.15.195:0/1036991359 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0940628d0 con 0x7ff07c0a2c20 2024-06-06T07:41:36.565 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 8 on host 'smithi195' 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 msgr2=0x7ff07c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 0x7ff07c0a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff094069e50 tx=0x7ff08c00d040 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 msgr2=0x7ff094101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b530 tx=0x7ff08800b840 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 shutdown_connections 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff07c0a2c20 0x7ff07c0a50e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff094100550 0x7ff0941031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff094103ad0 0x7ff094101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 --2- 172.21.15.195:0/1036991359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff094100ef0 0x7ff094101820 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.566+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 >> 172.21.15.195:0/1036991359 conn(0x7ff0940fc0f0 msgr2=0x7ff094106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.567+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 shutdown_connections 2024-06-06T07:41:36.567 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:41:36.567+0000 7ff09af7b700 1 -- 172.21.15.195:0/1036991359 wait complete. 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:37.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:36 smithi181 ceph-mon[36790]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:37.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:36 smithi087 ceph-mon[32276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.180 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.180 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.180 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:37.180 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:37.180 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.181 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:36 smithi195 ceph-mon[37040]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T07:41:37.206 DEBUG:teuthology.orchestra.run.smithi195:osd.8> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.8.service 2024-06-06T07:41:37.209 INFO:tasks.cephadm:Deploying osd.9 on smithi087 with /dev/vg_nvme/lv_1... 2024-06-06T07:41:37.209 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T07:41:37.457 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:41:38.674 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T07:41:38.674 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T07:41:38.675 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:38.675 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.675 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.675 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-mon[37040]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:38.675 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T07:41:38.539+0000 7f28c8dd5700 -1 osd.8 0 waiting for initial osdmap 2024-06-06T07:41:38.675 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 07:41:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T07:41:38.548+0000 7f28c4bc8700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.805 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:38 smithi087 ceph-mon[32276]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:38.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T07:41:38.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T07:41:38.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:38.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: from='osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:41:38.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:38 smithi181 ceph-mon[36790]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122814 s, 854 MB/s 2024-06-06T07:41:39.498 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-06-06T07:41:39.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:41:39.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T07:41:39.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] boot 2024-06-06T07:41:39.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T07:41:39.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:39 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] boot 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T07:41:39.800 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:39 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:41:39.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T07:41:39.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:39.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: osd.8 [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] boot 2024-06-06T07:41:39.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T07:41:39.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:39 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:41:40.243 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi087:vg_nvme/lv_1 2024-06-06T07:41:40.507 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:41:40.802 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:41:40.802 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:41:40.802 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:40 smithi087 ceph-mon[32276]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T07:41:40.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:40.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:40 smithi181 ceph-mon[36790]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T07:41:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:41:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:41:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:40 smithi195 ceph-mon[37040]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T07:41:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:41 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi195 2024-06-06T07:41:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:41 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 5696M 2024-06-06T07:41:41.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:41 smithi087 ceph-mon[32276]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T07:41:41.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:41 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi195 2024-06-06T07:41:41.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:41 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 5696M 2024-06-06T07:41:41.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:41 smithi181 ceph-mon[36790]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T07:41:42.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:41 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi195 2024-06-06T07:41:42.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:41 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 5696M 2024-06-06T07:41:42.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:41 smithi195 ceph-mon[37040]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T07:41:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 -- 172.21.15.87:0/1789245510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 msgr2=0x7f592410a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 --2- 172.21.15.87:0/1789245510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592410a650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b0a0 tx=0x7f591c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 -- 172.21.15.87:0/1789245510 shutdown_connections 2024-06-06T07:41:42.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 --2- 172.21.15.87:0/1789245510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f59241053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 --2- 172.21.15.87:0/1789245510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592410a650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 --2- 172.21.15.87:0/1789245510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 0x7f5924107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 -- 172.21.15.87:0/1789245510 >> 172.21.15.87:0/1789245510 conn(0x7f59240fc990 msgr2=0x7f59240fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 -- 172.21.15.87:0/1789245510 shutdown_connections 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.198+0000 7f5929c1e700 1 -- 172.21.15.87:0/1789245510 wait complete. 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 Processor -- start 2024-06-06T07:41:42.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 -- start start 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f5924198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 0x7f5924198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f592410e100 con 0x7f5924108240 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f592410df80 con 0x7f5924102fe0 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5929c1e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f592410e280 con 0x7f5924105910 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5923fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5923fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58678/0 (socket says 172.21.15.87:58678) 2024-06-06T07:41:42.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.199+0000 7f5923fff700 1 -- 172.21.15.87:0/4107814192 learned_addr learned my addr 172.21.15.87:0/4107814192 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5922ffd700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 0x7f5924198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f59237fe700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f5924198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 -- 172.21.15.87:0/4107814192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 msgr2=0x7f5924198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 0x7f5924198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 -- 172.21.15.87:0/4107814192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 msgr2=0x7f5924198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f5924198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 -- 172.21.15.87:0/4107814192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241a0260 con 0x7f5924108240 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f59237fe700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f5924198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5923fff700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d010 tx=0x7f591c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c021070 con 0x7f5924108240 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5929c1e700 1 -- 172.21.15.87:0/4107814192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59241a04f0 con 0x7f5924108240 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c007ca0 con 0x7f5924108240 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0055d0 con 0x7f5924108240 2024-06-06T07:41:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.200+0000 7f5929c1e700 1 -- 172.21.15.87:0/4107814192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241a09a0 con 0x7f5924108240 2024-06-06T07:41:42.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.201+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904005330 con 0x7f5924108240 2024-06-06T07:41:42.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.202+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f591c024070 con 0x7f5924108240 2024-06-06T07:41:42.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.203+0000 7f5920ff9700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 0x7f590c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:41:42.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.203+0000 7f59237fe700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 0x7f590c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:41:42.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.203+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f591c00f020 con 0x7f5924108240 2024-06-06T07:41:42.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.203+0000 7f59237fe700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 0x7f590c0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5918005fd0 tx=0x7f5918005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:41:42.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.207+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f591c02b020 con 0x7f5924108240 2024-06-06T07:41:42.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:41:42.414+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5904000bc0 con 0x7f590c0a2c20 2024-06-06T07:41:42.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:42 smithi181 ceph-mon[36790]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:42 smithi181 ceph-mon[36790]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T07:41:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:42.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:42 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:42.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:42 smithi087 ceph-mon[32276]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:42.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:42 smithi087 ceph-mon[32276]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T07:41:42.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:42.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:42.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:42 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:42 smithi195 ceph-mon[37040]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:42 smithi195 ceph-mon[37040]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T07:41:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:41:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:41:43.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:42 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:43.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:43 smithi181 ceph-mon[36790]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:43.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:43 smithi181 ceph-mon[36790]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T07:41:43.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:43 smithi087 ceph-mon[32276]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:43.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:43 smithi087 ceph-mon[32276]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T07:41:44.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:43 smithi195 ceph-mon[37040]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:41:44.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:43 smithi195 ceph-mon[37040]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T07:41:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:44 smithi087 ceph-mon[32276]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:45.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:44 smithi195 ceph-mon[37040]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:45.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:44 smithi181 ceph-mon[36790]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:45.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]: dispatch 2024-06-06T07:41:45.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]': finished 2024-06-06T07:41:45.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:45 smithi087 ceph-mon[32276]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T07:41:45.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:45 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:41:46.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]: dispatch 2024-06-06T07:41:46.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]': finished 2024-06-06T07:41:46.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:45 smithi195 ceph-mon[37040]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T07:41:46.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:45 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:41:46.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]: dispatch 2024-06-06T07:41:46.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3549450655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b1b810-b726-42ff-9bea-814b83c404f1"}]': finished 2024-06-06T07:41:46.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:45 smithi181 ceph-mon[36790]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T07:41:46.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:45 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:41:46.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:46 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1720546595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:46.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:46 smithi087 ceph-mon[32276]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:41:47.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:46 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1720546595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:47.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:46 smithi195 ceph-mon[37040]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:41:47.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:46 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1720546595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:41:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:46 smithi181 ceph-mon[36790]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-06T07:41:49.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:49 smithi195 ceph-mon[37040]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:41:49.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:49 smithi181 ceph-mon[36790]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:41:49.582 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:49 smithi087 ceph-mon[32276]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T07:41:51.478 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:51 smithi087 ceph-mon[32276]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:41:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:51 smithi195 ceph-mon[37040]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:41:51.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:51 smithi181 ceph-mon[36790]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-06T07:41:53.490 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:53 smithi087 ceph-mon[32276]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-06T07:41:53.490 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T07:41:53.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:53 smithi195 ceph-mon[37040]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-06T07:41:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T07:41:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:53 smithi181 ceph-mon[36790]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-06T07:41:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T07:41:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:41:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:54 smithi195 ceph-mon[37040]: Deploying daemon osd.9 on smithi087 2024-06-06T07:41:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:54 smithi181 ceph-mon[36790]: Deploying daemon osd.9 on smithi087 2024-06-06T07:41:54.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:54 smithi087 ceph-mon[32276]: Deploying daemon osd.9 on smithi087 2024-06-06T07:41:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:55 smithi181 ceph-mon[36790]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T07:41:55.466 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:55 smithi087 ceph-mon[32276]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T07:41:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:55 smithi195 ceph-mon[37040]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T07:41:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:57 smithi195 ceph-mon[37040]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:57.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:57 smithi087 ceph-mon[32276]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:57 smithi181 ceph-mon[36790]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:59.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:59 smithi087 ceph-mon[32276]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:59.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:59.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:59.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:41:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:59 smithi195 ceph-mon[37040]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:59.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:41:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:59 smithi181 ceph-mon[36790]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:41:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:41:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:41:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:41:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:01.209 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:01 smithi087 ceph-mon[32276]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:01 smithi195 ceph-mon[37040]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:01 smithi181 ceph-mon[36790]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:02.170 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 9 on host 'smithi087' 2024-06-06T07:42:02.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.169+0000 7f5920ff9700 1 -- 172.21.15.87:0/4107814192 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5904000bc0 con 0x7f590c0a2c20 2024-06-06T07:42:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 msgr2=0x7f590c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 0x7f590c0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5918005fd0 tx=0x7f5918005dc0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 msgr2=0x7f592419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d010 tx=0x7f591c004ef0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 shutdown_connections 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f590c0a2c20 0x7f590c0a50e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5924102fe0 0x7f5924198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5924108240 0x7f592419fbc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 --2- 172.21.15.87:0/4107814192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5924105910 0x7f5924198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 >> 172.21.15.87:0/4107814192 conn(0x7f59240fc990 msgr2=0x7f5924103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 shutdown_connections 2024-06-06T07:42:02.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:02.172+0000 7f590a7fc700 1 -- 172.21.15.87:0/4107814192 wait complete. 2024-06-06T07:42:02.924 DEBUG:teuthology.orchestra.run.smithi087:osd.9> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.9.service 2024-06-06T07:42:02.926 INFO:tasks.cephadm:Deploying osd.10 on smithi181 with /dev/vg_nvme/lv_1... 2024-06-06T07:42:02.926 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T07:42:03.174 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.296 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:03 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T07:42:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:03 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T07:42:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:03.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:03 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:04.206 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-mon[32276]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T07:42:04.207 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-mon[32276]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T07:42:04.207 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-mon[32276]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:42:04.207 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:04.207 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T07:42:04.194+0000 7f37cdfc2700 -1 osd.9 0 waiting for initial osdmap 2024-06-06T07:42:04.207 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 07:42:04 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T07:42:04.205+0000 7f37c9db5700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:42:04.445 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:04 smithi181 ceph-mon[36790]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T07:42:04.446 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:04 smithi181 ceph-mon[36790]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T07:42:04.446 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:04 smithi181 ceph-mon[36790]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:42:04.446 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:04 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:04 smithi195 ceph-mon[37040]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T07:42:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:04 smithi195 ceph-mon[37040]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T07:42:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:04 smithi195 ceph-mon[37040]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-06-06T07:42:04.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:04 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.104 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T07:42:05.105 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T07:42:05.105 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-06-06T07:42:05.105 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-06-06T07:42:05.105 INFO:teuthology.orchestra.run.smithi181.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0144972 s, 723 MB/s 2024-06-06T07:42:05.105 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-06-06T07:42:05.406 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:05 smithi181 ceph-mon[36790]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:05.406 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:05 smithi181 ceph-mon[36790]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:42:05.406 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:05 smithi181 ceph-mon[36790]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T07:42:05.406 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:05 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.406 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:05 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:05 smithi195 ceph-mon[37040]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:05 smithi195 ceph-mon[37040]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:42:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:05 smithi195 ceph-mon[37040]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T07:42:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:05 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:05 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:05 smithi087 ceph-mon[32276]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:05 smithi087 ceph-mon[32276]: from='osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-06-06T07:42:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:05 smithi087 ceph-mon[32276]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T07:42:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:05 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:05 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:05.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-06-06T07:42:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:42:06.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:06 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:42:06.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:06 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:42:06.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:06 smithi087 ceph-mon[32276]: osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] boot 2024-06-06T07:42:06.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:06 smithi087 ceph-mon[32276]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T07:42:06.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:06 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:42:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:06 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:42:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:06 smithi195 ceph-mon[37040]: osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] boot 2024-06-06T07:42:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:06 smithi195 ceph-mon[37040]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T07:42:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:06.513 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:06 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:42:06.514 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:06 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:42:06.514 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:06 smithi181 ceph-mon[36790]: osd.9 [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] boot 2024-06-06T07:42:06.514 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:06 smithi181 ceph-mon[36790]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T07:42:06.514 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:07 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T07:42:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:07 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:07.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:07 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 -- 172.21.15.181:0/3508446619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 msgr2=0x7f5f3c109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 --2- 172.21.15.181:0/3508446619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c109840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5f340099c0 tx=0x7f5f34009cd0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 -- 172.21.15.181:0/3508446619 shutdown_connections 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 --2- 172.21.15.181:0/3508446619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c109840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 --2- 172.21.15.181:0/3508446619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 0x7f5f3c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 --2- 172.21.15.181:0/3508446619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 0x7f5f3c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 -- 172.21.15.181:0/3508446619 >> 172.21.15.181:0/3508446619 conn(0x7f5f3c0fc990 msgr2=0x7f5f3c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 -- 172.21.15.181:0/3508446619 shutdown_connections 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.657+0000 7f5f435c3700 1 -- 172.21.15.181:0/3508446619 wait complete. 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 Processor -- start 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 -- start start 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 0x7f5f3c105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 0x7f5f3c1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10d060 con 0x7f5f3c1025f0 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10cee0 con 0x7f5f3c102f90 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f435c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10d1e0 con 0x7f5f3c1013f0 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34756/0 (socket says 172.21.15.181:34756) 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 -- 172.21.15.181:0/1999626612 learned_addr learned my addr 172.21.15.181:0/1999626612 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f4135f700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 0x7f5f3c105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f40b5e700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 0x7f5f3c1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 -- 172.21.15.181:0/1999626612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 msgr2=0x7f5f3c105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 0x7f5f3c105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 -- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 msgr2=0x7f5f3c1039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 0x7f5f3c1039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 -- 172.21.15.181:0/1999626612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c1047c0 con 0x7f5f3c102f90 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.658+0000 7f5f41b60700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005910 tx=0x7f5f34009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.659+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34027070 con 0x7f5f3c102f90 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.659+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f34003f40 con 0x7f5f3c102f90 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.659+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c1a2890 con 0x7f5f3c102f90 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.659+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34022df0 con 0x7f5f3c102f90 2024-06-06T07:42:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.659+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1a2d60 con 0x7f5f3c102f90 2024-06-06T07:42:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.660+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c069180 con 0x7f5f3c102f90 2024-06-06T07:42:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.661+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5f3402a070 con 0x7f5f3c102f90 2024-06-06T07:42:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.661+0000 7f5f327fc700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 0x7f5f280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.661+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f5f340cf9d0 con 0x7f5f3c102f90 2024-06-06T07:42:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.661+0000 7f5f4135f700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 0x7f5f280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.662+0000 7f5f4135f700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 0x7f5f280a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c102450 tx=0x7f5f3800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:07.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.665+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5f34031020 con 0x7f5f3c102f90 2024-06-06T07:42:07.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:07.870+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5f3c0feb20 con 0x7f5f280a2c20 2024-06-06T07:42:08.483 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:08 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi087 2024-06-06T07:42:08.484 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:08 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 4272M 2024-06-06T07:42:08.484 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:08.484 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:08.484 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:08 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:08 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi087 2024-06-06T07:42:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:08 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 4272M 2024-06-06T07:42:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:08.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:08 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:08 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi087 2024-06-06T07:42:08.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:08 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 4272M 2024-06-06T07:42:08.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:08.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:08.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:08 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:09 smithi195 ceph-mon[37040]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:09 smithi195 ceph-mon[37040]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:09 smithi181 ceph-mon[36790]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:09.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:09 smithi181 ceph-mon[36790]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:09.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:09 smithi087 ceph-mon[32276]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:09.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:09 smithi087 ceph-mon[32276]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/694384173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]': finished 2024-06-06T07:42:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T07:42:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:11 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/694384173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]': finished 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T07:42:11.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:11 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/694384173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]: dispatch 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c"}]': finished 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T07:42:11.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:11 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:12 smithi195 ceph-mon[37040]: from='client.? 172.21.15.181:0/2533371854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:12 smithi181 ceph-mon[36790]: from='client.? 172.21.15.181:0/2533371854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.181:0/2533371854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:13 smithi195 ceph-mon[37040]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:13.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:13 smithi087 ceph-mon[32276]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:13 smithi181 ceph-mon[36790]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:14.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:14 smithi195 ceph-mon[37040]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:14.527 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:14 smithi087 ceph-mon[32276]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:14 smithi181 ceph-mon[36790]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:17.426 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:17 smithi181 ceph-mon[36790]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:17.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:17 smithi195 ceph-mon[37040]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:17 smithi087 ceph-mon[32276]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:19.290 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:19 smithi181 ceph-mon[36790]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:19.291 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:19 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T07:42:19.291 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:19 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:19 smithi195 ceph-mon[37040]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:19 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T07:42:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:19 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:19.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:19 smithi087 ceph-mon[32276]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:19.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:19 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T07:42:19.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:19 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:20.286 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:20 smithi181 ceph-mon[36790]: Deploying daemon osd.10 on smithi181 2024-06-06T07:42:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:20 smithi195 ceph-mon[37040]: Deploying daemon osd.10 on smithi181 2024-06-06T07:42:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:20 smithi087 ceph-mon[32276]: Deploying daemon osd.10 on smithi181 2024-06-06T07:42:21.450 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:21 smithi181 ceph-mon[36790]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:21 smithi195 ceph-mon[37040]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:21.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:21 smithi087 ceph-mon[32276]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:23 smithi195 ceph-mon[37040]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:23.553 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:23 smithi181 ceph-mon[36790]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:23 smithi087 ceph-mon[32276]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:25.344 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:25 smithi181 ceph-mon[36790]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:25.344 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.344 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.344 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:25 smithi195 ceph-mon[37040]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:25.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:25 smithi087 ceph-mon[32276]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:25.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:25.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:27.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:27 smithi181 ceph-mon[36790]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:27 smithi195 ceph-mon[37040]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:27.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:27 smithi087 ceph-mon[32276]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:27.887 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 10 on host 'smithi181' 2024-06-06T07:42:27.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.887+0000 7f5f327fc700 1 -- 172.21.15.181:0/1999626612 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5f3c0feb20 con 0x7f5f280a2c20 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 msgr2=0x7f5f280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 0x7f5f280a50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c102450 tx=0x7f5f3800d040 comp rx=0 tx=0).stop 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 msgr2=0x7f5f3c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005910 tx=0x7f5f34009580 comp rx=0 tx=0).stop 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 shutdown_connections 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5f280a2c20 0x7f5f280a50e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3c102f90 0x7f5f3c103f10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:27.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f3c1013f0 0x7f5f3c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:27.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 --2- 172.21.15.181:0/1999626612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f3c1025f0 0x7f5f3c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:27.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 >> 172.21.15.181:0/1999626612 conn(0x7f5f3c0fc990 msgr2=0x7f5f3c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:27.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 shutdown_connections 2024-06-06T07:42:27.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-06-06T07:42:27.890+0000 7f5f435c3700 1 -- 172.21.15.181:0/1999626612 wait complete. 2024-06-06T07:42:28.531 DEBUG:teuthology.orchestra.run.smithi181:osd.10> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.10.service 2024-06-06T07:42:28.534 INFO:tasks.cephadm:Deploying osd.11 on smithi195 with /dev/vg_nvme/lv_1... 2024-06-06T07:42:28.535 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T07:42:28.768 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:29.137 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.138 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.303 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:29.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:30.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:29 smithi195 ceph-mon[37040]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T07:42:30.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:29 smithi195 ceph-mon[37040]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T07:42:30.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:29 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:30.141 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:29 smithi195 ceph-mon[37040]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.141 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:29 smithi195 ceph-mon[37040]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.297 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-mon[36790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T07:42:30.297 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-mon[36790]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T07:42:30.297 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:30.297 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-mon[36790]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.297 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-mon[36790]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.297 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T07:42:29.892+0000 7f3ed4165700 -1 osd.10 0 waiting for initial osdmap 2024-06-06T07:42:30.298 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 07:42:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T07:42:29.902+0000 7f3ecff58700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:42:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:29 smithi087 ceph-mon[32276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T07:42:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:29 smithi087 ceph-mon[32276]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T07:42:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:29 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:29 smithi087 ceph-mon[32276]: from='osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:29 smithi087 ceph-mon[32276]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00864734 s, 1.2 GB/s 2024-06-06T07:42:30.821 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-06-06T07:42:30.899 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:30 smithi181 ceph-mon[36790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:42:30.899 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:30 smithi181 ceph-mon[36790]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T07:42:31.159 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:30 smithi195 ceph-mon[37040]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:42:31.159 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:30 smithi195 ceph-mon[37040]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T07:42:31.159 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.159 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.159 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:30 smithi195 ceph-mon[37040]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:31.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:30 smithi181 ceph-mon[36790]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:30 smithi087 ceph-mon[32276]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-06-06T07:42:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:30 smithi087 ceph-mon[32276]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T07:42:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:31.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:30 smithi087 ceph-mon[32276]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-06T07:42:31.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_1 2024-06-06T07:42:31.825 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] boot 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T07:42:32.186 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] boot 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T07:42:32.256 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:42:32.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:42:32.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:32.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: osd.10 [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] boot 2024-06-06T07:42:32.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T07:42:32.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:42:33.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T07:42:33.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:33.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:32 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:33.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:32 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:33.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:33.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:33.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:32 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 -- 172.21.15.195:0/3719771625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 msgr2=0x7feff410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 --2- 172.21.15.195:0/3719771625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff410aa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fefec00b0a0 tx=0x7fefec00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 -- 172.21.15.195:0/3719771625 shutdown_connections 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 --2- 172.21.15.195:0/3719771625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff410aa00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 --2- 172.21.15.195:0/3719771625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 0x7feff4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 --2- 172.21.15.195:0/3719771625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 0x7feff4103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 -- 172.21.15.195:0/3719771625 >> 172.21.15.195:0/3719771625 conn(0x7feff40fc130 msgr2=0x7feff40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 -- 172.21.15.195:0/3719771625 shutdown_connections 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 -- 172.21.15.195:0/3719771625 wait complete. 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.471+0000 7feff926c700 1 Processor -- start 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 -- start start 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 0x7feff4103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 0x7feff4101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feff410e190 con 0x7feff41007d0 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feff410e010 con 0x7feff41037b0 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff926c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feff410e310 con 0x7feff4104150 2024-06-06T07:42:33.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:56768/0 (socket says 172.21.15.195:56768) 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 -- 172.21.15.195:0/2432676890 learned_addr learned my addr 172.21.15.195:0/2432676890 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7fefea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 0x7feff4101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff2ffd700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 0x7feff4103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 -- 172.21.15.195:0/2432676890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 msgr2=0x7feff4101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 0x7feff4101760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 -- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 msgr2=0x7feff4103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 0x7feff4103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 -- 172.21.15.195:0/2432676890 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff4102550 con 0x7feff4104150 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff37fe700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f00 tx=0x7fefec004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.472+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec021070 con 0x7feff4104150 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.473+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefec007b80 con 0x7feff4104150 2024-06-06T07:42:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.473+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff41a2810 con 0x7feff4104150 2024-06-06T07:42:33.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.473+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec01d450 con 0x7feff4104150 2024-06-06T07:42:33.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.473+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff41a2c70 con 0x7feff4104150 2024-06-06T07:42:33.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.473+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff4069180 con 0x7feff4104150 2024-06-06T07:42:33.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.475+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefec004120 con 0x7feff4104150 2024-06-06T07:42:33.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.475+0000 7feff0ff9700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:33.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.475+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7fefec00f020 con 0x7feff4104150 2024-06-06T07:42:33.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.475+0000 7feff2ffd700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:33.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.476+0000 7feff2ffd700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 0x7fefe00a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feff406a9e0 tx=0x7fefe4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:33.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.479+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fefec02a020 con 0x7feff4104150 2024-06-06T07:42:33.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:33.683+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feff40628d0 con 0x7fefe00a2c20 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi181 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 4272M 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi181 2024-06-06T07:42:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 4272M 2024-06-06T07:42:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T07:42:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi181 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 4272M 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T07:42:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:35.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:34 smithi195 ceph-mon[37040]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:35.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:34 smithi195 ceph-mon[37040]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T07:42:35.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:34 smithi195 ceph-mon[37040]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:35.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:34 smithi181 ceph-mon[36790]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:35.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:34 smithi181 ceph-mon[36790]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T07:42:35.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:34 smithi181 ceph-mon[36790]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:35.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:34 smithi087 ceph-mon[32276]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:42:35.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:34 smithi087 ceph-mon[32276]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T07:42:35.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:34 smithi087 ceph-mon[32276]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:36.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:35 smithi181 ceph-mon[36790]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T07:42:36.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:35 smithi195 ceph-mon[37040]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T07:42:36.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:35 smithi087 ceph-mon[32276]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T07:42:37.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-06T07:42:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/1938813988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]': finished 2024-06-06T07:42:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T07:42:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/1938813988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]': finished 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T07:42:37.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:37.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-06T07:42:37.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/1938813988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]: dispatch 2024-06-06T07:42:37.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53318824-a6a1-4e13-8840-030854cc881d"}]': finished 2024-06-06T07:42:37.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T07:42:37.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:37 smithi195 ceph-mon[37040]: from='client.? 172.21.15.195:0/1274362536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:37 smithi181 ceph-mon[36790]: from='client.? 172.21.15.195:0/1274362536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:37 smithi087 ceph-mon[32276]: from='client.? 172.21.15.195:0/1274362536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T07:42:39.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:38 smithi195 ceph-mon[37040]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-06T07:42:39.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:38 smithi181 ceph-mon[36790]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-06T07:42:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:38 smithi087 ceph-mon[32276]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-06T07:42:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:41 smithi195 ceph-mon[37040]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T07:42:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:41 smithi181 ceph-mon[36790]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T07:42:41.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:41 smithi087 ceph-mon[32276]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T07:42:43.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:43 smithi195 ceph-mon[37040]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T07:42:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:43 smithi181 ceph-mon[36790]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T07:42:43.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:43 smithi087 ceph-mon[32276]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T07:42:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T07:42:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T07:42:44.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T07:42:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:45.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:45 smithi195 ceph-mon[37040]: Deploying daemon osd.11 on smithi195 2024-06-06T07:42:45.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:45 smithi195 ceph-mon[37040]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:45 smithi181 ceph-mon[36790]: Deploying daemon osd.11 on smithi195 2024-06-06T07:42:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:45 smithi181 ceph-mon[36790]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:45.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:45 smithi087 ceph-mon[32276]: Deploying daemon osd.11 on smithi195 2024-06-06T07:42:45.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:45 smithi087 ceph-mon[32276]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:47.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:47 smithi195 ceph-mon[37040]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:47 smithi181 ceph-mon[36790]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:47.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:47 smithi087 ceph-mon[32276]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:49.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:49 smithi195 ceph-mon[37040]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:49.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:49 smithi181 ceph-mon[36790]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:49.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:49 smithi087 ceph-mon[32276]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:50 smithi195 ceph-mon[37040]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:50 smithi181 ceph-mon[36790]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:50 smithi087 ceph-mon[32276]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:53.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:53.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.102 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.101+0000 7feff0ff9700 1 -- 172.21.15.195:0/2432676890 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feff40628d0 con 0x7fefe00a2c20 2024-06-06T07:42:54.102 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 11 on host 'smithi195' 2024-06-06T07:42:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 msgr2=0x7fefe00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 0x7fefe00a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feff406a9e0 tx=0x7fefe4009190 comp rx=0 tx=0).stop 2024-06-06T07:42:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 msgr2=0x7feff4101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f00 tx=0x7fefec004dd0 comp rx=0 tx=0).stop 2024-06-06T07:42:54.104 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 shutdown_connections 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feff4104150 0x7feff4101ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feff41007d0 0x7feff4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 --2- 172.21.15.195:0/2432676890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff41037b0 0x7feff4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 >> 172.21.15.195:0/2432676890 conn(0x7feff40fc130 msgr2=0x7feff41069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 shutdown_connections 2024-06-06T07:42:54.105 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:42:54.104+0000 7feff926c700 1 -- 172.21.15.195:0/2432676890 wait complete. 2024-06-06T07:42:54.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:54 smithi195 ceph-mon[37040]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:54 smithi195 ceph-mon[37040]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:54 smithi181 ceph-mon[36790]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:54 smithi181 ceph-mon[36790]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:54 smithi087 ceph-mon[32276]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:54 smithi087 ceph-mon[32276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T07:42:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:42:54.755 DEBUG:teuthology.orchestra.run.smithi195:osd.11> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.11.service 2024-06-06T07:42:54.757 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-06T07:42:54.758 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T07:42:54.996 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:42:55.353 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:55.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T07:42:55.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T07:42:55.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:55.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-mon[37040]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:55.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T07:42:55.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T07:42:55.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:55.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.434 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:55 smithi087 ceph-mon[32276]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: from='osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:55 smithi181 ceph-mon[36790]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-06-06T07:42:55.991 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T07:42:55.747+0000 7fce69840700 -1 osd.11 0 waiting for initial osdmap 2024-06-06T07:42:55.991 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 07:42:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T07:42:55.761+0000 7fce6162b700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T07:42:56.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:56 smithi195 ceph-mon[37040]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:42:56.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:56 smithi195 ceph-mon[37040]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T07:42:56.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:56 smithi195 ceph-mon[37040]: from='osd.11 ' entity='osd.11' 2024-06-06T07:42:56.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:56 smithi087 ceph-mon[32276]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:42:56.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:56 smithi087 ceph-mon[32276]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T07:42:56.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.455 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:56 smithi087 ceph-mon[32276]: from='osd.11 ' entity='osd.11' 2024-06-06T07:42:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:56 smithi181 ceph-mon[36790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-06-06T07:42:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:56 smithi181 ceph-mon[36790]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T07:42:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:56.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:56 smithi181 ceph-mon[36790]: from='osd.11 ' entity='osd.11' 2024-06-06T07:42:56.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4209787260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 msgr2=0x7f1bb8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:56.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4209787260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8102940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40099c0 tx=0x7f1bb4009cd0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4209787260 shutdown_connections 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4209787260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 0x7f1bb810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4209787260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8102940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4209787260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 0x7f1bb8103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4209787260 >> 172.21.15.87:0/4209787260 conn(0x7f1bb80fdae0 msgr2=0x7f1bb80fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4209787260 shutdown_connections 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.803+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4209787260 wait complete. 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 Processor -- start 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 -- start start 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:56.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 0x7f1bb8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 0x7f1bb819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810e100 con 0x7f1bb8102520 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810df80 con 0x7f1bb81040c0 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbfb1c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb810e280 con 0x7f1bb8103720 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48658/0 (socket says 172.21.15.87:48658) 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 -- 172.21.15.87:0/4026877508 learned_addr learned my addr 172.21.15.87:0/4026877508 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd0b7700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 0x7f1bb8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbe0b9700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 0x7f1bb819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 -- 172.21.15.87:0/4026877508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 msgr2=0x7f1bb8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 0x7f1bb8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 -- 172.21.15.87:0/4026877508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 msgr2=0x7f1bb819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 0x7f1bb819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.804+0000 7f1bbd8b8700 1 -- 172.21.15.87:0/4026877508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb81a0380 con 0x7f1bb8102520 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1bbd8b8700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005b30 tx=0x7f1bb4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4026070 con 0x7f1bb8102520 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81a0610 con 0x7f1bb8102520 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb40058e0 con 0x7f1bb8102520 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400fde0 con 0x7f1bb8102520 2024-06-06T07:42:56.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.805+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81a0af0 con 0x7f1bb8102520 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.806+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb8069180 con 0x7f1bb8102520 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.807+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1bb4029070 con 0x7f1bb8102520 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.808+0000 7f1baeffd700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 0x7f1ba40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.808+0000 7f1bbd0b7700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 0x7f1ba40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.808+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f1bb40cfc00 con 0x7f1bb8102520 2024-06-06T07:42:56.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.808+0000 7f1bbd0b7700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 0x7f1ba40a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8007c60 tx=0x7f1ba80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:56.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:56.811+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1bb4030080 con 0x7f1bb8102520 2024-06-06T07:42:57.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.007+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1bb80630a0 con 0x7f1bb8102520 2024-06-06T07:42:57.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.007+0000 7f1baeffd700 1 -- 172.21.15.87:0/4026877508 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f1bb4013070 con 0x7f1bb8102520 2024-06-06T07:42:57.008 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:42:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 msgr2=0x7f1ba40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 0x7f1ba40a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8007c60 tx=0x7f1ba80073d0 comp rx=0 tx=0).stop 2024-06-06T07:42:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 msgr2=0x7f1bb8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005b30 tx=0x7f1bb4009580 comp rx=0 tx=0).stop 2024-06-06T07:42:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 shutdown_connections 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ba40a2c20 0x7f1ba40a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bb81040c0 0x7f1bb819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb8102520 0x7f1bb8198720 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 --2- 172.21.15.87:0/4026877508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bb8103720 0x7f1bb8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 >> 172.21.15.87:0/4026877508 conn(0x7f1bb80fdae0 msgr2=0x7f1bb8106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 shutdown_connections 2024-06-06T07:42:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:57.010+0000 7f1bbfb1c700 1 -- 172.21.15.87:0/4026877508 wait complete. 2024-06-06T07:42:57.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: purged_snaps scrub starts 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: purged_snaps scrub ok 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] boot 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4026877508' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: purged_snaps scrub starts 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: purged_snaps scrub ok 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] boot 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.463 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4026877508' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T07:42:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: purged_snaps scrub starts 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: purged_snaps scrub ok 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: osd.11 [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] boot 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:42:57.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4026877508' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T07:42:57.797 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1717659776,"num_in_osds":12,"osd_in_since":1717659756,"num_remapped_pgs":0} 2024-06-06T07:42:57.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:42:58.056 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:42:59.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T07:42:59.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: Detected new or changed devices on smithi195 2024-06-06T07:42:59.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 4272M 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:58 smithi195 ceph-mon[37040]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:42:59.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T07:42:59.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: Detected new or changed devices on smithi195 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 4272M 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:59.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:59.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:58 smithi181 ceph-mon[36790]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: Detected new or changed devices on smithi195 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 4272M 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:42:59.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:42:59.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:42:59.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:58 smithi087 ceph-mon[32276]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:42:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- 172.21.15.87:0/3681858872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7401010b0 msgr2=0x7fe740101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 --2- 172.21.15.87:0/3681858872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7401010b0 0x7fe740101490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe730009990 tx=0x7fe730009ca0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- 172.21.15.87:0/3681858872 shutdown_connections 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 --2- 172.21.15.87:0/3681858872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe740105f50 0x7fe74010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 --2- 172.21.15.87:0/3681858872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7401010b0 0x7fe740101490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 --2- 172.21.15.87:0/3681858872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 0x7fe740105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- 172.21.15.87:0/3681858872 >> 172.21.15.87:0/3681858872 conn(0x7fe7400766b0 msgr2=0x7fe740076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- 172.21.15.87:0/3681858872 shutdown_connections 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- 172.21.15.87:0/3681858872 wait complete. 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 Processor -- start 2024-06-06T07:42:59.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.674+0000 7fe74683d700 1 -- start start 2024-06-06T07:42:59.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7401010b0 0x7fe74019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 0x7fe74019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401128b0 con 0x7fe740105f50 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112730 con 0x7fe7401010b0 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe74683d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112a30 con 0x7fe740101a60 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36880/0 (socket says 172.21.15.87:36880) 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 -- 172.21.15.87:0/3834153828 learned_addr learned my addr 172.21.15.87:0/3834153828 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe73f7fe700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 0x7fe74019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe73ffff700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7401010b0 0x7fe74019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 -- 172.21.15.87:0/3834153828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 msgr2=0x7fe74019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:59.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 0x7fe74019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 -- 172.21.15.87:0/3834153828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7401010b0 msgr2=0x7fe74019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7401010b0 0x7fe74019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.675+0000 7fe744dda700 1 -- 172.21.15.87:0/3834153828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe740199210 con 0x7fe740105f50 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe744dda700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800d860 con 0x7fe740105f50 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73800e430 con 0x7fe740105f50 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738017660 con 0x7fe740105f50 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe740199500 con 0x7fe740105f50 2024-06-06T07:42:59.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.676+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7401a57c0 con 0x7fe740105f50 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.677+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7401090e0 con 0x7fe740105f50 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.678+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe73800e5e0 con 0x7fe740105f50 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.678+0000 7fe73d7fa700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 0x7fe7280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.679+0000 7fe73ffff700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 0x7fe7280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.679+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7380ce6c0 con 0x7fe740105f50 2024-06-06T07:42:59.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.679+0000 7fe73ffff700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 0x7fe7280a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe73000b5c0 tx=0x7fe730005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:42:59.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.683+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe738013030 con 0x7fe740105f50 2024-06-06T07:42:59.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.880+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe740199fe0 con 0x7fe740105f50 2024-06-06T07:42:59.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.881+0000 7fe73d7fa700 1 -- 172.21.15.87:0/3834153828 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18482 (secure 0 0 0) 0x7fe73809bac0 con 0x7fe740105f50 2024-06-06T07:42:59.881 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:42:59.882 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":81,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:42:57.757402+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:42:59.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 msgr2=0x7fe7280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:59.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 0x7fe7280a5160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe73000b5c0 tx=0x7fe730005960 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 msgr2=0x7fe740198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800bc80 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 shutdown_connections 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe7280a2ca0 0x7fe7280a5160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7401010b0 0x7fe74019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe740105f50 0x7fe740198ab0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 --2- 172.21.15.87:0/3834153828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740101a60 0x7fe74019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 >> 172.21.15.87:0/3834153828 conn(0x7fe7400766b0 msgr2=0x7fe740100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 shutdown_connections 2024-06-06T07:42:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:42:59.883+0000 7fe74683d700 1 -- 172.21.15.87:0/3834153828 wait complete. 2024-06-06T07:43:00.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:42:59 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3834153828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:43:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:42:59 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3834153828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:43:00.348 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:42:59 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3834153828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:43:00.605 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T07:39:10.366934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T07:43:00.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-06T07:43:00.840 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:01.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:00 smithi195 ceph-mon[37040]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:01.263 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:00 smithi087 ceph-mon[32276]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:01.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:00 smithi181 ceph-mon[36790]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:02.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 -- 172.21.15.87:0/3254263638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006abc0 msgr2=0x7f7e6006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:02.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 --2- 172.21.15.87:0/3254263638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006abc0 0x7f7e6006b040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009990 tx=0x7f7e50009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 -- 172.21.15.87:0/3254263638 shutdown_connections 2024-06-06T07:43:02.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 --2- 172.21.15.87:0/3254263638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006b580 0x7f7e6010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 --2- 172.21.15.87:0/3254263638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 0x7f7e601088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 --2- 172.21.15.87:0/3254263638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006abc0 0x7f7e6006b040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 -- 172.21.15.87:0/3254263638 >> 172.21.15.87:0/3254263638 conn(0x7f7e60078340 msgr2=0x7f7e60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 -- 172.21.15.87:0/3254263638 shutdown_connections 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.483+0000 7f7e67539700 1 -- 172.21.15.87:0/3254263638 wait complete. 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 Processor -- start 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 -- start start 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006abc0 0x7f7e60101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 0x7f7e60101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:02.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60112860 con 0x7f7e6006b580 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e601126e0 con 0x7f7e6006abc0 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e67539700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e601129e0 con 0x7f7e60108500 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36896/0 (socket says 172.21.15.87:36896) 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 -- 172.21.15.87:0/2978371781 learned_addr learned my addr 172.21.15.87:0/2978371781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e652d5700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006abc0 0x7f7e60101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e65ad6700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 0x7f7e60101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 -- 172.21.15.87:0/2978371781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 msgr2=0x7f7e60101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 0x7f7e60101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 -- 172.21.15.87:0/2978371781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006abc0 msgr2=0x7f7e60101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006abc0 0x7f7e60101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.484+0000 7f7e64ad4700 1 -- 172.21.15.87:0/2978371781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e601a72e0 con 0x7f7e6006b580 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e64ad4700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e50000c00 tx=0x7f7e50003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50027070 con 0x7f7e6006b580 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50004110 con 0x7f7e6006b580 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000f460 con 0x7f7e6006b580 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e601a7570 con 0x7f7e6006b580 2024-06-06T07:43:02.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.485+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601a7970 con 0x7f7e6006b580 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.486+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6010a9e0 con 0x7f7e6006b580 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.487+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7e5002a070 con 0x7f7e6006b580 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.488+0000 7f7e567fc700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 0x7f7e4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.488+0000 7f7e652d5700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 0x7f7e4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.488+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7e500cfb20 con 0x7f7e6006b580 2024-06-06T07:43:02.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.488+0000 7f7e652d5700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 0x7f7e4c0a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009a10 tx=0x7f7e5c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:02.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.492+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e5005e850 con 0x7f7e6006b580 2024-06-06T07:43:02.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.688+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e601029c0 con 0x7f7e6006b580 2024-06-06T07:43:02.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.688+0000 7f7e567fc700 1 -- 172.21.15.87:0/2978371781 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f7e50013070 con 0x7f7e6006b580 2024-06-06T07:43:02.689 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.690+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 msgr2=0x7f7e4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.690+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 0x7f7e4c0a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009a10 tx=0x7f7e5c009190 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.690+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 msgr2=0x7f7e601019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.690+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e50000c00 tx=0x7f7e50003680 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 shutdown_connections 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7e4c0a2c20 0x7f7e4c0a50e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6006abc0 0x7f7e60101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e60108500 0x7f7e60101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 --2- 172.21.15.87:0/2978371781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e6006b580 0x7f7e601019d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:02.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 >> 172.21.15.87:0/2978371781 conn(0x7f7e60078340 msgr2=0x7f7e601000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:02.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 shutdown_connections 2024-06-06T07:43:02.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:02.691+0000 7f7e67539700 1 -- 172.21.15.87:0/2978371781 wait complete. 2024-06-06T07:43:03.266 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:03 smithi087 ceph-mon[32276]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:03.266 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2978371781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.a on smithi087 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.d on smithi087 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.g on smithi087 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.j on smithi087 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.b on smithi181 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.e on smithi181 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.h on smithi181 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.k on smithi181 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.c on smithi195 2024-06-06T07:43:03.295 INFO:tasks.cephadm:Adding mds.f on smithi195 2024-06-06T07:43:03.296 INFO:tasks.cephadm:Adding mds.i on smithi195 2024-06-06T07:43:03.296 INFO:tasks.cephadm:Adding mds.l on smithi195 2024-06-06T07:43:03.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l' 2024-06-06T07:43:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:03 smithi195 ceph-mon[37040]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2978371781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:43:03.545 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:43:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:03 smithi181 ceph-mon[36790]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2978371781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:43:05.200 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 -- 172.21.15.195:0/1365584052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478102fd0 msgr2=0x7ff4781053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/1365584052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478102fd0 0x7ff4781053c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff468009990 tx=0x7ff468009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 -- 172.21.15.195:0/1365584052 shutdown_connections 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/1365584052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478108230 0x7ff47810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/1365584052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478102fd0 0x7ff4781053c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/1365584052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 0x7ff478107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 -- 172.21.15.195:0/1365584052 >> 172.21.15.195:0/1365584052 conn(0x7ff4780fc9c0 msgr2=0x7ff4780fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 -- 172.21.15.195:0/1365584052 shutdown_connections 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.198+0000 7ff47f4e6700 1 -- 172.21.15.195:0/1365584052 wait complete. 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 Processor -- start 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 -- start start 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478102fd0 0x7ff478102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 0x7ff478100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:05.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810e0f0 con 0x7ff478102fd0 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810df70 con 0x7ff478105900 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47f4e6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810e270 con 0x7ff478108230 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47da83700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47da83700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:45438/0 (socket says 172.21.15.195:45438) 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47da83700 1 -- 172.21.15.195:0/37332935 learned_addr learned my addr 172.21.15.195:0/37332935 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.199+0000 7ff47ca81700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 0x7ff478100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47d282700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478102fd0 0x7ff478102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 -- 172.21.15.195:0/37332935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 msgr2=0x7ff478100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 0x7ff478100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 -- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478102fd0 msgr2=0x7ff478102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478102fd0 0x7ff478102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 -- 172.21.15.195:0/37332935 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff478101d70 con 0x7ff478108230 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47da83700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff470007d40 tx=0x7ff47000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470016020 con 0x7ff478108230 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47000ad40 con 0x7ff478108230 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4781a27f0 con 0x7ff478108230 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700115e0 con 0x7ff478108230 2024-06-06T07:43:05.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.200+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4781a2d00 con 0x7ff478108230 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.201+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff478069180 con 0x7ff478108230 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.202+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff470019020 con 0x7ff478108230 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.203+0000 7ff46e7fc700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 0x7ff4640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.203+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4700ceab0 con 0x7ff478108230 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.203+0000 7ff47d282700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 0x7ff4640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:05.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.203+0000 7ff47d282700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 0x7ff4640a5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b5c0 tx=0x7ff468005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:05.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.207+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff470020020 con 0x7ff478108230 2024-06-06T07:43:05.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:05 smithi195 ceph-mon[37040]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:05.415 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.414+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}) -- 0x7ff4780629d0 con 0x7ff4640a2ca0 2024-06-06T07:43:05.425 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.425+0000 7ff46e7fc700 1 -- 172.21.15.195:0/37332935 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff4780629d0 con 0x7ff4640a2ca0 2024-06-06T07:43:05.425 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mds.all update... 2024-06-06T07:43:05.427 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 msgr2=0x7ff4640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:05.427 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 0x7ff4640a5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b5c0 tx=0x7ff468005fb0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.427 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 msgr2=0x7ff4781014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff470007d40 tx=0x7ff47000a340 comp rx=0 tx=0).stop 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 shutdown_connections 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4640a2ca0 0x7ff4640a5160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff478108230 0x7ff4781014c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff478102fd0 0x7ff478102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 --2- 172.21.15.195:0/37332935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff478105900 0x7ff478100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 >> 172.21.15.195:0/37332935 conn(0x7ff4780fc9c0 msgr2=0x7ff478103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 shutdown_connections 2024-06-06T07:43:05.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-06T07:43:05.427+0000 7ff47f4e6700 1 -- 172.21.15.195:0/37332935 wait complete. 2024-06-06T07:43:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:05 smithi181 ceph-mon[36790]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:05.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:05 smithi087 ceph-mon[32276]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:06.061 DEBUG:teuthology.orchestra.run.smithi087:mds.a> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.a.service 2024-06-06T07:43:06.063 DEBUG:teuthology.orchestra.run.smithi087:mds.d> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.d.service 2024-06-06T07:43:06.107 DEBUG:teuthology.orchestra.run.smithi087:mds.g> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.g.service 2024-06-06T07:43:06.151 DEBUG:teuthology.orchestra.run.smithi087:mds.j> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.j.service 2024-06-06T07:43:06.194 DEBUG:teuthology.orchestra.run.smithi181:mds.b> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.b.service 2024-06-06T07:43:06.196 DEBUG:teuthology.orchestra.run.smithi181:mds.e> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.e.service 2024-06-06T07:43:06.239 DEBUG:teuthology.orchestra.run.smithi181:mds.h> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.h.service 2024-06-06T07:43:06.281 DEBUG:teuthology.orchestra.run.smithi181:mds.k> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.k.service 2024-06-06T07:43:06.323 DEBUG:teuthology.orchestra.run.smithi195:mds.c> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.c.service 2024-06-06T07:43:06.326 DEBUG:teuthology.orchestra.run.smithi195:mds.f> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.f.service 2024-06-06T07:43:06.369 DEBUG:teuthology.orchestra.run.smithi195:mds.i> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.i.service 2024-06-06T07:43:06.412 DEBUG:teuthology.orchestra.run.smithi195:mds.l> sudo journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.l.service 2024-06-06T07:43:06.455 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-06T07:43:06.455 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:06.691 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.702 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:06.703 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:06.703 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.703 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: Deploying daemon mds.l on smithi195 2024-06-06T07:43:06.703 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:06 smithi195 ceph-mon[37040]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:06.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:43:06.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-06-06T07:43:06.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.727 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: Deploying daemon mds.l on smithi195 2024-06-06T07:43:06.728 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:06 smithi181 ceph-mon[36790]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:06.812 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:06.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:06.814 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:06.814 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:06.814 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: Deploying daemon mds.l on smithi195 2024-06-06T07:43:06.814 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:06 smithi087 ceph-mon[32276]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:08.260 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 systemd[1]: Starting Ceph mds.l for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:08.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2488103734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac100e50 msgr2=0x7ff1ac1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2488103734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac100e50 0x7ff1ac1012d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009990 tx=0x7ff1a0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2488103734 shutdown_connections 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2488103734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac107570 0x7ff1ac107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2488103734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac101810 0x7ff1ac10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2488103734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac100e50 0x7ff1ac1012d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2488103734 >> 172.21.15.87:0/2488103734 conn(0x7ff1ac0766b0 msgr2=0x7ff1ac076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2488103734 shutdown_connections 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.429+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2488103734 wait complete. 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 Processor -- start 2024-06-06T07:43:08.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 -- start start 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac100e50 0x7ff1ac19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac101810 0x7ff1ac19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1127b0 con 0x7ff1ac107570 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac112630 con 0x7ff1ac101810 2024-06-06T07:43:08.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1b08b2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac112930 con 0x7ff1ac100e50 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35088/0 (socket says 172.21.15.87:35088) 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 -- 172.21.15.87:0/2158780998 learned_addr learned my addr 172.21.15.87:0/2158780998 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1a9d9b700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac101810 0x7ff1ac19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aa59c700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac100e50 0x7ff1ac19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 -- 172.21.15.87:0/2158780998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac100e50 msgr2=0x7ff1ac19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac100e50 0x7ff1ac19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 -- 172.21.15.87:0/2158780998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac101810 msgr2=0x7ff1ac19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac101810 0x7ff1ac19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 -- 172.21.15.87:0/2158780998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac1990f0 con 0x7ff1ac107570 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff1aad9d700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4007ee0 tx=0x7ff1a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.430+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4004bb0 con 0x7ff1ac107570 2024-06-06T07:43:08.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.431+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a4004d50 con 0x7ff1ac107570 2024-06-06T07:43:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.431+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac1993e0 con 0x7ff1ac107570 2024-06-06T07:43:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.431+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4005960 con 0x7ff1ac107570 2024-06-06T07:43:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.431+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a56a0 con 0x7ff1ac107570 2024-06-06T07:43:08.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.431+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac069180 con 0x7ff1ac107570 2024-06-06T07:43:08.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.433+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1a401a020 con 0x7ff1ac107570 2024-06-06T07:43:08.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.433+0000 7ff19b7fe700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 0x7ff1940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:08.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.433+0000 7ff1aa59c700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 0x7ff1940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:08.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.433+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff1a40ce9b0 con 0x7ff1ac107570 2024-06-06T07:43:08.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.434+0000 7ff1aa59c700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 0x7ff1940a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac199ad0 tx=0x7ff19c00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:08.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.437+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1a4010070 con 0x7ff1ac107570 2024-06-06T07:43:08.650 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 podman[67344]: 2024-06-06 07:43:08.283713382 +0000 UTC m=+0.017518863 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:08.650 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 podman[67344]: 2024-06-06 07:43:08.397578033 +0000 UTC m=+0.131383490 container create 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T07:43:08.650 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 podman[67344]: 2024-06-06 07:43:08.650664206 +0000 UTC m=+0.384469673 container init 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:43:08.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.681+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1ac1a5870 con 0x7ff1ac107570 2024-06-06T07:43:08.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.681+0000 7ff19b7fe700 1 -- 172.21.15.87:0/2158780998 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff1a409bdb0 con 0x7ff1ac107570 2024-06-06T07:43:08.682 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:08.682 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 msgr2=0x7ff1940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 0x7ff1940a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac199ad0 tx=0x7ff19c00a3b0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 msgr2=0x7ff1ac198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4007ee0 tx=0x7ff1a400d3b0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 shutdown_connections 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff1940a2c20 0x7ff1940a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1ac107570 0x7ff1ac198990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ac100e50 0x7ff1ac19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 --2- 172.21.15.87:0/2158780998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac101810 0x7ff1ac19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 >> 172.21.15.87:0/2158780998 conn(0x7ff1ac0766b0 msgr2=0x7ff1ac0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 shutdown_connections 2024-06-06T07:43:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:08.684+0000 7ff1b08b2700 1 -- 172.21.15.87:0/2158780998 wait complete. 2024-06-06T07:43:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 1 2024-06-06T07:43:09.010 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 podman[67344]: 2024-06-06 07:43:08.653407901 +0000 UTC m=+0.387213381 container start 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:43:09.010 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 ceph-mds[67363]: -- 172.21.15.195:0/1381747387 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f76231a40 con 0x560f76204c00 2024-06-06T07:43:09.010 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l[67359]: starting mds.l at 2024-06-06T07:43:09.010 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 bash[67344]: 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac 2024-06-06T07:43:09.011 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 07:43:08 smithi195 systemd[1]: Started Ceph mds.l for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:09.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2158780998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6832/2222795089,v1:172.21.15.195:6833/2222795089] up:boot 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: fsmap 1 up:standby 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:09.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:09 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2158780998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6832/2222795089,v1:172.21.15.195:6833/2222795089] up:boot 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: fsmap 1 up:standby 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:09.476 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:09 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2158780998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6832/2222795089,v1:172.21.15.195:6833/2222795089] up:boot 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: fsmap 1 up:standby 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:09.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:09 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:10.508 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:10 smithi181 ceph-mon[36790]: Deploying daemon mds.k on smithi181 2024-06-06T07:43:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:10 smithi195 ceph-mon[37040]: Deploying daemon mds.k on smithi181 2024-06-06T07:43:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:10 smithi087 ceph-mon[32276]: Deploying daemon mds.k on smithi181 2024-06-06T07:43:11.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:11 smithi181 ceph-mon[36790]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:11.307 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 systemd[1]: Starting Ceph mds.k for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:11.451 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:11.479 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:11 smithi087 ceph-mon[32276]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:11.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:11 smithi195 ceph-mon[37040]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:11.685 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 podman[66399]: 2024-06-06 07:43:11.415080644 +0000 UTC m=+0.111322504 container create 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:43:11.685 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 podman[66399]: 2024-06-06 07:43:11.321008404 +0000 UTC m=+0.017250282 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:11.685 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 podman[66399]: 2024-06-06 07:43:11.659586225 +0000 UTC m=+0.355828072 container init 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-06T07:43:11.685 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 podman[66399]: 2024-06-06 07:43:11.662191734 +0000 UTC m=+0.358433592 container start 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-06T07:43:11.685 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 ceph-mds[66418]: -- 172.21.15.181:0/40231723 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55890e24fa40 con 0x55890e222800 2024-06-06T07:43:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:12.056 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k[66414]: starting mds.k at 2024-06-06T07:43:12.057 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 bash[66399]: 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe 2024-06-06T07:43:12.057 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 07:43:11 smithi181 systemd[1]: Started Ceph mds.k for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:13.099 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.099 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.099 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.099 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: Deploying daemon mds.j on smithi087 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6832/2618702621,v1:172.21.15.181:6833/2618702621] up:boot 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: fsmap 2 up:standby 2024-06-06T07:43:13.100 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:12 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:43:13.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: Deploying daemon mds.j on smithi087 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6832/2618702621,v1:172.21.15.181:6833/2618702621] up:boot 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: fsmap 2 up:standby 2024-06-06T07:43:13.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:12 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:43:13.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: Deploying daemon mds.j on smithi087 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6832/2618702621,v1:172.21.15.181:6833/2618702621] up:boot 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: fsmap 2 up:standby 2024-06-06T07:43:13.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:12 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:43:13.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 -- 172.21.15.87:0/604031879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee60107840 msgr2=0x7fee6010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:13.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 --2- 172.21.15.87:0/604031879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee60107840 0x7fee6010f3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b0d0 tx=0x7fee5800b3e0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 -- 172.21.15.87:0/604031879 shutdown_connections 2024-06-06T07:43:13.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 --2- 172.21.15.87:0/604031879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee60107840 0x7fee6010f3c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 --2- 172.21.15.87:0/604031879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60102740 0x7fee60102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 --2- 172.21.15.87:0/604031879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee601030f0 0x7fee60107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 -- 172.21.15.87:0/604031879 >> 172.21.15.87:0/604031879 conn(0x7fee600795f0 msgr2=0x7fee60079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 -- 172.21.15.87:0/604031879 shutdown_connections 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.466+0000 7fee64952700 1 -- 172.21.15.87:0/604031879 wait complete. 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 Processor -- start 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 -- start start 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60102740 0x7fee6019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60107840 0x7fee60198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee60112850 con 0x7fee601030f0 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee601126d0 con 0x7fee60102740 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee64952700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee601129d0 con 0x7fee60107840 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.467+0000 7fee5dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35116/0 (socket says 172.21.15.87:35116) 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 -- 172.21.15.87:0/2081753079 learned_addr learned my addr 172.21.15.87:0/2081753079 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5e59c700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60102740 0x7fee6019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5ed9d700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60107840 0x7fee60198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 -- 172.21.15.87:0/2081753079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60107840 msgr2=0x7fee60198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:13.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60107840 0x7fee60198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 -- 172.21.15.87:0/2081753079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60102740 msgr2=0x7fee6019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60102740 0x7fee6019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 -- 172.21.15.87:0/2081753079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60199290 con 0x7fee601030f0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee5dd9b700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b530 tx=0x7fee5000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50015020 con 0x7fee601030f0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee50011d10 con 0x7fee601030f0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000f2e0 con 0x7fee601030f0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee60199580 con 0x7fee601030f0 2024-06-06T07:43:13.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.468+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a5870 con 0x7fee601030f0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.469+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee60069180 con 0x7fee601030f0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.471+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fee50018020 con 0x7fee601030f0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.472+0000 7fee4f7fe700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 0x7fee480a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.472+0000 7fee5e59c700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 0x7fee480a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.472+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fee500ce860 con 0x7fee601030f0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.472+0000 7fee5e59c700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 0x7fee480a51e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee54009a10 tx=0x7fee54009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:13.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.475+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee5001f020 con 0x7fee601030f0 2024-06-06T07:43:13.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.725+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee6002dca0 con 0x7fee601030f0 2024-06-06T07:43:13.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.725+0000 7fee4f7fe700 1 -- 172.21.15.87:0/2081753079 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fee5009bbc0 con 0x7fee601030f0 2024-06-06T07:43:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:13.726 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.727+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 msgr2=0x7fee480a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.727+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 0x7fee480a51e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee54009a10 tx=0x7fee54009190 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.727+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 msgr2=0x7fee6019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.727+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b530 tx=0x7fee5000b840 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 shutdown_connections 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fee480a2d20 0x7fee480a51e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60102740 0x7fee6019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee60107840 0x7fee60198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 --2- 172.21.15.87:0/2081753079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee601030f0 0x7fee6019ee60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 >> 172.21.15.87:0/2081753079 conn(0x7fee600795f0 msgr2=0x7fee601006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:13.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 shutdown_connections 2024-06-06T07:43:13.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:13.728+0000 7fee64952700 1 -- 172.21.15.87:0/2081753079 wait complete. 2024-06-06T07:43:13.729 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 3 2024-06-06T07:43:14.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:13 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2081753079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:14.114 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:13 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2081753079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:13 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2081753079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:15.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:14 smithi087 ceph-mon[32276]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:14 smithi195 ceph-mon[37040]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:15.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:14 smithi181 ceph-mon[36790]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:16.110 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:15 smithi087 systemd[1]: Starting Ceph mds.j for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:16.467 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 podman[69779]: 2024-06-06 07:43:16.097082608 +0000 UTC m=+0.016237335 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:16.468 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 podman[69779]: 2024-06-06 07:43:16.214229907 +0000 UTC m=+0.133384621 container create 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-06T07:43:16.468 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 podman[69779]: 2024-06-06 07:43:16.441565972 +0000 UTC m=+0.360720687 container init 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T07:43:16.468 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 podman[69779]: 2024-06-06 07:43:16.444511906 +0000 UTC m=+0.363666637 container start 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T07:43:16.468 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 ceph-mds[69798]: -- 172.21.15.87:0/3735310526 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e40dada40 con 0x563e40d80800 2024-06-06T07:43:16.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:16.772 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:16.859 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j[69794]: starting mds.j at 2024-06-06T07:43:16.860 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 bash[69779]: 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea 2024-06-06T07:43:16.860 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 07:43:16 smithi087 systemd[1]: Started Ceph mds.j for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:17.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:17 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:17.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:17 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:17.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:17 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:18.343 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:18 smithi087 ceph-mon[32276]: Deploying daemon mds.d on smithi087 2024-06-06T07:43:18.343 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:18 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.87:6834/2593068869,v1:172.21.15.87:6835/2593068869] up:boot 2024-06-06T07:43:18.343 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:18 smithi087 ceph-mon[32276]: fsmap 3 up:standby 2024-06-06T07:43:18.343 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:18 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:43:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:18 smithi195 ceph-mon[37040]: Deploying daemon mds.d on smithi087 2024-06-06T07:43:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:18 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.87:6834/2593068869,v1:172.21.15.87:6835/2593068869] up:boot 2024-06-06T07:43:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:18 smithi195 ceph-mon[37040]: fsmap 3 up:standby 2024-06-06T07:43:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:18 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:43:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:18 smithi181 ceph-mon[36790]: Deploying daemon mds.d on smithi087 2024-06-06T07:43:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:18 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.87:6834/2593068869,v1:172.21.15.87:6835/2593068869] up:boot 2024-06-06T07:43:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:18 smithi181 ceph-mon[36790]: fsmap 3 up:standby 2024-06-06T07:43:18.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:18 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:43:19.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 -- 172.21.15.87:0/1460453341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406aa00 msgr2=0x7fab5406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:19.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 --2- 172.21.15.87:0/1460453341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406aa00 0x7fab5406ae60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab48009990 tx=0x7fab48009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 -- 172.21.15.87:0/1460453341 shutdown_connections 2024-06-06T07:43:19.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 --2- 172.21.15.87:0/1460453341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab541084d0 0x7fab541088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 --2- 172.21.15.87:0/1460453341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab5406b3a0 0x7fab5410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 --2- 172.21.15.87:0/1460453341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406aa00 0x7fab5406ae60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.221+0000 7fab5888c700 1 -- 172.21.15.87:0/1460453341 >> 172.21.15.87:0/1460453341 conn(0x7fab54078330 msgr2=0x7fab54078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:19.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.222+0000 7fab5888c700 1 -- 172.21.15.87:0/1460453341 shutdown_connections 2024-06-06T07:43:19.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.222+0000 7fab5888c700 1 -- 172.21.15.87:0/1460453341 wait complete. 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.222+0000 7fab5888c700 1 Processor -- start 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 -- start start 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 0x7fab54101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab541084d0 0x7fab54101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab54112750 con 0x7fab5406b3a0 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab541125d0 con 0x7fab5406aa00 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5888c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab541128d0 con 0x7fab541084d0 2024-06-06T07:43:19.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49924/0 (socket says 172.21.15.87:49924) 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 -- 172.21.15.87:0/2996172212 learned_addr learned my addr 172.21.15.87:0/2996172212 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab52d9d700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab541084d0 0x7fab54101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5259c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 0x7fab54101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 -- 172.21.15.87:0/2996172212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab541084d0 msgr2=0x7fab54101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab541084d0 0x7fab54101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 -- 172.21.15.87:0/2996172212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 msgr2=0x7fab54101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 0x7fab54101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 -- 172.21.15.87:0/2996172212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab541a7150 con 0x7fab5406b3a0 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab5259c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 0x7fab54101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.223+0000 7fab51d9b700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab48000c00 tx=0x7fab480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:19.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.224+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48027070 con 0x7fab5406b3a0 2024-06-06T07:43:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.224+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab541a73e0 con 0x7fab5406b3a0 2024-06-06T07:43:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.224+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48003780 con 0x7fab5406b3a0 2024-06-06T07:43:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.224+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48022dd0 con 0x7fab5406b3a0 2024-06-06T07:43:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.224+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab541a7890 con 0x7fab5406b3a0 2024-06-06T07:43:19.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.225+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab54103b80 con 0x7fab5406b3a0 2024-06-06T07:43:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.226+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab4802a070 con 0x7fab5406b3a0 2024-06-06T07:43:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.227+0000 7fab437fe700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 0x7fab3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.227+0000 7fab5259c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 0x7fab3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.227+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fab480cfb80 con 0x7fab5406b3a0 2024-06-06T07:43:19.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.227+0000 7fab5259c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 0x7fab3c0a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab44009d50 tx=0x7fab44009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:19.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.234+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab48013070 con 0x7fab5406b3a0 2024-06-06T07:43:19.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:19 smithi087 ceph-mon[32276]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:19.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.479+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5402dca0 con 0x7fab5406b3a0 2024-06-06T07:43:19.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.479+0000 7fab437fe700 1 -- 172.21.15.87:0/2996172212 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fab4809d0a0 con 0x7fab5406b3a0 2024-06-06T07:43:19.480 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:19.480 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 msgr2=0x7fab3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 0x7fab3c0a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab44009d50 tx=0x7fab44009040 comp rx=0 tx=0).stop 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 msgr2=0x7fab54101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab48000c00 tx=0x7fab480047b0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 shutdown_connections 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab3c0a2c20 0x7fab3c0a50e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5406aa00 0x7fab54101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab541084d0 0x7fab54101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 --2- 172.21.15.87:0/2996172212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5406b3a0 0x7fab54101990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 >> 172.21.15.87:0/2996172212 conn(0x7fab54078330 msgr2=0x7fab540ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 shutdown_connections 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:19.482+0000 7fab5888c700 1 -- 172.21.15.87:0/2996172212 wait complete. 2024-06-06T07:43:19.483 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 4 2024-06-06T07:43:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:19 smithi195 ceph-mon[37040]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:19.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:19 smithi181 ceph-mon[36790]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:20.415 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:20 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2996172212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:20.416 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:20 smithi087 systemd[1]: Starting Ceph mds.d for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:20 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2996172212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:20 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2996172212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:21.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:21 smithi087 ceph-mon[32276]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:21.269 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:20 smithi087 podman[70238]: 2024-06-06 07:43:20.873280711 +0000 UTC m=+0.017032293 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:21.269 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 podman[70238]: 2024-06-06 07:43:21.017348493 +0000 UTC m=+0.161100099 container create 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-06T07:43:21.269 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 podman[70238]: 2024-06-06 07:43:21.269852094 +0000 UTC m=+0.413603658 container init 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:43:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:21 smithi195 ceph-mon[37040]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:21 smithi181 ceph-mon[36790]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:21.610 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 podman[70238]: 2024-06-06 07:43:21.272844164 +0000 UTC m=+0.416595738 container start 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T07:43:21.610 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 ceph-mds[70257]: -- 172.21.15.87:0/1615652308 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b5d621a40 con 0x563b5d5f4c00 2024-06-06T07:43:21.610 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d[70253]: starting mds.d at 2024-06-06T07:43:21.610 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 bash[70238]: 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af 2024-06-06T07:43:21.610 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 07:43:21 smithi087 systemd[1]: Started Ceph mds.d for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:22.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:22.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: Deploying daemon mds.e on smithi181 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.87:6836/2477022406,v1:172.21.15.87:6837/2477022406] up:boot 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: fsmap 4 up:standby 2024-06-06T07:43:22.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:22 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: Deploying daemon mds.e on smithi181 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.87:6836/2477022406,v1:172.21.15.87:6837/2477022406] up:boot 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: fsmap 4 up:standby 2024-06-06T07:43:22.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:22 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:43:22.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: Deploying daemon mds.e on smithi181 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.87:6836/2477022406,v1:172.21.15.87:6837/2477022406] up:boot 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: fsmap 4 up:standby 2024-06-06T07:43:22.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:22 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:43:22.987 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:23.791 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:23 smithi181 systemd[1]: Starting Ceph mds.e for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:24.056 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 podman[66714]: 2024-06-06 07:43:23.909019117 +0000 UTC m=+0.016920658 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:24.057 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 podman[66714]: 2024-06-06 07:43:24.029470878 +0000 UTC m=+0.137372448 container create 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 podman[66714]: 2024-06-06 07:43:24.265692355 +0000 UTC m=+0.373593877 container init 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 podman[66714]: 2024-06-06 07:43:24.268362108 +0000 UTC m=+0.376263637 container start 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 ceph-mds[66733]: -- 172.21.15.181:0/178892774 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575f1f4da40 con 0x5575f1f20c00 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: starting mds.e at 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 bash[66714]: 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd 2024-06-06T07:43:24.533 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 07:43:24 smithi181 systemd[1]: Started Ceph mds.e for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:24.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 -- 172.21.15.87:0/1306447182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101d50 msgr2=0x7fa23810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:24.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 --2- 172.21.15.87:0/1306447182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101d50 0x7fa23810e240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa23000b0a0 tx=0x7fa23000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 -- 172.21.15.87:0/1306447182 shutdown_connections 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 --2- 172.21.15.87:0/1306447182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238107400 0x7fa2381077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 --2- 172.21.15.87:0/1306447182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101d50 0x7fa23810e240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 --2- 172.21.15.87:0/1306447182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238101390 0x7fa238101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 -- 172.21.15.87:0/1306447182 >> 172.21.15.87:0/1306447182 conn(0x7fa238078340 msgr2=0x7fa238078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 -- 172.21.15.87:0/1306447182 shutdown_connections 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.765+0000 7fa23f241700 1 -- 172.21.15.87:0/1306447182 wait complete. 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 Processor -- start 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 -- start start 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238101d50 0x7fa2381035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238107400 0x7fa238103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:24.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381116a0 con 0x7fa238101390 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111520 con 0x7fa238101d50 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23f241700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa238111820 con 0x7fa238107400 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49954/0 (socket says 172.21.15.87:49954) 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23d7de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238107400 0x7fa238103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 -- 172.21.15.87:0/3476963390 learned_addr learned my addr 172.21.15.87:0/3476963390 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa237fff700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238101d50 0x7fa2381035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 -- 172.21.15.87:0/3476963390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238107400 msgr2=0x7fa238103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238107400 0x7fa238103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 -- 172.21.15.87:0/3476963390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238101d50 msgr2=0x7fa2381035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238101d50 0x7fa2381035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.766+0000 7fa23cfdd700 1 -- 172.21.15.87:0/3476963390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2381a70b0 con 0x7fa238101390 2024-06-06T07:43:24.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa23cfdd700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c9b0 tx=0x7fa22c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:24.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c007b00 con 0x7fa238101390 2024-06-06T07:43:24.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2381a73a0 con 0x7fa238101390 2024-06-06T07:43:24.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c00ce50 con 0x7fa238101390 2024-06-06T07:43:24.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c018660 con 0x7fa238101390 2024-06-06T07:43:24.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.767+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a7930 con 0x7fa238101390 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.768+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa238069180 con 0x7fa238101390 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.770+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa22c00f450 con 0x7fa238101390 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.770+0000 7fa235ffb700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 0x7fa2280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.771+0000 7fa237fff700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 0x7fa2280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.771+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa22c0cecb0 con 0x7fa238101390 2024-06-06T07:43:24.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.771+0000 7fa237fff700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 0x7fa2280a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2380788b0 tx=0x7fa22400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:24.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:24.774+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa22c014030 con 0x7fa238101390 2024-06-06T07:43:25.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.012+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2381045a0 con 0x7fa238101390 2024-06-06T07:43:25.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.013+0000 7fa235ffb700 1 -- 172.21.15.87:0/3476963390 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7fa22c09c140 con 0x7fa238101390 2024-06-06T07:43:25.013 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:25.013 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T07:43:25.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 msgr2=0x7fa2280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:25.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 0x7fa2280a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2380788b0 tx=0x7fa22400b3c0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 msgr2=0x7fa238103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c9b0 tx=0x7fa22c00ccc0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 shutdown_connections 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa2280a2c20 0x7fa2280a50e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa238107400 0x7fa238103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa238101390 0x7fa238103070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 --2- 172.21.15.87:0/3476963390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa238101d50 0x7fa2381035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 >> 172.21.15.87:0/3476963390 conn(0x7fa238078340 msgr2=0x7fa238100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 shutdown_connections 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:25.015+0000 7fa23f241700 1 -- 172.21.15.87:0/3476963390 wait complete. 2024-06-06T07:43:25.016 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 5 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:25.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:25 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3476963390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:25.378 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:25.378 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.378 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:25.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:25.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:25.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:25 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3476963390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:25.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:25 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3476963390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:26 smithi195 ceph-mon[37040]: Deploying daemon mds.f on smithi195 2024-06-06T07:43:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:26 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6834/1343084853,v1:172.21.15.181:6835/1343084853] up:boot 2024-06-06T07:43:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:26 smithi195 ceph-mon[37040]: fsmap 5 up:standby 2024-06-06T07:43:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:26 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:43:26.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:26 smithi181 ceph-mon[36790]: Deploying daemon mds.f on smithi195 2024-06-06T07:43:26.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:26 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6834/1343084853,v1:172.21.15.181:6835/1343084853] up:boot 2024-06-06T07:43:26.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:26 smithi181 ceph-mon[36790]: fsmap 5 up:standby 2024-06-06T07:43:26.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:26 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:43:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:26 smithi087 ceph-mon[32276]: Deploying daemon mds.f on smithi195 2024-06-06T07:43:26.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:26 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6834/1343084853,v1:172.21.15.181:6835/1343084853] up:boot 2024-06-06T07:43:26.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:26 smithi087 ceph-mon[32276]: fsmap 5 up:standby 2024-06-06T07:43:26.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:26 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:43:26.727 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:26 smithi195 systemd[1]: Starting Ceph mds.f for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:26.981 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:26 smithi195 podman[67647]: 2024-06-06 07:43:26.630130388 +0000 UTC m=+0.016908276 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:26.981 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:26 smithi195 podman[67647]: 2024-06-06 07:43:26.736516624 +0000 UTC m=+0.123294518 container create e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T07:43:26.981 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:26 smithi195 podman[67647]: 2024-06-06 07:43:26.981764406 +0000 UTC m=+0.368542281 container init e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T07:43:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:26 smithi195 podman[67647]: 2024-06-06 07:43:26.984414479 +0000 UTC m=+0.371192355 container start e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-mds[67666]: -- 172.21.15.195:0/2323613387 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9db65fa40 con 0x55d9db632800 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: starting mds.f at 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:27 smithi195 bash[67647]: e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 2024-06-06T07:43:27.261 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 07:43:27 smithi195 systemd[1]: Started Ceph mds.f for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:27.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:27 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:27.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:27.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:27 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:27.680 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:27.929 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:28.331 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:28 smithi195 ceph-mon[37040]: Deploying daemon mds.c on smithi195 2024-06-06T07:43:28.331 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:28 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6834/922595027,v1:172.21.15.195:6835/922595027] up:boot 2024-06-06T07:43:28.331 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:28 smithi195 ceph-mon[37040]: fsmap 6 up:standby 2024-06-06T07:43:28.331 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:28 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:43:28.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:28 smithi087 ceph-mon[32276]: Deploying daemon mds.c on smithi195 2024-06-06T07:43:28.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:28 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6834/922595027,v1:172.21.15.195:6835/922595027] up:boot 2024-06-06T07:43:28.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:28 smithi087 ceph-mon[32276]: fsmap 6 up:standby 2024-06-06T07:43:28.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:28 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:43:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:28 smithi181 ceph-mon[36790]: Deploying daemon mds.c on smithi195 2024-06-06T07:43:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:28 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6834/922595027,v1:172.21.15.195:6835/922595027] up:boot 2024-06-06T07:43:28.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:28 smithi181 ceph-mon[36790]: fsmap 6 up:standby 2024-06-06T07:43:28.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:28 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:43:29.351 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:29 smithi087 ceph-mon[32276]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:29.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:29 smithi195 ceph-mon[37040]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:29 smithi181 ceph-mon[36790]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 -- 172.21.15.87:0/4157107669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 msgr2=0x7fd40c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 --2- 172.21.15.87:0/4157107669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c1012b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd400009990 tx=0x7fd400009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 -- 172.21.15.87:0/4157107669 shutdown_connections 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 --2- 172.21.15.87:0/4157107669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c1017f0 0x7fd40c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 --2- 172.21.15.87:0/4157107669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c107550 0x7fd40c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 --2- 172.21.15.87:0/4157107669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c1012b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 -- 172.21.15.87:0/4157107669 >> 172.21.15.87:0/4157107669 conn(0x7fd40c0766b0 msgr2=0x7fd40c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:29.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 -- 172.21.15.87:0/4157107669 shutdown_connections 2024-06-06T07:43:29.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.602+0000 7fd4111dd700 1 -- 172.21.15.87:0/4157107669 wait complete. 2024-06-06T07:43:29.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.603+0000 7fd4111dd700 1 Processor -- start 2024-06-06T07:43:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.603+0000 7fd4111dd700 1 -- start start 2024-06-06T07:43:29.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c1017f0 0x7fd40c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c107550 0x7fd40c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c112830 con 0x7fd40c100e30 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1126b0 con 0x7fd40c107550 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd4111dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1129b0 con 0x7fd40c1017f0 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60684/0 (socket says 172.21.15.87:60684) 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 -- 172.21.15.87:0/692627899 learned_addr learned my addr 172.21.15.87:0/692627899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40a59c700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c1017f0 0x7fd40c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40b59e700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c107550 0x7fd40c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 -- 172.21.15.87:0/692627899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c1017f0 msgr2=0x7fd40c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c1017f0 0x7fd40c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 -- 172.21.15.87:0/692627899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c107550 msgr2=0x7fd40c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:29.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c107550 0x7fd40c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 -- 172.21.15.87:0/692627899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c199130 con 0x7fd40c100e30 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.604+0000 7fd40ad9d700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c990 tx=0x7fd3fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.605+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc007b00 con 0x7fd40c100e30 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.605+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3fc007ca0 con 0x7fd40c100e30 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.605+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c199420 con 0x7fd40c100e30 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.605+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc018780 con 0x7fd40c100e30 2024-06-06T07:43:29.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.605+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a5760 con 0x7fd40c100e30 2024-06-06T07:43:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.606+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c069180 con 0x7fd40c100e30 2024-06-06T07:43:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.607+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3fc00f450 con 0x7fd40c100e30 2024-06-06T07:43:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.608+0000 7fd3f3fff700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 0x7fd3f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.608+0000 7fd40a59c700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 0x7fd3f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:29.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.608+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd3fc0cece0 con 0x7fd40c100e30 2024-06-06T07:43:29.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.608+0000 7fd40a59c700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 0x7fd3f40a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd400006010 tx=0x7fd40000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:29.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.612+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3fc014030 con 0x7fd40c100e30 2024-06-06T07:43:29.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.850+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd40c199f00 con 0x7fd40c100e30 2024-06-06T07:43:29.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.851+0000 7fd3f3fff700 1 -- 172.21.15.87:0/692627899 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7fd3fc09c1d0 con 0x7fd40c100e30 2024-06-06T07:43:29.851 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:29.852 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T07:43:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 msgr2=0x7fd3f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 0x7fd3f40a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd400006010 tx=0x7fd40000b540 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 msgr2=0x7fd40c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c990 tx=0x7fd3fc00cd50 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 shutdown_connections 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd3f40a2ca0 0x7fd3f40a5160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd40c107550 0x7fd40c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd40c100e30 0x7fd40c19e810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 --2- 172.21.15.87:0/692627899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd40c1017f0 0x7fd40c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.853+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 >> 172.21.15.87:0/692627899 conn(0x7fd40c0766b0 msgr2=0x7fd40c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.854+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 shutdown_connections 2024-06-06T07:43:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:29.854+0000 7fd4111dd700 1 -- 172.21.15.87:0/692627899 wait complete. 2024-06-06T07:43:29.855 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 7 2024-06-06T07:43:30.210 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:43:29 smithi195 systemd[1]: Started Ceph mds.c for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/692627899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:30.447 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:30 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/692627899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:30 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/692627899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:30.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:30.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:30.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:30 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:31 smithi195 ceph-mon[37040]: Deploying daemon mds.h on smithi181 2024-06-06T07:43:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:31 smithi195 ceph-mon[37040]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:31 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/494555841,v1:172.21.15.195:6837/494555841] up:boot 2024-06-06T07:43:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:31 smithi195 ceph-mon[37040]: fsmap 7 up:standby 2024-06-06T07:43:31.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:31 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:43:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:31 smithi181 ceph-mon[36790]: Deploying daemon mds.h on smithi181 2024-06-06T07:43:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:31 smithi181 ceph-mon[36790]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:31 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/494555841,v1:172.21.15.195:6837/494555841] up:boot 2024-06-06T07:43:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:31 smithi181 ceph-mon[36790]: fsmap 7 up:standby 2024-06-06T07:43:31.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:31 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:43:31.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:31 smithi087 ceph-mon[32276]: Deploying daemon mds.h on smithi181 2024-06-06T07:43:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:31 smithi087 ceph-mon[32276]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:31 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/494555841,v1:172.21.15.195:6837/494555841] up:boot 2024-06-06T07:43:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:31 smithi087 ceph-mon[32276]: fsmap 7 up:standby 2024-06-06T07:43:31.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:31 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:43:32.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:32.528 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 systemd[1]: Starting Ceph mds.h for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:32.720 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:32.779 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 podman[67030]: 2024-06-06 07:43:32.431427421 +0000 UTC m=+0.017830273 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:32.779 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 podman[67030]: 2024-06-06 07:43:32.534874316 +0000 UTC m=+0.121277197 container create f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:43:32.779 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 podman[67030]: 2024-06-06 07:43:32.779508907 +0000 UTC m=+0.365911740 container init f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:43:33.056 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 podman[67030]: 2024-06-06 07:43:32.782164097 +0000 UTC m=+0.368566946 container start f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T07:43:33.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 ceph-mds[67049]: -- 172.21.15.181:0/3245233960 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d977107a40 con 0x55d9770db000 2024-06-06T07:43:33.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: starting mds.h at 2024-06-06T07:43:33.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 bash[67030]: f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf 2024-06-06T07:43:33.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:43:32 smithi181 systemd[1]: Started Ceph mds.h for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:33.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:33.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:33 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:33.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:33.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:33 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:33.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:33 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 -- 172.21.15.87:0/1763123884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5460101620 msgr2=0x7f5460101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 --2- 172.21.15.87:0/1763123884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5460101620 0x7f5460101a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54540099c0 tx=0x7f5454009cd0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 -- 172.21.15.87:0/1763123884 shutdown_connections 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 --2- 172.21.15.87:0/1763123884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54601066b0 0x7f546010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 --2- 172.21.15.87:0/1763123884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5460101620 0x7f5460101a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 --2- 172.21.15.87:0/1763123884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101fd0 0x7f5460105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 -- 172.21.15.87:0/1763123884 >> 172.21.15.87:0/1763123884 conn(0x7f5460078360 msgr2=0x7f5460078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:34.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 -- 172.21.15.87:0/1763123884 shutdown_connections 2024-06-06T07:43:34.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.466+0000 7f5465afc700 1 -- 172.21.15.87:0/1763123884 wait complete. 2024-06-06T07:43:34.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 Processor -- start 2024-06-06T07:43:34.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 -- start start 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101620 0x7f5460197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 0x7f5460198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54601116c0 con 0x7f54601066b0 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5460111540 con 0x7f5460101620 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f5465afc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5460111840 con 0x7f5460101fd0 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f545ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f545ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60692/0 (socket says 172.21.15.87:60692) 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.467+0000 7f545ffff700 1 -- 172.21.15.87:0/3471856586 learned_addr learned my addr 172.21.15.87:0/3471856586 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:34.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545effd700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 0x7f5460198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545f7fe700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101620 0x7f5460197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 -- 172.21.15.87:0/3471856586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 msgr2=0x7f5460198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 0x7f5460198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 -- 172.21.15.87:0/3471856586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101620 msgr2=0x7f5460197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101620 0x7f5460197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 -- 172.21.15.87:0/3471856586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546019cb50 con 0x7f54601066b0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545effd700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 0x7f5460198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545ffff700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f545800ef30 tx=0x7f545800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580092e0 con 0x7f54601066b0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5458004bb0 con 0x7f54601066b0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546019cde0 con 0x7f54601066b0 2024-06-06T07:43:34.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458004280 con 0x7f54601066b0 2024-06-06T07:43:34.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.468+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601a4700 con 0x7f54601066b0 2024-06-06T07:43:34.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.469+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5460069180 con 0x7f54601066b0 2024-06-06T07:43:34.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.470+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f545801b030 con 0x7f54601066b0 2024-06-06T07:43:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.471+0000 7f545cff9700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 0x7f54480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.471+0000 7f545f7fe700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 0x7f54480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.471+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f54580ce710 con 0x7f54601066b0 2024-06-06T07:43:34.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.471+0000 7f545f7fe700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 0x7f54480a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5454000c00 tx=0x7f545401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:34.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.475+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5458022020 con 0x7f54601066b0 2024-06-06T07:43:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:34 smithi195 ceph-mon[37040]: Deploying daemon mds.i on smithi195 2024-06-06T07:43:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:34 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:boot 2024-06-06T07:43:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:34 smithi195 ceph-mon[37040]: fsmap 8 up:standby 2024-06-06T07:43:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:34 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:43:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:34 smithi181 ceph-mon[36790]: Deploying daemon mds.i on smithi195 2024-06-06T07:43:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:34 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:boot 2024-06-06T07:43:34.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:34 smithi181 ceph-mon[36790]: fsmap 8 up:standby 2024-06-06T07:43:34.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:34 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:43:34.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:34 smithi087 ceph-mon[32276]: Deploying daemon mds.i on smithi195 2024-06-06T07:43:34.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:34 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:boot 2024-06-06T07:43:34.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:34 smithi087 ceph-mon[32276]: fsmap 8 up:standby 2024-06-06T07:43:34.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:34 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:43:34.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.716+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f546002dca0 con 0x7f54601066b0 2024-06-06T07:43:34.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.716+0000 7f545cff9700 1 -- 172.21.15.87:0/3471856586 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f5458010040 con 0x7f54601066b0 2024-06-06T07:43:34.718 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:34.718 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 msgr2=0x7f54480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 0x7f54480a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5454000c00 tx=0x7f545401a040 comp rx=0 tx=0).stop 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 msgr2=0x7f546019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f545800ef30 tx=0x7f545800c450 comp rx=0 tx=0).stop 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 shutdown_connections 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f54480a2c20 0x7f54480a50e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5460101620 0x7f5460197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54601066b0 0x7f546019c450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 --2- 172.21.15.87:0/3471856586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5460101fd0 0x7f5460198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.718+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 >> 172.21.15.87:0/3471856586 conn(0x7f5460078360 msgr2=0x7f54600ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.719+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 shutdown_connections 2024-06-06T07:43:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:34.719+0000 7f5465afc700 1 -- 172.21.15.87:0/3471856586 wait complete. 2024-06-06T07:43:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 9 2024-06-06T07:43:35.229 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 systemd[1]: Starting Ceph mds.i for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:35.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:35 smithi087 ceph-mon[32276]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:35.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:35 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3471856586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:35.497 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:35 smithi195 ceph-mon[37040]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:35.497 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:35 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3471856586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:35.497 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 podman[68260]: 2024-06-06 07:43:35.254478416 +0000 UTC m=+0.017467930 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:35.497 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 podman[68260]: 2024-06-06 07:43:35.376013493 +0000 UTC m=+0.139002997 container create c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:43:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:35 smithi181 ceph-mon[36790]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:35 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3471856586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 podman[68260]: 2024-06-06 07:43:35.636908367 +0000 UTC m=+0.399897883 container init c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 podman[68260]: 2024-06-06 07:43:35.639563676 +0000 UTC m=+0.402553182 container start c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 ceph-mds[68279]: -- 172.21.15.195:0/3196485901 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55709524da40 con 0x557095220800 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: starting mds.i at 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 bash[68260]: c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc 2024-06-06T07:43:35.760 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 07:43:35 smithi195 systemd[1]: Started Ceph mds.i for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: Deploying daemon mds.g on smithi087 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:37.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:boot 2024-06-06T07:43:37.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: fsmap 9 up:standby 2024-06-06T07:43:37.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:36 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:43:37.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: Deploying daemon mds.g on smithi087 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:boot 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: fsmap 9 up:standby 2024-06-06T07:43:37.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:36 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:43:37.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: Deploying daemon mds.g on smithi087 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:boot 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: fsmap 9 up:standby 2024-06-06T07:43:37.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:36 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:43:37.431 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:37.692 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:39.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:39 smithi087 ceph-mon[32276]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:39 smithi195 ceph-mon[37040]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:39 smithi181 ceph-mon[36790]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 -- 172.21.15.87:0/3138427890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 msgr2=0x7ffabc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 --2- 172.21.15.87:0/3138427890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc101800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009990 tx=0x7ffab0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 -- 172.21.15.87:0/3138427890 shutdown_connections 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 --2- 172.21.15.87:0/3138427890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1073f0 0x7ffabc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 --2- 172.21.15.87:0/3138427890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc101d40 0x7ffabc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 --2- 172.21.15.87:0/3138427890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc101800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 -- 172.21.15.87:0/3138427890 >> 172.21.15.87:0/3138427890 conn(0x7ffabc078300 msgr2=0x7ffabc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 -- 172.21.15.87:0/3138427890 shutdown_connections 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.484+0000 7ffac23fb700 1 -- 172.21.15.87:0/3138427890 wait complete. 2024-06-06T07:43:39.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 Processor -- start 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 -- start start 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc101d40 0x7ffabc1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc1073f0 0x7ffabc103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc111660 con 0x7ffabc101380 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1114e0 con 0x7ffabc101d40 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffac23fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1117e0 con 0x7ffabc1073f0 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38284/0 (socket says 172.21.15.87:38284) 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.485+0000 7ffabbfff700 1 -- 172.21.15.87:0/2119366669 learned_addr learned my addr 172.21.15.87:0/2119366669 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabb7fe700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc101d40 0x7ffabc1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:39.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffac0998700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc1073f0 0x7ffabc103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 -- 172.21.15.87:0/2119366669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc1073f0 msgr2=0x7ffabc103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc1073f0 0x7ffabc103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 -- 172.21.15.87:0/2119366669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc101d40 msgr2=0x7ffabc1035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc101d40 0x7ffabc1035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 -- 172.21.15.87:0/2119366669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a7110 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffabbfff700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c920 tx=0x7ffaac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac007b00 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a7400 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac004d10 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005630 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.486+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a7960 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.487+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc069180 con 0x7ffabc101380 2024-06-06T07:43:39.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.488+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffaac004830 con 0x7ffabc101380 2024-06-06T07:43:39.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.489+0000 7ffab97fa700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 0x7ffaa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:39.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.489+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffaac0ce710 con 0x7ffabc101380 2024-06-06T07:43:39.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.489+0000 7ffabb7fe700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 0x7ffaa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:39.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.490+0000 7ffabb7fe700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 0x7ffaa40a50e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b5c0 tx=0x7ffab0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:39.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.494+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffaac014030 con 0x7ffabc101380 2024-06-06T07:43:39.728 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 07:43:39 smithi087 systemd[1]: Started Ceph mds.g for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:39.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.744+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffabc1a7bf0 con 0x7ffabc101380 2024-06-06T07:43:39.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.744+0000 7ffab97fa700 1 -- 172.21.15.87:0/2119366669 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7ffaac09bb10 con 0x7ffabc101380 2024-06-06T07:43:39.745 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:39.745 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 msgr2=0x7ffaa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 0x7ffaa40a50e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b5c0 tx=0x7ffab0005960 comp rx=0 tx=0).stop 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 msgr2=0x7ffabc103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c920 tx=0x7ffaac00cc30 comp rx=0 tx=0).stop 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 shutdown_connections 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ffaa40a2c20 0x7ffaa40a50e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc1073f0 0x7ffabc103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffabc101380 0x7ffabc103060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 --2- 172.21.15.87:0/2119366669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc101d40 0x7ffabc1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 >> 172.21.15.87:0/2119366669 conn(0x7ffabc078300 msgr2=0x7ffabc100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 shutdown_connections 2024-06-06T07:43:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:39.747+0000 7ffac23fb700 1 -- 172.21.15.87:0/2119366669 wait complete. 2024-06-06T07:43:39.748 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 10 2024-06-06T07:43:40.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:40.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:40.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:40.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:40.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: Deploying daemon mds.a on smithi087 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2119366669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.87:6838/200596410,v1:172.21.15.87:6839/200596410] up:boot 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: fsmap 10 up:standby 2024-06-06T07:43:40.728 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:40 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:43:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: Deploying daemon mds.a on smithi087 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2119366669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.87:6838/200596410,v1:172.21.15.87:6839/200596410] up:boot 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: fsmap 10 up:standby 2024-06-06T07:43:41.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:40 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:43:41.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:41.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: Deploying daemon mds.a on smithi087 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2119366669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.87:6838/200596410,v1:172.21.15.87:6839/200596410] up:boot 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: fsmap 10 up:standby 2024-06-06T07:43:41.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:40 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:43:42.360 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 systemd[1]: Starting Ceph mds.a for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:42.465 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:42.714 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 podman[71488]: 2024-06-06 07:43:42.322490307 +0000 UTC m=+0.016790369 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:42.715 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 podman[71488]: 2024-06-06 07:43:42.437227669 +0000 UTC m=+0.131527716 container create 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-06T07:43:42.715 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 podman[71488]: 2024-06-06 07:43:42.714853864 +0000 UTC m=+0.409153915 container init 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-06T07:43:42.904 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:43.027 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 podman[71488]: 2024-06-06 07:43:42.71815484 +0000 UTC m=+0.412454889 container start 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T07:43:43.027 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 ceph-mds[71537]: -- 172.21.15.87:0/2549761704 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56329cc4da40 con 0x56329cc20800 2024-06-06T07:43:43.027 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a[71521]: starting mds.a at 2024-06-06T07:43:43.027 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 bash[71488]: 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 2024-06-06T07:43:43.027 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 07:43:42 smithi087 systemd[1]: Started Ceph mds.a for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:43.300 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:43.300 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.301 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.301 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.301 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:43.301 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:43.301 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:43 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:43.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:43.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:43 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:43.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:43.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.527 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.527 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:43.527 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T07:43:43.527 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T07:43:43.527 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:43 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:44.340 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:44 smithi087 ceph-mon[32276]: Deploying daemon mds.b on smithi181 2024-06-06T07:43:44.340 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:44 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.87:6840/3223638025,v1:172.21.15.87:6841/3223638025] up:boot 2024-06-06T07:43:44.341 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:44 smithi087 ceph-mon[32276]: fsmap 11 up:standby 2024-06-06T07:43:44.341 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:44 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:43:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:44 smithi195 ceph-mon[37040]: Deploying daemon mds.b on smithi181 2024-06-06T07:43:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:44 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.87:6840/3223638025,v1:172.21.15.87:6841/3223638025] up:boot 2024-06-06T07:43:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:44 smithi195 ceph-mon[37040]: fsmap 11 up:standby 2024-06-06T07:43:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:44 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:43:44.550 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:44 smithi181 ceph-mon[36790]: Deploying daemon mds.b on smithi181 2024-06-06T07:43:44.550 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:44 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.87:6840/3223638025,v1:172.21.15.87:6841/3223638025] up:boot 2024-06-06T07:43:44.550 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:44 smithi181 ceph-mon[36790]: fsmap 11 up:standby 2024-06-06T07:43:44.550 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:44 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:43:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 -- 172.21.15.87:0/1404064585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa01027a0 msgr2=0x7f7aa0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1404064585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa01027a0 0x7f7aa0102b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a940099c0 tx=0x7f7a94009cd0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 -- 172.21.15.87:0/1404064585 shutdown_connections 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1404064585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa0107870 0x7f7aa010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1404064585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa01027a0 0x7f7aa0102b80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1404064585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 -- 172.21.15.87:0/1404064585 >> 172.21.15.87:0/1404064585 conn(0x7f7aa0079520 msgr2=0x7f7aa0079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 -- 172.21.15.87:0/1404064585 shutdown_connections 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 -- 172.21.15.87:0/1404064585 wait complete. 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.588+0000 7f7aa5229700 1 Processor -- start 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 -- start start 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa01027a0 0x7f7aa0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0112800 con 0x7f7aa0107870 2024-06-06T07:43:44.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0112680 con 0x7f7aa01027a0 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7aa5229700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0112980 con 0x7f7aa0103150 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7a9f59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7a9e59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7a9f59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38318/0 (socket says 172.21.15.87:38318) 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7a9f59e700 1 -- 172.21.15.87:0/1187743376 learned_addr learned my addr 172.21.15.87:0/1187743376 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:44.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.589+0000 7f7a9ed9d700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa01027a0 0x7f7aa0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 -- 172.21.15.87:0/1187743376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 msgr2=0x7f7aa0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 -- 172.21.15.87:0/1187743376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa01027a0 msgr2=0x7f7aa0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa01027a0 0x7f7aa0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 -- 172.21.15.87:0/1187743376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa01a70e0 con 0x7f7aa0107870 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9e59c700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a9f59e700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a98007d70 tx=0x7f7a9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98016020 con 0x7f7aa0107870 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa01a73d0 con 0x7f7aa0107870 2024-06-06T07:43:44.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9800ac90 con 0x7f7aa0107870 2024-06-06T07:43:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a980115e0 con 0x7f7aa0107870 2024-06-06T07:43:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.590+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa01a7960 con 0x7f7aa0107870 2024-06-06T07:43:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.591+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa0069180 con 0x7f7aa0107870 2024-06-06T07:43:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.592+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a98019020 con 0x7f7aa0107870 2024-06-06T07:43:44.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.593+0000 7f7a87fff700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 0x7f7a880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.593+0000 7f7a9ed9d700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 0x7f7a880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.593+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7a980cee80 con 0x7f7aa0107870 2024-06-06T07:43:44.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.593+0000 7f7a9ed9d700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 0x7f7a880a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a94000c00 tx=0x7f7a94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:44.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.597+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a98020020 con 0x7f7aa0107870 2024-06-06T07:43:44.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.844+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aa01a7d20 con 0x7f7aa0107870 2024-06-06T07:43:44.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.845+0000 7f7a87fff700 1 -- 172.21.15.87:0/1187743376 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f7a9809c280 con 0x7f7aa0107870 2024-06-06T07:43:44.846 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:44.847 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 msgr2=0x7f7a880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 0x7f7a880a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a94000c00 tx=0x7f7a94005fb0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 msgr2=0x7f7aa0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a98007d70 tx=0x7f7a9800a340 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 shutdown_connections 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7a880a2c20 0x7f7a880a50e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa01027a0 0x7f7aa0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aa0107870 0x7f7aa0075210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 --2- 172.21.15.87:0/1187743376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa0103150 0x7f7aa0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:44.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 >> 172.21.15.87:0/1187743376 conn(0x7f7aa0079520 msgr2=0x7f7aa01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:44.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 shutdown_connections 2024-06-06T07:43:44.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:44.848+0000 7f7aa5229700 1 -- 172.21.15.87:0/1187743376 wait complete. 2024-06-06T07:43:44.850 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 12 2024-06-06T07:43:45.197 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 systemd[1]: Starting Ceph mds.b for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T07:43:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:45 smithi195 ceph-mon[37040]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1187743376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:45.542 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:45 smithi181 ceph-mon[36790]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:45.542 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1187743376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:45.543 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 podman[67333]: 2024-06-06 07:43:45.299238939 +0000 UTC m=+0.104799766 container create b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T07:43:45.543 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 podman[67333]: 2024-06-06 07:43:45.21139214 +0000 UTC m=+0.016952980 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:43:45.543 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 podman[67333]: 2024-06-06 07:43:45.54341673 +0000 UTC m=+0.348977558 container init b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-06T07:43:45.544 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:45 smithi087 ceph-mon[32276]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:45.544 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1187743376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:45.806 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 podman[67333]: 2024-06-06 07:43:45.546854276 +0000 UTC m=+0.352415106 container start b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:43:45.806 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 ceph-mds[67353]: -- 172.21.15.181:0/950422870 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9951e1a40 con 0x55f9951b4800 2024-06-06T07:43:45.807 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: starting mds.b at 2024-06-06T07:43:45.807 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 bash[67333]: b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 2024-06-06T07:43:45.807 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:43:45 smithi181 systemd[1]: Started Ceph mds.b for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:46.949 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.950 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:46.950 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:boot 2024-06-06T07:43:46.950 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: fsmap 12 up:standby 2024-06-06T07:43:46.950 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:46.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:boot 2024-06-06T07:43:46.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: fsmap 12 up:standby 2024-06-06T07:43:46.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:43:47.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:boot 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: fsmap 12 up:standby 2024-06-06T07:43:47.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:43:47.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:43:47.816 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:48.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:48.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:48.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:48.862 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:49.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:49.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:49.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:43:49.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 -- 172.21.15.87:0/2533449264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5164108500 msgr2=0x7f51641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2533449264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5164108500 0x7f51641088e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5154009990 tx=0x7f5154009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 -- 172.21.15.87:0/2533449264 shutdown_connections 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2533449264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406b580 0x7f516410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2533449264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5164108500 0x7f51641088e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2533449264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516406abc0 0x7f516406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 -- 172.21.15.87:0/2533449264 >> 172.21.15.87:0/2533449264 conn(0x7f5164078340 msgr2=0x7f5164078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.517+0000 7f5168aa5700 1 -- 172.21.15.87:0/2533449264 shutdown_connections 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.518+0000 7f5168aa5700 1 -- 172.21.15.87:0/2533449264 wait complete. 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.518+0000 7f5168aa5700 1 Processor -- start 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.518+0000 7f5168aa5700 1 -- start start 2024-06-06T07:43:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406abc0 0x7f5164101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5164108500 0x7f5164101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112780 con 0x7f516406b580 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112600 con 0x7f5164108500 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112900 con 0x7f516406abc0 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49802/0 (socket says 172.21.15.87:49802) 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f516259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406abc0 0x7f5164101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 -- 172.21.15.87:0/2266475635 learned_addr learned my addr 172.21.15.87:0/2266475635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5162d9d700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5164108500 0x7f5164101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 -- 172.21.15.87:0/2266475635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406abc0 msgr2=0x7f5164101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406abc0 0x7f5164101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 -- 172.21.15.87:0/2266475635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5164108500 msgr2=0x7f5164101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5164108500 0x7f5164101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 -- 172.21.15.87:0/2266475635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641a71c0 con 0x7f516406b580 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5161d9b700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f515800d950 tx=0x7f515800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158016070 con 0x7f516406b580 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515800fd70 con 0x7f516406b580 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641a74b0 con 0x7f516406b580 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515800e610 con 0x7f516406b580 2024-06-06T07:43:49.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.519+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a79f0 con 0x7f516406b580 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.520+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516410a9e0 con 0x7f516406b580 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.521+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5158019070 con 0x7f516406b580 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.522+0000 7f51537fe700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 0x7f514c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.522+0000 7f516259c700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 0x7f514c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.522+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51580ce650 con 0x7f516406b580 2024-06-06T07:43:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.523+0000 7f516259c700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 0x7f514c0a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f515400b5c0 tx=0x7f5154005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:49.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.527+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f515809ba50 con 0x7f516406b580 2024-06-06T07:43:49.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.775+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51641029c0 con 0x7f516406b580 2024-06-06T07:43:49.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.775+0000 7f51537fe700 1 -- 172.21.15.87:0/2266475635 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f515809b350 con 0x7f516406b580 2024-06-06T07:43:49.776 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:49.776 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.777+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 msgr2=0x7f514c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.777+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 0x7f514c0a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f515400b5c0 tx=0x7f5154005960 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.777+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 msgr2=0x7f51641019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.777+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f515800d950 tx=0x7f515800dc60 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 shutdown_connections 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f514c0a2c20 0x7f514c0a50e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5164108500 0x7f5164101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f516406abc0 0x7f5164101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 --2- 172.21.15.87:0/2266475635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f516406b580 0x7f51641019d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:49.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 >> 172.21.15.87:0/2266475635 conn(0x7f5164078340 msgr2=0x7f5164100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 shutdown_connections 2024-06-06T07:43:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:49.778+0000 7f5168aa5700 1 -- 172.21.15.87:0/2266475635 wait complete. 2024-06-06T07:43:49.779 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 13 2024-06-06T07:43:50.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:49 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2266475635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:50.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:49 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2266475635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:49 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2266475635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:43:50.618 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-06T07:43:51.095 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:51.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:50 smithi087 ceph-mon[32276]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:50 smithi195 ceph-mon[37040]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:51.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:51.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:50 smithi181 ceph-mon[36790]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 -- 172.21.15.87:0/3955694326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64107430 msgr2=0x7fae64107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3955694326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64107430 0x7fae64107810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae58009990 tx=0x7fae58009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 -- 172.21.15.87:0/3955694326 shutdown_connections 2024-06-06T07:43:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3955694326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae64101d80 0x7fae6410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3955694326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64107430 0x7fae64107810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3955694326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae641013c0 0x7fae64101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 -- 172.21.15.87:0/3955694326 >> 172.21.15.87:0/3955694326 conn(0x7fae640783d0 msgr2=0x7fae640787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 -- 172.21.15.87:0/3955694326 shutdown_connections 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.811+0000 7fae6aaee700 1 -- 172.21.15.87:0/3955694326 wait complete. 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 Processor -- start 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 -- start start 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:52.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae64101d80 0x7fae6419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 0x7fae64198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae641117a0 con 0x7fae641013c0 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111620 con 0x7fae64107430 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6aaee700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111920 con 0x7fae64101d80 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49816/0 (socket says 172.21.15.87:49816) 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 -- 172.21.15.87:0/3559351551 learned_addr learned my addr 172.21.15.87:0/3559351551 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 -- 172.21.15.87:0/3559351551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae64101d80 msgr2=0x7fae6419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae64101d80 0x7fae6419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 -- 172.21.15.87:0/3559351551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 msgr2=0x7fae64198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6908b700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 0x7fae64198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 0x7fae64198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6888a700 1 -- 172.21.15.87:0/3559351551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64199410 con 0x7fae641013c0 2024-06-06T07:43:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.812+0000 7fae6908b700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 0x7fae64198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae6888a700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58027070 con 0x7fae641013c0 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae58004190 con 0x7fae641013c0 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5800f460 con 0x7fae641013c0 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae641996a0 con 0x7fae641013c0 2024-06-06T07:43:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.813+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a59e0 con 0x7fae641013c0 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.814+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64102ae0 con 0x7fae641013c0 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.815+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae5802a070 con 0x7fae641013c0 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.816+0000 7fae61ffb700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 0x7fae540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.816+0000 7fae63fff700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 0x7fae540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.816+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fae580cfe90 con 0x7fae641013c0 2024-06-06T07:43:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.816+0000 7fae63fff700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 0x7fae540a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae6419a280 tx=0x7fae50006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:52.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:52.819+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae58013070 con 0x7fae641013c0 2024-06-06T07:43:53.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.032+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fae64199ff0 con 0x7fae641013c0 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.032+0000 7fae61ffb700 1 -- 172.21.15.87:0/3559351551 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fae5809d320 con 0x7fae641013c0 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:43:53.033 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 msgr2=0x7fae540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 0x7fae540a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae6419a280 tx=0x7fae50006d20 comp rx=0 tx=0).stop 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 msgr2=0x7fae6419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58003730 comp rx=0 tx=0).stop 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 shutdown_connections 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fae540a2ca0 0x7fae540a5160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:53.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64107430 0x7fae64198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:53.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae641013c0 0x7fae6419ea60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:53.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 --2- 172.21.15.87:0/3559351551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae64101d80 0x7fae6419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:53.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 >> 172.21.15.87:0/3559351551 conn(0x7fae640783d0 msgr2=0x7fae64100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:53.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 shutdown_connections 2024-06-06T07:43:53.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:53.035+0000 7fae6aaee700 1 -- 172.21.15.87:0/3559351551 wait complete. 2024-06-06T07:43:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:53 smithi087 ceph-mon[32276]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:53 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:53 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3559351551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T07:43:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:53 smithi195 ceph-mon[37040]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:53 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:53 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3559351551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T07:43:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:53 smithi181 ceph-mon[36790]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:53 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:53.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:53 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3559351551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T07:43:53.792 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-06T07:43:53.792 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-06T07:43:54.042 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:54.426 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:54.426 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:54.426 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:54.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:43:54.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:43:54.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:43:55.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:55 smithi181 ceph-mon[36790]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:55.481 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:55 smithi087 ceph-mon[32276]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:55 smithi195 ceph-mon[37040]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:55.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 -- 172.21.15.87:0/2554587176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88108530 msgr2=0x7f6d88108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:55.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 --2- 172.21.15.87:0/2554587176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88108530 0x7f6d88108910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009990 tx=0x7f6d78009ca0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 -- 172.21.15.87:0/2554587176 shutdown_connections 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 --2- 172.21.15.87:0/2554587176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 0x7f6d8810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 --2- 172.21.15.87:0/2554587176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88108530 0x7f6d88108910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 --2- 172.21.15.87:0/2554587176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8806ac00 0x7f6d8806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 -- 172.21.15.87:0/2554587176 >> 172.21.15.87:0/2554587176 conn(0x7f6d88077400 msgr2=0x7f6d88077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 -- 172.21.15.87:0/2554587176 shutdown_connections 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 -- 172.21.15.87:0/2554587176 wait complete. 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.731+0000 7f6d8d757700 1 Processor -- start 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 -- start start 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 0x7f6d8819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88108530 0x7f6d8819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:55.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88112780 con 0x7f6d8806ac00 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88112600 con 0x7f6d8806b5c0 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d8d757700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88112900 con 0x7f6d88108530 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49834/0 (socket says 172.21.15.87:49834) 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 -- 172.21.15.87:0/4144566318 learned_addr learned my addr 172.21.15.87:0/4144566318 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d7e7fc700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 0x7f6d8819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d877fe700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88108530 0x7f6d8819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 -- 172.21.15.87:0/4144566318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88108530 msgr2=0x7f6d8819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88108530 0x7f6d8819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 -- 172.21.15.87:0/4144566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 msgr2=0x7f6d8819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 0x7f6d8819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.732+0000 7f6d86ffd700 1 -- 172.21.15.87:0/4144566318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8819fe30 con 0x7f6d8806ac00 2024-06-06T07:43:55.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d86ffd700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78027070 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d78003780 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d881a00c0 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78022dd0 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.733+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d881a79a0 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.734+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d88103bf0 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.735+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6d7802a070 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.735+0000 7f6d84ff9700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 0x7f6d740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.736+0000 7f6d7e7fc700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 0x7f6d740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.736+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6d780cf880 con 0x7f6d8806ac00 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.736+0000 7f6d7e7fc700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 0x7f6d740a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d881033f0 tx=0x7f6d7000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:55.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.740+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d78013070 con 0x7f6d8806ac00 2024-06-06T07:43:55.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:55.936+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d8819c180 con 0x7f6d8806ac00 2024-06-06T07:43:56.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.194+0000 7f6d84ff9700 1 -- 172.21.15.87:0/4144566318 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f6d7809cda0 con 0x7f6d8806ac00 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 msgr2=0x7f6d740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 0x7f6d740a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d881033f0 tx=0x7f6d7000b3c0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 msgr2=0x7f6d8819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d780047b0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 shutdown_connections 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d740a2c20 0x7f6d740a50e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88108530 0x7f6d8819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d8806ac00 0x7f6d8819ae20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 --2- 172.21.15.87:0/4144566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d8806b5c0 0x7f6d8819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 >> 172.21.15.87:0/4144566318 conn(0x7f6d88077400 msgr2=0x7f6d8810d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 shutdown_connections 2024-06-06T07:43:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:56.197+0000 7f6d8d757700 1 -- 172.21.15.87:0/4144566318 wait complete. 2024-06-06T07:43:56.198 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-06-06T07:43:56.468 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:56 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T07:43:56.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:56 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T07:43:56.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:56 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T07:43:56.892 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-06T07:43:57.131 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:43:57.456 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:57 smithi087 ceph-mon[32276]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:57.456 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T07:43:57.456 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:57 smithi087 ceph-mon[32276]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T07:43:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:57 smithi195 ceph-mon[37040]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T07:43:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:57 smithi195 ceph-mon[37040]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T07:43:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:57 smithi181 ceph-mon[36790]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4144566318' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T07:43:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:57 smithi181 ceph-mon[36790]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T07:43:58.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:58 smithi195 ceph-mon[37040]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T07:43:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:58 smithi181 ceph-mon[36790]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T07:43:58.572 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:58 smithi087 ceph-mon[32276]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T07:43:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 -- 172.21.15.87:0/1806123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 msgr2=0x7fb018107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1806123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb018107110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0080099c0 tx=0x7fb008009cd0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 -- 172.21.15.87:0/1806123722 shutdown_connections 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1806123722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018102740 0x7fb018102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1806123722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018107840 0x7fb01810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1806123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb018107110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 -- 172.21.15.87:0/1806123722 >> 172.21.15.87:0/1806123722 conn(0x7fb0180795f0 msgr2=0x7fb018079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 -- 172.21.15.87:0/1806123722 shutdown_connections 2024-06-06T07:43:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.822+0000 7fb0202ad700 1 -- 172.21.15.87:0/1806123722 wait complete. 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 Processor -- start 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 -- start start 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018102740 0x7fb01819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018107840 0x7fb0181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb018112770 con 0x7fb0181030f0 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181125f0 con 0x7fb018107840 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb0202ad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181128f0 con 0x7fb018102740 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42598/0 (socket says 172.21.15.87:42598) 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 -- 172.21.15.87:0/1329478471 learned_addr learned my addr 172.21.15.87:0/1329478471 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01e049700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018102740 0x7fb01819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01e84a700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018107840 0x7fb0181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 -- 172.21.15.87:0/1329478471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018102740 msgr2=0x7fb01819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018102740 0x7fb01819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 -- 172.21.15.87:0/1329478471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018107840 msgr2=0x7fb0181989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018107840 0x7fb0181989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.823+0000 7fb01d848700 1 -- 172.21.15.87:0/1329478471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb018199150 con 0x7fb0181030f0 2024-06-06T07:43:58.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb01d848700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0080059d0 tx=0x7fb00800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008027070 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181993e0 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb008003680 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008017650 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.824+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181a5700 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.825+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb018069180 con 0x7fb0181030f0 2024-06-06T07:43:58.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.827+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb00802a070 con 0x7fb0181030f0 2024-06-06T07:43:58.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.827+0000 7fb00f7fe700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 0x7fb0040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:43:58.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.828+0000 7fb01e049700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 0x7fb0040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:43:58.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.828+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fb0080cfd20 con 0x7fb0181030f0 2024-06-06T07:43:58.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.828+0000 7fb01e049700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 0x7fb0040a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb014005950 tx=0x7fb01400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:43:58.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:58.831+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb008031080 con 0x7fb0181030f0 2024-06-06T07:43:59.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.031+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb01802dca0 con 0x7fb0181030f0 2024-06-06T07:43:59.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.316+0000 7fb00f7fe700 1 -- 172.21.15.87:0/1329478471 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7fb008013070 con 0x7fb0181030f0 2024-06-06T07:43:59.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 msgr2=0x7fb0040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:59.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 0x7fb0040a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb014005950 tx=0x7fb01400a300 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 msgr2=0x7fb01819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0080059d0 tx=0x7fb00800fa00 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 shutdown_connections 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb0040a2ca0 0x7fb0040a5160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb018107840 0x7fb0181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018102740 0x7fb01819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 --2- 172.21.15.87:0/1329478471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0181030f0 0x7fb01819ed70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 >> 172.21.15.87:0/1329478471 conn(0x7fb0180795f0 msgr2=0x7fb018100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 shutdown_connections 2024-06-06T07:43:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:43:59.319+0000 7fb0202ad700 1 -- 172.21.15.87:0/1329478471 wait complete. 2024-06-06T07:43:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-06-06T07:43:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:59 smithi181 ceph-mon[36790]: pgmap v289: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:59 smithi181 ceph-mon[36790]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:43:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:43:59 smithi181 ceph-mon[36790]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T07:43:59.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:59 smithi087 ceph-mon[32276]: pgmap v289: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:59.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:59 smithi087 ceph-mon[32276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:43:59.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:43:59 smithi087 ceph-mon[32276]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T07:43:59.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:59 smithi195 ceph-mon[37040]: pgmap v289: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:43:59.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:59 smithi195 ceph-mon[37040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:43:59.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:43:59 smithi195 ceph-mon[37040]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T07:44:00.011 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-06T07:44:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:00.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T07:44:00.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T07:44:00.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:00 smithi087 ceph-mon[32276]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T07:44:00.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:00 smithi087 ceph-mon[32276]: pgmap v292: 97 pgs: 20 active+clean, 21 creating+peering, 56 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:00 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T07:44:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:00 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T07:44:00.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:00 smithi181 ceph-mon[36790]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T07:44:00.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:00 smithi181 ceph-mon[36790]: pgmap v292: 97 pgs: 20 active+clean, 21 creating+peering, 56 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:00 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T07:44:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:00 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1329478471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T07:44:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:00 smithi195 ceph-mon[37040]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T07:44:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:00 smithi195 ceph-mon[37040]: pgmap v292: 97 pgs: 20 active+clean, 21 creating+peering, 56 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:01.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:01 smithi087 ceph-mon[32276]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T07:44:01.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:01 smithi195 ceph-mon[37040]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T07:44:01.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:01 smithi181 ceph-mon[36790]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 -- 172.21.15.87:0/3204105927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806aac0 msgr2=0x7f855810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 --2- 172.21.15.87:0/3204105927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806aac0 0x7f855810ea00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8540009990 tx=0x7f8540009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 -- 172.21.15.87:0/3204105927 shutdown_connections 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 --2- 172.21.15.87:0/3204105927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855810ef40 0x7f8558111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 --2- 172.21.15.87:0/3204105927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f855806a1a0 0x7f855806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 --2- 172.21.15.87:0/3204105927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806aac0 0x7f855810ea00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 -- 172.21.15.87:0/3204105927 >> 172.21.15.87:0/3204105927 conn(0x7f8558078340 msgr2=0x7f8558078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 -- 172.21.15.87:0/3204105927 shutdown_connections 2024-06-06T07:44:01.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.891+0000 7f855d824700 1 -- 172.21.15.87:0/3204105927 wait complete. 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 Processor -- start 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 -- start start 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 0x7f855819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f855810ef40 0x7f855819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8558114a90 con 0x7f855806a1a0 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8558114910 con 0x7f855806aac0 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f855d824700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8558114c10 con 0x7f855810ef40 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f8556ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f8556ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42616/0 (socket says 172.21.15.87:42616) 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.892+0000 7f8556ffd700 1 -- 172.21.15.87:0/1684096805 learned_addr learned my addr 172.21.15.87:0/1684096805 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 -- 172.21.15.87:0/1684096805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f855810ef40 msgr2=0x7f855819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:01.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f855810ef40 0x7f855819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 -- 172.21.15.87:0/1684096805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 msgr2=0x7f855819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f854e7fc700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 0x7f855819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 0x7f855819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 -- 172.21.15.87:0/1684096805 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855819ff60 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f854e7fc700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 0x7f855819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f8556ffd700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f854800b530 tx=0x7f854800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548015020 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85581a0250 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8548011d10 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548010e50 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.893+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85581a7ad0 con 0x7f855806a1a0 2024-06-06T07:44:01.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.894+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8558069180 con 0x7f855806a1a0 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.895+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8548018020 con 0x7f855806a1a0 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.896+0000 7f855c822700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 0x7f85440a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.896+0000 7f854e7fc700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 0x7f85440a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.896+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f85480cece0 con 0x7f855806a1a0 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.896+0000 7f854e7fc700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 0x7f85440a5280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f854000b5c0 tx=0x7f8540005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:01.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:01.899+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f854801f020 con 0x7f855806a1a0 2024-06-06T07:44:02.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.138+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f855802dca0 con 0x7f855806a1a0 2024-06-06T07:44:02.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.328+0000 7f855c822700 1 -- 172.21.15.87:0/1684096805 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f854809bdf0 con 0x7f855806a1a0 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 msgr2=0x7f85440a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 0x7f85440a5280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f854000b5c0 tx=0x7f8540005fb0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 msgr2=0x7f855819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f854800b530 tx=0x7f854800b8f0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 shutdown_connections 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85440a2dc0 0x7f85440a5280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f855810ef40 0x7f855819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f855806a1a0 0x7f855819afa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 --2- 172.21.15.87:0/1684096805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855806aac0 0x7f855819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:02.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 >> 172.21.15.87:0/1684096805 conn(0x7f8558078340 msgr2=0x7f8558110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:02.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.331+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 shutdown_connections 2024-06-06T07:44:02.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:02.332+0000 7f855d824700 1 -- 172.21.15.87:0/1684096805 wait complete. 2024-06-06T07:44:02.333 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 2 and data pool 3 2024-06-06T07:44:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:02 smithi087 ceph-mon[32276]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T07:44:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:02 smithi087 ceph-mon[32276]: pgmap v295: 97 pgs: 49 active+clean, 20 creating+peering, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:02.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T07:44:02.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:02 smithi087 ceph-mon[32276]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T07:44:02.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:02 smithi195 ceph-mon[37040]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T07:44:02.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:02 smithi195 ceph-mon[37040]: pgmap v295: 97 pgs: 49 active+clean, 20 creating+peering, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:02.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T07:44:02.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:02 smithi195 ceph-mon[37040]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T07:44:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:02 smithi181 ceph-mon[36790]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T07:44:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:02 smithi181 ceph-mon[36790]: pgmap v295: 97 pgs: 49 active+clean, 20 creating+peering, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T07:44:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:02 smithi181 ceph-mon[36790]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T07:44:03.119 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-06T07:44:03.120 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-06T07:44:03.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T07:44:03.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:03 smithi087 ceph-mon[32276]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T07:44:03.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:03 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T07:44:03.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:03 smithi087 ceph-mon[32276]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T07:44:03.374 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T07:44:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:03 smithi195 ceph-mon[37040]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T07:44:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:03 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T07:44:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:03 smithi195 ceph-mon[37040]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T07:44:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1684096805' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T07:44:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:03 smithi181 ceph-mon[36790]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T07:44:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:03 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T07:44:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:03 smithi181 ceph-mon[36790]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T07:44:04.558 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:04 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:active 2024-06-06T07:44:04.558 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:04 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.558 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:04 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.558 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:04 smithi087 ceph-mon[32276]: pgmap v297: 97 pgs: 69 active+clean, 20 creating+peering, 8 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 1 op/s 2024-06-06T07:44:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:04 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:active 2024-06-06T07:44:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:04 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:04 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:04 smithi195 ceph-mon[37040]: pgmap v297: 97 pgs: 69 active+clean, 20 creating+peering, 8 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 1 op/s 2024-06-06T07:44:04.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:04 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] up:active 2024-06-06T07:44:04.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:04 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:04 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:04.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:04 smithi181 ceph-mon[36790]: pgmap v297: 97 pgs: 69 active+clean, 20 creating+peering, 8 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 1 op/s 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 -- 172.21.15.87:0/543331952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14107420 msgr2=0x7fca14107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 --2- 172.21.15.87:0/543331952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14107420 0x7fca14107800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca10009990 tx=0x7fca10009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 -- 172.21.15.87:0/543331952 shutdown_connections 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 --2- 172.21.15.87:0/543331952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101d30 0x7fca1410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 --2- 172.21.15.87:0/543331952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14107420 0x7fca14107800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 --2- 172.21.15.87:0/543331952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14101370 0x7fca141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 -- 172.21.15.87:0/543331952 >> 172.21.15.87:0/543331952 conn(0x7fca14078340 msgr2=0x7fca14078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 -- 172.21.15.87:0/543331952 shutdown_connections 2024-06-06T07:44:05.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 -- 172.21.15.87:0/543331952 wait complete. 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.134+0000 7fca1b702700 1 Processor -- start 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 -- start start 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101370 0x7fca1419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 0x7fca141979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca14111730 con 0x7fca14101d30 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca141115b0 con 0x7fca14107420 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1b702700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca141118b0 con 0x7fca14101370 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42636/0 (socket says 172.21.15.87:42636) 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 -- 172.21.15.87:0/2348838390 learned_addr learned my addr 172.21.15.87:0/2348838390 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca1949e700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101370 0x7fca1419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca19c9f700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 0x7fca141979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:05.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 -- 172.21.15.87:0/2348838390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101370 msgr2=0x7fca1419d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101370 0x7fca1419d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 -- 172.21.15.87:0/2348838390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 msgr2=0x7fca141979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 0x7fca141979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 -- 172.21.15.87:0/2348838390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca14198130 con 0x7fca14101d30 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca19c9f700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 0x7fca141979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.135+0000 7fca18c9d700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b530 tx=0x7fca0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.136+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04015020 con 0x7fca14101d30 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.136+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca14198420 con 0x7fca14101d30 2024-06-06T07:44:05.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.136+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04011d10 con 0x7fca14101d30 2024-06-06T07:44:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.136+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0400f2e0 con 0x7fca14101d30 2024-06-06T07:44:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.136+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca141a4760 con 0x7fca14101d30 2024-06-06T07:44:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.137+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca14069180 con 0x7fca14101d30 2024-06-06T07:44:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.139+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca04018020 con 0x7fca14101d30 2024-06-06T07:44:05.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.139+0000 7fca0a7fc700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 0x7fca000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.139+0000 7fca1949e700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 0x7fca000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.139+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fca040ceb10 con 0x7fca14101d30 2024-06-06T07:44:05.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.140+0000 7fca1949e700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 0x7fca000a5160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b5c0 tx=0x7fca10005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:05.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.143+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca0401f020 con 0x7fca14101d30 2024-06-06T07:44:05.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:05.352+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fca14198f30 con 0x7fca14101d30 2024-06-06T07:44:05.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:05 smithi087 ceph-mon[32276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T07:44:05.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:05 smithi087 ceph-mon[32276]: Cluster is now healthy 2024-06-06T07:44:05.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:05 smithi195 ceph-mon[37040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T07:44:05.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:05 smithi195 ceph-mon[37040]: Cluster is now healthy 2024-06-06T07:44:05.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:05 smithi181 ceph-mon[36790]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T07:44:05.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:05 smithi181 ceph-mon[36790]: Cluster is now healthy 2024-06-06T07:44:06.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.355+0000 7fca0a7fc700 1 -- 172.21.15.87:0/2348838390 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7fca14198f30 con 0x7fca14101d30 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.357+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 msgr2=0x7fca000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.357+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 0x7fca000a5160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b5c0 tx=0x7fca10005960 comp rx=0 tx=0).stop 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.357+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 msgr2=0x7fca1419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.357+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b530 tx=0x7fca0400b840 comp rx=0 tx=0).stop 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 shutdown_connections 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fca000a2ca0 0x7fca000a5160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca14107420 0x7fca141979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca14101370 0x7fca1419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:06.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 --2- 172.21.15.87:0/2348838390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14101d30 0x7fca1419dd50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:06.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 >> 172.21.15.87:0/2348838390 conn(0x7fca14078340 msgr2=0x7fca14100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:06.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 shutdown_connections 2024-06-06T07:44:06.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:06.358+0000 7fca1b702700 1 -- 172.21.15.87:0/2348838390 wait complete. 2024-06-06T07:44:06.515 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 07:44:06 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: 2024-06-06T07:44:06.205+0000 7f8dad03b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-06T07:44:06.515 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:06 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T07:44:06.515 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:06 smithi181 ceph-mon[36790]: pgmap v298: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T07:44:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:06 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T07:44:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:06 smithi087 ceph-mon[32276]: pgmap v298: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T07:44:06.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:06 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T07:44:06.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:06 smithi195 ceph-mon[37040]: pgmap v298: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T07:44:07.083 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-06T07:44:07.332 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:07 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T07:44:07.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:07 smithi087 ceph-mon[32276]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T07:44:07.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:07 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T07:44:07.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:07 smithi195 ceph-mon[37040]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T07:44:07.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:07 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2348838390' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T07:44:07.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:07 smithi181 ceph-mon[36790]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T07:44:08.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:08 smithi087 ceph-mon[32276]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T07:44:08.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:08 smithi195 ceph-mon[37040]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T07:44:08.817 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:08 smithi181 ceph-mon[36790]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- 172.21.15.87:0/2049279847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 msgr2=0x7fdefc105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 --2- 172.21.15.87:0/2049279847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc105fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099c0 tx=0x7fdee8009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- 172.21.15.87:0/2049279847 shutdown_connections 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 --2- 172.21.15.87:0/2049279847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 --2- 172.21.15.87:0/2049279847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 0x7fdefc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 --2- 172.21.15.87:0/2049279847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc105fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- 172.21.15.87:0/2049279847 >> 172.21.15.87:0/2049279847 conn(0x7fdefc078330 msgr2=0x7fdefc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- 172.21.15.87:0/2049279847 shutdown_connections 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- 172.21.15.87:0/2049279847 wait complete. 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 Processor -- start 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.113+0000 7fdf030c3700 1 -- start start 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 0x7fdefc103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc111680 con 0x7fdefc101fa0 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc111500 con 0x7fdefc1066c0 2024-06-06T07:44:09.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc111800 con 0x7fdefc1015f0 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36992/0 (socket says 172.21.15.87:36992) 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 -- 172.21.15.87:0/1703829486 learned_addr learned my addr 172.21.15.87:0/1703829486 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf00e5f700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 0x7fdefc103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf01660700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 -- 172.21.15.87:0/1703829486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 msgr2=0x7fdefc103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 0x7fdefc103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 -- 172.21.15.87:0/1703829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 msgr2=0x7fdefc103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 -- 172.21.15.87:0/1703829486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc1a70a0 con 0x7fdefc101fa0 2024-06-06T07:44:09.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf01660700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdefbfff700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009fd0 tx=0x7fdee8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004830 con 0x7fdefc101fa0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee8005020 con 0x7fdefc101fa0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc1a7330 con 0x7fdefc101fa0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.114+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8003f90 con 0x7fdefc101fa0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.115+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a7780 con 0x7fdefc101fa0 2024-06-06T07:44:09.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.116+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc050a00 con 0x7fdefc101fa0 2024-06-06T07:44:09.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.117+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdee8005540 con 0x7fdefc101fa0 2024-06-06T07:44:09.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.117+0000 7fdef9ffb700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 0x7fdeec0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:09.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.118+0000 7fdf00e5f700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 0x7fdeec0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:09.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.118+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fdee80d0030 con 0x7fdefc101fa0 2024-06-06T07:44:09.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.118+0000 7fdf00e5f700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 0x7fdeec0a5200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdef0005950 tx=0x7fdef000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:09.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.122+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdee8013070 con 0x7fdefc101fa0 2024-06-06T07:44:09.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:09.318+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdefc02de60 con 0x7fdefc101fa0 2024-06-06T07:44:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:09 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T07:44:09.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:09 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T07:44:09.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:09 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T07:44:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.372+0000 7fdef9ffb700 1 -- 172.21.15.87:0/1703829486 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7fdee809cff0 con 0x7fdefc101fa0 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 msgr2=0x7fdeec0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 0x7fdeec0a5200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdef0005950 tx=0x7fdef000a300 comp rx=0 tx=0).stop 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 msgr2=0x7fdefc1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009fd0 tx=0x7fdee8005f10 comp rx=0 tx=0).stop 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 shutdown_connections 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fdeec0a2d40 0x7fdeec0a5200 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdefc1066c0 0x7fdefc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:10.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdefc1015f0 0x7fdefc103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 --2- 172.21.15.87:0/1703829486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdefc101fa0 0x7fdefc1035c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 >> 172.21.15.87:0/1703829486 conn(0x7fdefc078330 msgr2=0x7fdefc06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 shutdown_connections 2024-06-06T07:44:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:10.375+0000 7fdf030c3700 1 -- 172.21.15.87:0/1703829486 wait complete. 2024-06-06T07:44:10.377 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data_ec' created 2024-06-06T07:44:10.755 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:10 smithi087 ceph-mon[32276]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T07:44:10.755 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:10 smithi087 ceph-mon[32276]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T07:44:10.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:10 smithi195 ceph-mon[37040]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T07:44:10.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:10 smithi195 ceph-mon[37040]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T07:44:10.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:10 smithi181 ceph-mon[36790]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T07:44:10.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:10 smithi181 ceph-mon[36790]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T07:44:11.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-06T07:44:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:11.548 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T07:44:11.548 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:11 smithi087 ceph-mon[32276]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T07:44:11.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T07:44:11.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:11 smithi195 ceph-mon[37040]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T07:44:11.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1703829486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T07:44:11.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:11 smithi181 ceph-mon[36790]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T07:44:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:12 smithi087 ceph-mon[32276]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T07:44:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:12 smithi087 ceph-mon[32276]: pgmap v305: 129 pgs: 8 creating+peering, 23 unknown, 98 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 531 B/s wr, 1 op/s 2024-06-06T07:44:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:12 smithi195 ceph-mon[37040]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T07:44:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:12 smithi195 ceph-mon[37040]: pgmap v305: 129 pgs: 8 creating+peering, 23 unknown, 98 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 531 B/s wr, 1 op/s 2024-06-06T07:44:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:12 smithi181 ceph-mon[36790]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T07:44:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:12 smithi181 ceph-mon[36790]: pgmap v305: 129 pgs: 8 creating+peering, 23 unknown, 98 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 531 B/s wr, 1 op/s 2024-06-06T07:44:13.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/3052910695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101390 msgr2=0x7f8b28101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:13.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/3052910695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101390 0x7f8b28101810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009990 tx=0x7f8b1c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/3052910695 shutdown_connections 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/3052910695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 0x7f8b281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/3052910695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101d50 0x7f8b2810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/3052910695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101390 0x7f8b28101810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/3052910695 >> 172.21.15.87:0/3052910695 conn(0x7f8b28078340 msgr2=0x7f8b28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/3052910695 shutdown_connections 2024-06-06T07:44:13.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.215+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/3052910695 wait complete. 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 Processor -- start 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 -- start start 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101390 0x7f8b2819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 0x7f8b28198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28112780 con 0x7f8b28101d50 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28112600 con 0x7f8b28107400 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b2d9c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28112900 con 0x7f8b28101390 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37012/0 (socket says 172.21.15.87:37012) 2024-06-06T07:44:13.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b267fc700 1 -- 172.21.15.87:0/2231021970 learned_addr learned my addr 172.21.15.87:0/2231021970 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b26ffd700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101390 0x7f8b2819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.216+0000 7f8b277fe700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 0x7f8b28198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 -- 172.21.15.87:0/2231021970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101390 msgr2=0x7f8b2819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101390 0x7f8b2819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 -- 172.21.15.87:0/2231021970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 msgr2=0x7f8b28198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 0x7f8b28198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 -- 172.21.15.87:0/2231021970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b28199220 con 0x7f8b28101d50 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b267fc700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c000c00 tx=0x7f8b1c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c02b070 con 0x7f8b28101d50 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b281994b0 con 0x7f8b28101d50 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c00f460 con 0x7f8b28101d50 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c01d740 con 0x7f8b28101d50 2024-06-06T07:44:13.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.217+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2810ee90 con 0x7f8b28101d50 2024-06-06T07:44:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.218+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b28069180 con 0x7f8b28101d50 2024-06-06T07:44:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.219+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8b1c02e070 con 0x7f8b28101d50 2024-06-06T07:44:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.220+0000 7f8b2c9c1700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 0x7f8b100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:13.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.220+0000 7f8b26ffd700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 0x7f8b100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.220+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7f8b1c0d4700 con 0x7f8b28101d50 2024-06-06T07:44:13.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.220+0000 7f8b26ffd700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 0x7f8b100a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b28199c00 tx=0x7f8b1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:13.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.224+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b1c035020 con 0x7f8b28101d50 2024-06-06T07:44:13.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:13.422+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8b2802dca0 con 0x7f8b28101d50 2024-06-06T07:44:13.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:13 smithi087 ceph-mon[32276]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:44:13.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:13 smithi087 ceph-mon[32276]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T07:44:13.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:13 smithi195 ceph-mon[37040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:44:13.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:13 smithi195 ceph-mon[37040]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T07:44:13.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:13 smithi181 ceph-mon[36790]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T07:44:13.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:13 smithi181 ceph-mon[36790]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T07:44:14.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.390+0000 7f8b2c9c1700 1 -- 172.21.15.87:0/2231021970 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7f8b1c023070 con 0x7f8b28101d50 2024-06-06T07:44:14.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 msgr2=0x7f8b100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:14.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 0x7f8b100a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b28199c00 tx=0x7f8b1800b410 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 msgr2=0x7f8b2819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c000c00 tx=0x7f8b1c00bab0 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 shutdown_connections 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8b100a2c20 0x7f8b100a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b28107400 0x7f8b28198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b28101390 0x7f8b2819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 --2- 172.21.15.87:0/2231021970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b28101d50 0x7f8b2819ee40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 >> 172.21.15.87:0/2231021970 conn(0x7f8b28078340 msgr2=0x7f8b280fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 shutdown_connections 2024-06-06T07:44:14.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:14.393+0000 7f8b2d9c3700 1 -- 172.21.15.87:0/2231021970 wait complete. 2024-06-06T07:44:14.395 INFO:teuthology.orchestra.run.smithi087.stderr:set pool 4 allow_ec_overwrites to true 2024-06-06T07:44:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:14 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T07:44:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:14 smithi195 ceph-mon[37040]: pgmap v307: 129 pgs: 8 creating+peering, 14 unknown, 107 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:14.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:14 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T07:44:14.764 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:14 smithi087 ceph-mon[32276]: pgmap v307: 129 pgs: 8 creating+peering, 14 unknown, 107 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:14 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T07:44:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:14 smithi181 ceph-mon[36790]: pgmap v307: 129 pgs: 8 creating+peering, 14 unknown, 107 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:15.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-06T07:44:15.356 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:15.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T07:44:15.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:15 smithi087 ceph-mon[32276]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T07:44:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T07:44:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:15 smithi195 ceph-mon[37040]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T07:44:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2231021970' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T07:44:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:15 smithi181 ceph-mon[36790]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T07:44:16.512 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:16 smithi087 ceph-mon[32276]: pgmap v309: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:16 smithi195 ceph-mon[37040]: pgmap v309: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:16.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:16 smithi181 ceph-mon[36790]: pgmap v309: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:17.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 -- 172.21.15.87:0/2639013592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d81084f0 msgr2=0x7f13d81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 --2- 172.21.15.87:0/2639013592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d81084f0 0x7f13d81088d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009990 tx=0x7f13c8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 -- 172.21.15.87:0/2639013592 shutdown_connections 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 --2- 172.21.15.87:0/2639013592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d806b3c0 0x7f13d810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 --2- 172.21.15.87:0/2639013592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d81084f0 0x7f13d81088d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 --2- 172.21.15.87:0/2639013592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806aa00 0x7f13d806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 -- 172.21.15.87:0/2639013592 >> 172.21.15.87:0/2639013592 conn(0x7f13d8078300 msgr2=0x7f13d8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:17.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 -- 172.21.15.87:0/2639013592 shutdown_connections 2024-06-06T07:44:17.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.071+0000 7f13dcfae700 1 -- 172.21.15.87:0/2639013592 wait complete. 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.072+0000 7f13dcfae700 1 Processor -- start 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.072+0000 7f13dcfae700 1 -- start start 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806b3c0 0x7f13d8101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d81084f0 0x7f13d8101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81127d0 con 0x7f13d806aa00 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8112650 con 0x7f13d806b3c0 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8112950 con 0x7f13d81084f0 2024-06-06T07:44:17.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55746/0 (socket says 172.21.15.87:55746) 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 -- 172.21.15.87:0/1516986864 learned_addr learned my addr 172.21.15.87:0/1516986864 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13cdd9b700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806b3c0 0x7f13d8101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d6d9d700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d81084f0 0x7f13d8101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 -- 172.21.15.87:0/1516986864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d81084f0 msgr2=0x7f13d8101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d81084f0 0x7f13d8101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 -- 172.21.15.87:0/1516986864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806b3c0 msgr2=0x7f13d8101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806b3c0 0x7f13d8101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 -- 172.21.15.87:0/1516986864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d81a7190 con 0x7f13d806aa00 2024-06-06T07:44:17.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13d659c700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8027070 con 0x7f13d806aa00 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d81a7420 con 0x7f13d806aa00 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c8004050 con 0x7f13d806aa00 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c800f460 con 0x7f13d806aa00 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.073+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81a7870 con 0x7f13d806aa00 2024-06-06T07:44:17.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.074+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d810a9d0 con 0x7f13d806aa00 2024-06-06T07:44:17.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.076+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f13c802a070 con 0x7f13d806aa00 2024-06-06T07:44:17.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.076+0000 7f13cffff700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 0x7f13c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:17.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.076+0000 7f13cdd9b700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 0x7f13c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:17.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.076+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f13c80d0710 con 0x7f13d806aa00 2024-06-06T07:44:17.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.077+0000 7f13cdd9b700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 0x7f13c40a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13d8078870 tx=0x7f13c000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:17.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.080+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f13c8031070 con 0x7f13d806aa00 2024-06-06T07:44:17.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.322+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f13d8069180 con 0x7f13d806aa00 2024-06-06T07:44:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.404+0000 7f13cffff700 1 -- 172.21.15.87:0/1516986864 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f13c8013070 con 0x7f13d806aa00 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 msgr2=0x7f13c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 0x7f13c40a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13d8078870 tx=0x7f13c000a380 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 msgr2=0x7f13d8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8003680 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 shutdown_connections 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f13c40a2c20 0x7f13c40a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d81084f0 0x7f13d8101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d806aa00 0x7f13d8101450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 --2- 172.21.15.87:0/1516986864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d806b3c0 0x7f13d8101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.406+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 >> 172.21.15.87:0/1516986864 conn(0x7f13d8078300 msgr2=0x7f13d80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.407+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 shutdown_connections 2024-06-06T07:44:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:17.407+0000 7f13dcfae700 1 -- 172.21.15.87:0/1516986864 wait complete. 2024-06-06T07:44:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:added data pool 4 to fsmap 2024-06-06T07:44:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:17 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T07:44:17.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:17 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T07:44:17.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:17 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T07:44:18.132 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:44:18.374 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:18.722 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:18 smithi087 ceph-mon[32276]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T07:44:18.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:18 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T07:44:18.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:18 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:18.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:18 smithi087 ceph-mon[32276]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:18 smithi195 ceph-mon[37040]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T07:44:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:18 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T07:44:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:18 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:18 smithi195 ceph-mon[37040]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:18 smithi181 ceph-mon[36790]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T07:44:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:18 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1516986864' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T07:44:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:18 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:18.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:18 smithi181 ceph-mon[36790]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:19.594 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:19 smithi087 ceph-mon[32276]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:44:19.594 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:19 smithi087 ceph-mon[32276]: Cluster is now healthy 2024-06-06T07:44:19.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:19 smithi195 ceph-mon[37040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:44:19.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:19 smithi195 ceph-mon[37040]: Cluster is now healthy 2024-06-06T07:44:19.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:19 smithi181 ceph-mon[36790]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T07:44:19.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:19 smithi181 ceph-mon[36790]: Cluster is now healthy 2024-06-06T07:44:20.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 -- 172.21.15.87:0/2755285157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 msgr2=0x7fb560074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:20.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 --2- 172.21.15.87:0/2755285157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb560074aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb554009990 tx=0x7fb554009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 -- 172.21.15.87:0/2755285157 shutdown_connections 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 --2- 172.21.15.87:0/2755285157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb56010eec0 0x7fb5601112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 --2- 172.21.15.87:0/2755285157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb560074aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 --2- 172.21.15.87:0/2755285157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 -- 172.21.15.87:0/2755285157 >> 172.21.15.87:0/2755285157 conn(0x7fb560079640 msgr2=0x7fb560079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 -- 172.21.15.87:0/2755285157 shutdown_connections 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.147+0000 7fb566a7b700 1 -- 172.21.15.87:0/2755285157 wait complete. 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 Processor -- start 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 -- start start 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb56010eec0 0x7fb56019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:20.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb560114a90 con 0x7fb5600746c0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb560114910 con 0x7fb56010eec0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb566a7b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb560114c10 con 0x7fb560074fe0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55768/0 (socket says 172.21.15.87:55768) 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 -- 172.21.15.87:0/1110690008 learned_addr learned my addr 172.21.15.87:0/1110690008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb55ffff700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 -- 172.21.15.87:0/1110690008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 msgr2=0x7fb56019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 -- 172.21.15.87:0/1110690008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb56010eec0 msgr2=0x7fb56019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb56010eec0 0x7fb56019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 -- 172.21.15.87:0/1110690008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56019fef0 con 0x7fb5600746c0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb55ffff700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.148+0000 7fb564817700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb554000c00 tx=0x7fb554003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:20.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.149+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554027070 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.149+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5601a0180 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.149+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb554004190 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.149+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55400f460 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.149+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5601a7a60 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.150+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb560069180 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.151+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb55402a070 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.151+0000 7fb55dffb700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 0x7fb5500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.152+0000 7fb55ffff700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 0x7fb5500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.152+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fb5540d0630 con 0x7fb5600746c0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.152+0000 7fb55ffff700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 0x7fb5500a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb560110b80 tx=0x7fb54c005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:20.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.155+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb554013070 con 0x7fb5600746c0 2024-06-06T07:44:20.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.394+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5601a7c30 con 0x7fb5600746c0 2024-06-06T07:44:20.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.394+0000 7fb55dffb700 1 -- 172.21.15.87:0/1110690008 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7fb55409d3e0 con 0x7fb5600746c0 2024-06-06T07:44:20.396 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:20.396 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:44:17.324274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24467,"qdb_cluster":[24467]},"id":1}]} 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 msgr2=0x7fb5500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 0x7fb5500a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb560110b80 tx=0x7fb54c005f50 comp rx=0 tx=0).stop 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 msgr2=0x7fb56019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb554000c00 tx=0x7fb554003730 comp rx=0 tx=0).stop 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 shutdown_connections 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb5500a2c20 0x7fb5500a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb56010eec0 0x7fb56019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5600746c0 0x7fb56019aee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 --2- 172.21.15.87:0/1110690008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb560074fe0 0x7fb56019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 >> 172.21.15.87:0/1110690008 conn(0x7fb560079640 msgr2=0x7fb5600ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 shutdown_connections 2024-06-06T07:44:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:20.398+0000 7fb566a7b700 1 -- 172.21.15.87:0/1110690008 wait complete. 2024-06-06T07:44:20.399 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-06-06T07:44:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:20 smithi087 ceph-mon[32276]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:20 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1110690008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:20.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:20 smithi195 ceph-mon[37040]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:20.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:20 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1110690008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:20 smithi181 ceph-mon[36790]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:20 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1110690008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:21.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:44:21.400 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:23.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 -- 172.21.15.87:0/2596484878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 msgr2=0x7fd270107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 --2- 172.21.15.87:0/2596484878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd270107620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2640099c0 tx=0x7fd264009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 -- 172.21.15.87:0/2596484878 shutdown_connections 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 --2- 172.21.15.87:0/2596484878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 0x7fd270105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 --2- 172.21.15.87:0/2596484878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd270107b60 0x7fd27010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 --2- 172.21.15.87:0/2596484878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd270107620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 -- 172.21.15.87:0/2596484878 >> 172.21.15.87:0/2596484878 conn(0x7fd2700766b0 msgr2=0x7fd270076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 -- 172.21.15.87:0/2596484878 shutdown_connections 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.275+0000 7fd27782b700 1 -- 172.21.15.87:0/2596484878 wait complete. 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 Processor -- start 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 -- start start 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 0x7fd270199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:23.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd270107b60 0x7fd2701a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111830 con 0x7fd2701071a0 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701116b0 con 0x7fd270105330 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd27782b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701119b0 con 0x7fd270107b60 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55778/0 (socket says 172.21.15.87:55778) 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 -- 172.21.15.87:0/3098871898 learned_addr learned my addr 172.21.15.87:0/3098871898 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd2755c7700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 0x7fd270199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 -- 172.21.15.87:0/3098871898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd270107b60 msgr2=0x7fd2701a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd270107b60 0x7fd2701a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 -- 172.21.15.87:0/3098871898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 msgr2=0x7fd270199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 0x7fd270199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 -- 172.21.15.87:0/3098871898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2701a5130 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.276+0000 7fd274dc6700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd264009fd0 tx=0x7fd264003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.277+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264027070 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.277+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd264004150 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.277+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2701a5300 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.277+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26400f460 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.277+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701a57b0 con 0x7fd2701071a0 2024-06-06T07:44:23.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.278+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd270069180 con 0x7fd2701071a0 2024-06-06T07:44:23.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.279+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd26402a070 con 0x7fd2701071a0 2024-06-06T07:44:23.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.279+0000 7fd26e7fc700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 0x7fd2580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:23.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.280+0000 7fd2755c7700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 0x7fd2580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:23.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.280+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fd2640d0890 con 0x7fd2701071a0 2024-06-06T07:44:23.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.280+0000 7fd2755c7700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 0x7fd2580a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd268009f20 tx=0x7fd2680094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:23.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.283+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd264031070 con 0x7fd2701071a0 2024-06-06T07:44:23.283 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:23 smithi087 ceph-mon[32276]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:23.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.481+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2701a5aa0 con 0x7fd2701071a0 2024-06-06T07:44:23.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.482+0000 7fd26e7fc700 1 -- 172.21.15.87:0/3098871898 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24154 (secure 0 0 0) 0x7fd264013070 con 0x7fd2701071a0 2024-06-06T07:44:23.483 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:23.483 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":95,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:17.324120+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:44:23.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 msgr2=0x7fd2580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:23.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 0x7fd2580a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd268009f20 tx=0x7fd2680094f0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 msgr2=0x7fd27019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:23.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd264009fd0 tx=0x7fd264003730 comp rx=0 tx=0).stop 2024-06-06T07:44:23.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 shutdown_connections 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fd2580a2ca0 0x7fd2580a5160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270105330 0x7fd270199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd270107b60 0x7fd2701a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 --2- 172.21.15.87:0/3098871898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2701071a0 0x7fd27019a0b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 >> 172.21.15.87:0/3098871898 conn(0x7fd2700766b0 msgr2=0x7fd2700fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 shutdown_connections 2024-06-06T07:44:23.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:23.485+0000 7fd27782b700 1 -- 172.21.15.87:0/3098871898 wait complete. 2024-06-06T07:44:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:23 smithi195 ceph-mon[37040]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:23 smithi181 ceph-mon[36790]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:24.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:24 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3098871898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:24.443 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:44:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:24 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3098871898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:24 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3098871898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:24.685 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:25.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:25 smithi087 ceph-mon[32276]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:25.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:25 smithi181 ceph-mon[36790]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:25 smithi195 ceph-mon[37040]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:26.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 -- 172.21.15.87:0/3008103328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 msgr2=0x7f4f68107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 --2- 172.21.15.87:0/3008103328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f68107140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0099c0 tx=0x7f4f5c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 -- 172.21.15.87:0/3008103328 shutdown_connections 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 --2- 172.21.15.87:0/3008103328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f68107870 0x7f4f6810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 --2- 172.21.15.87:0/3008103328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f681027a0 0x7f4f68102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 --2- 172.21.15.87:0/3008103328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f68107140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 -- 172.21.15.87:0/3008103328 >> 172.21.15.87:0/3008103328 conn(0x7f4f68079520 msgr2=0x7f4f68079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 -- 172.21.15.87:0/3008103328 shutdown_connections 2024-06-06T07:44:26.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.390+0000 7f4f6da41700 1 -- 172.21.15.87:0/3008103328 wait complete. 2024-06-06T07:44:26.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 Processor -- start 2024-06-06T07:44:26.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 -- start start 2024-06-06T07:44:26.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 0x7f4f6819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:26.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f68107870 0x7f4f68198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68112800 con 0x7f4f68103150 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68112680 con 0x7f4f681027a0 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f6da41700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68112980 con 0x7f4f68107870 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f667fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f667fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55806/0 (socket says 172.21.15.87:55806) 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f667fc700 1 -- 172.21.15.87:0/1703681456 learned_addr learned my addr 172.21.15.87:0/1703681456 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.391+0000 7f4f677fe700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f68107870 0x7f4f68198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f66ffd700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 0x7f4f6819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 -- 172.21.15.87:0/1703681456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f68107870 msgr2=0x7f4f68198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f68107870 0x7f4f68198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 -- 172.21.15.87:0/1703681456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 msgr2=0x7f4f6819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:26.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 0x7f4f6819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 -- 172.21.15.87:0/1703681456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f681991e0 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f66ffd700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 0x7f4f6819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f667fc700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fd0 tx=0x7f4f5c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c027070 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f68199470 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f5c004150 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00f460 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.392+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681a5740 con 0x7f4f68103150 2024-06-06T07:44:26.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.393+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f68069180 con 0x7f4f68103150 2024-06-06T07:44:26.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.394+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f5c02a070 con 0x7f4f68103150 2024-06-06T07:44:26.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.395+0000 7f4f6ca3f700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 0x7f4f500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:26.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.395+0000 7f4f66ffd700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 0x7f4f500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:26.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.395+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f4f5c0d09b0 con 0x7f4f68103150 2024-06-06T07:44:26.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.395+0000 7f4f66ffd700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 0x7f4f500a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f681073e0 tx=0x7f4f58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:26.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.398+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f5c031070 con 0x7f4f68103150 2024-06-06T07:44:26.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.594+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f6810aab0 con 0x7f4f68103150 2024-06-06T07:44:26.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.596+0000 7f4f6ca3f700 1 -- 172.21.15.87:0/1703681456 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24154 (secure 0 0 0) 0x7f4f5c013070 con 0x7f4f68103150 2024-06-06T07:44:26.596 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:26.596 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":95,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:17.324120+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 msgr2=0x7f4f500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 0x7f4f500a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f681073e0 tx=0x7f4f58008040 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 msgr2=0x7f4f6819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fd0 tx=0x7f4f5c003730 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 shutdown_connections 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4f500a2c20 0x7f4f500a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f681027a0 0x7f4f6819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f68107870 0x7f4f68198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 --2- 172.21.15.87:0/1703681456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f68103150 0x7f4f6819edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 >> 172.21.15.87:0/1703681456 conn(0x7f4f68079520 msgr2=0x7f4f68101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.598+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 shutdown_connections 2024-06-06T07:44:26.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:26.599+0000 7f4f6da41700 1 -- 172.21.15.87:0/1703681456 wait complete. 2024-06-06T07:44:27.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:27 smithi087 ceph-mon[32276]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:27.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:27 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1703681456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:27.462 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:44:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:27 smithi195 ceph-mon[37040]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:27 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1703681456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:27 smithi181 ceph-mon[36790]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:27 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1703681456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:27.699 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:29.352 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:29 smithi087 ceph-mon[32276]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:29.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:29 smithi195 ceph-mon[37040]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:29 smithi181 ceph-mon[36790]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- 172.21.15.87:0/2615434240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c101680 msgr2=0x7fe35c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2615434240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c101680 0x7fe35c101a60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3580099c0 tx=0x7fe358009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- 172.21.15.87:0/2615434240 shutdown_connections 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2615434240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c1066d0 0x7fe35c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2615434240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c101680 0x7fe35c101a60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2615434240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c102030 0x7fe35c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- 172.21.15.87:0/2615434240 >> 172.21.15.87:0/2615434240 conn(0x7fe35c0773e0 msgr2=0x7fe35c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:29.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- 172.21.15.87:0/2615434240 shutdown_connections 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- 172.21.15.87:0/2615434240 wait complete. 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 Processor -- start 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.590+0000 7fe3641f5700 1 -- start start 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c101680 0x7fe35c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 0x7fe35c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c111730 con 0x7fe35c102030 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1115b0 con 0x7fe35c101680 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1118b0 con 0x7fe35c1066d0 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:29.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37344/0 (socket says 172.21.15.87:37344) 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 -- 172.21.15.87:0/2708067147 learned_addr learned my addr 172.21.15.87:0/2708067147 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361f91700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c101680 0x7fe35c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe362792700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 0x7fe35c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 -- 172.21.15.87:0/2708067147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 msgr2=0x7fe35c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 0x7fe35c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 -- 172.21.15.87:0/2708067147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c101680 msgr2=0x7fe35c19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c101680 0x7fe35c19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 -- 172.21.15.87:0/2708067147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c198090 con 0x7fe35c102030 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe362792700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 0x7fe35c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe361790700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c9e0 tx=0x7fe34c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c007aa0 con 0x7fe35c102030 2024-06-06T07:44:29.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c007c40 con 0x7fe35c102030 2024-06-06T07:44:29.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c198380 con 0x7fe35c102030 2024-06-06T07:44:29.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c018780 con 0x7fe35c102030 2024-06-06T07:44:29.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.591+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c1a4640 con 0x7fe35c102030 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.592+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c069180 con 0x7fe35c102030 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.594+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe34c00f450 con 0x7fe35c102030 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.594+0000 7fe352ffd700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 0x7fe3480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.594+0000 7fe361f91700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 0x7fe3480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.594+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fe34c0cf3f0 con 0x7fe35c102030 2024-06-06T07:44:29.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.595+0000 7fe361f91700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 0x7fe3480a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe358006010 tx=0x7fe358005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:29.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.598+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe34c014030 con 0x7fe35c102030 2024-06-06T07:44:29.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.838+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe35c02dca0 con 0x7fe35c102030 2024-06-06T07:44:29.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.838+0000 7fe352ffd700 1 -- 172.21.15.87:0/2708067147 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7fe34c09bac0 con 0x7fe35c102030 2024-06-06T07:44:29.839 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:29.839 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:44:17.324274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24467,"qdb_cluster":[24467]},"id":1}]} 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 msgr2=0x7fe3480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 0x7fe3480a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe358006010 tx=0x7fe358005ab0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 msgr2=0x7fe35c19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c9e0 tx=0x7fe34c00cda0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 shutdown_connections 2024-06-06T07:44:29.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe3480a2c20 0x7fe3480a50e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c101680 0x7fe35c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe35c1066d0 0x7fe35c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 --2- 172.21.15.87:0/2708067147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe35c102030 0x7fe35c19dcb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 >> 172.21.15.87:0/2708067147 conn(0x7fe35c0773e0 msgr2=0x7fe35c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 shutdown_connections 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:29.841+0000 7fe3641f5700 1 -- 172.21.15.87:0/2708067147 wait complete. 2024-06-06T07:44:29.842 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 17 2024-06-06T07:44:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:30 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2708067147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:30 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2708067147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:30 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2708067147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:44:30.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:44:30.907 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:31.311 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:31 smithi087 ceph-mon[32276]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:31 smithi195 ceph-mon[37040]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:31 smithi181 ceph-mon[36790]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 -- 172.21.15.87:0/2171960936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c101610 msgr2=0x7fc78c1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 --2- 172.21.15.87:0/2171960936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c101610 0x7fc78c1019f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0099c0 tx=0x7fc77c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 -- 172.21.15.87:0/2171960936 shutdown_connections 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 --2- 172.21.15.87:0/2171960936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c1066f0 0x7fc78c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 --2- 172.21.15.87:0/2171960936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c101610 0x7fc78c1019f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 --2- 172.21.15.87:0/2171960936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101fc0 0x7fc78c106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 -- 172.21.15.87:0/2171960936 >> 172.21.15.87:0/2171960936 conn(0x7fc78c078430 msgr2=0x7fc78c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 -- 172.21.15.87:0/2171960936 shutdown_connections 2024-06-06T07:44:32.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.769+0000 7fc793cfe700 1 -- 172.21.15.87:0/2171960936 wait complete. 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 Processor -- start 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 -- start start 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101610 0x7fc78c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 0x7fc78c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c111770 con 0x7fc78c1066f0 2024-06-06T07:44:32.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1115f0 con 0x7fc78c101610 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.770+0000 7fc793cfe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1118f0 con 0x7fc78c101fc0 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37362/0 (socket says 172.21.15.87:37362) 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 -- 172.21.15.87:0/17934581 learned_addr learned my addr 172.21.15.87:0/17934581 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc791a9a700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101610 0x7fc78c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc791299700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 0x7fc78c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 -- 172.21.15.87:0/17934581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 msgr2=0x7fc78c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 0x7fc78c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 -- 172.21.15.87:0/17934581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101610 msgr2=0x7fc78c199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101610 0x7fc78c199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 -- 172.21.15.87:0/17934581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1a5170 con 0x7fc78c1066f0 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc791299700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 0x7fc78c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:32.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc79229b700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc78400ada0 tx=0x7fc78400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:32.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784017020 con 0x7fc78c1066f0 2024-06-06T07:44:32.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1979a0 con 0x7fc78c1066f0 2024-06-06T07:44:32.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78400a0d0 con 0x7fc78c1066f0 2024-06-06T07:44:32.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784005480 con 0x7fc78c1066f0 2024-06-06T07:44:32.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.771+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c197f00 con 0x7fc78c1066f0 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.772+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c069180 con 0x7fc78c1066f0 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.774+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc784005620 con 0x7fc78c1066f0 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.774+0000 7fc782ffd700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 0x7fc7780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.775+0000 7fc791a9a700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 0x7fc7780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.775+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fc7840cefb0 con 0x7fc78c1066f0 2024-06-06T07:44:32.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.775+0000 7fc791a9a700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 0x7fc7780a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009fd0 tx=0x7fc77c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:32.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.778+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc784021020 con 0x7fc78c1066f0 2024-06-06T07:44:32.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.973+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc78c02dca0 con 0x7fc78c1066f0 2024-06-06T07:44:32.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.975+0000 7fc782ffd700 1 -- 172.21.15.87:0/17934581 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24154 (secure 0 0 0) 0x7fc78409bd60 con 0x7fc78c1066f0 2024-06-06T07:44:32.975 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:32.975 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":95,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:17.324120+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:44:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 msgr2=0x7fc7780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 0x7fc7780a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009fd0 tx=0x7fc77c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 msgr2=0x7fc78c1a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc78400ada0 tx=0x7fc78400c330 comp rx=0 tx=0).stop 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 shutdown_connections 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc7780a2c20 0x7fc7780a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c101610 0x7fc78c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78c1066f0 0x7fc78c1a4c30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 --2- 172.21.15.87:0/17934581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78c101fc0 0x7fc78c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 >> 172.21.15.87:0/17934581 conn(0x7fc78c078430 msgr2=0x7fc78c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 shutdown_connections 2024-06-06T07:44:32.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:32.977+0000 7fc793cfe700 1 -- 172.21.15.87:0/17934581 wait complete. 2024-06-06T07:44:33.395 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:33 smithi087 ceph-mon[32276]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:33.395 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:33 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/17934581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:33 smithi195 ceph-mon[37040]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:33 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/17934581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:33 smithi181 ceph-mon[36790]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:33 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/17934581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:33.878 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-06T07:44:02.139989+0000', 'modified': '2024-06-06T07:44:17.324274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/851837058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 851837058}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 851837058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24467, 'qdb_cluster': [24467]} 2024-06-06T07:44:33.878 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-06T07:44:33.879 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:44:34.122 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:35.399 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:35 smithi087 ceph-mon[32276]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:35 smithi195 ceph-mon[37040]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:35 smithi181 ceph-mon[36790]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:35.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 -- 172.21.15.87:0/1321722940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 msgr2=0x7f71ec101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/1321722940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec101800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009990 tx=0x7f71dc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 -- 172.21.15.87:0/1321722940 shutdown_connections 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/1321722940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 0x7f71ec1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/1321722940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/1321722940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec101800 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 -- 172.21.15.87:0/1321722940 >> 172.21.15.87:0/1321722940 conn(0x7f71ec078300 msgr2=0x7f71ec078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 -- 172.21.15.87:0/1321722940 shutdown_connections 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.972+0000 7f71f1bbc700 1 -- 172.21.15.87:0/1321722940 wait complete. 2024-06-06T07:44:35.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 Processor -- start 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 -- start start 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 0x7f71ec197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec111690 con 0x7f71ec101380 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec111510 con 0x7f71ec1073f0 2024-06-06T07:44:35.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71f1bbc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec111810 con 0x7f71ec101d40 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37378/0 (socket says 172.21.15.87:37378) 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.973+0000 7f71eb7fe700 1 -- 172.21.15.87:0/649733677 learned_addr learned my addr 172.21.15.87:0/649733677 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71ebfff700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 0x7f71ec197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eaffd700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 -- 172.21.15.87:0/649733677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 msgr2=0x7f71ec19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 -- 172.21.15.87:0/649733677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 msgr2=0x7f71ec197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 0x7f71ec197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 -- 172.21.15.87:0/649733677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec1980b0 con 0x7f71ec101380 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eaffd700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c9b0 tx=0x7f71e000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71e8ff9700 1 -- 172.21.15.87:0/649733677 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0007b00 con 0x7f71ec101380 2024-06-06T07:44:35.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71e8ff9700 1 -- 172.21.15.87:0/649733677 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e0007ca0 con 0x7f71ec101380 2024-06-06T07:44:35.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec1983a0 con 0x7f71ec101380 2024-06-06T07:44:35.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71e8ff9700 1 -- 172.21.15.87:0/649733677 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0018780 con 0x7f71ec101380 2024-06-06T07:44:35.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.974+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec1a4660 con 0x7f71ec101380 2024-06-06T07:44:35.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.976+0000 7f71e8ff9700 1 -- 172.21.15.87:0/649733677 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71e000f450 con 0x7f71ec101380 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.977+0000 7f71e8ff9700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 0x7f71d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.977+0000 7f71eaffd700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 0x7f71d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.977+0000 7f71e8ff9700 1 -- 172.21.15.87:0/649733677 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f71e00cf210 con 0x7f71ec101380 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.977+0000 7f71eaffd700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 0x7f71d40a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b5c0 tx=0x7f71dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 Processor -- start 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 -- start start 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e89210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559b53d7bf10 con 0x559b53e89750 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559b53dda800 con 0x559b53e8ef10 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b53d42410 con 0x559b53e8f830 2024-06-06T07:44:35.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37388/0 (socket says 172.21.15.87:37388) 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 -- 172.21.15.87:0/3085938782 learned_addr learned my addr 172.21.15.87:0/3085938782 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71eaffd700 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e89210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 -- 172.21.15.87:0/3085938782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 msgr2=0x559b53e89210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e89210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 -- 172.21.15.87:0/3085938782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 msgr2=0x559b53e8f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e8f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 -- 172.21.15.87:0/3085938782 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b53d29880 con 0x559b53e89750 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.978+0000 7f71ebfff700 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71e4008550 tx=0x7f71e4008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1e20b23f713c0e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71d1ffb700 1 -- 172.21.15.87:0/3085938782 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4003c00 con 0x559b53e89750 2024-06-06T07:44:35.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71d1ffb700 1 -- 172.21.15.87:0/3085938782 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003da0 con 0x559b53e89750 2024-06-06T07:44:35.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71d1ffb700 1 -- 172.21.15.87:0/3085938782 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4012930 con 0x559b53e89750 2024-06-06T07:44:35.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 -- 172.21.15.87:0/3085938782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 msgr2=0x559b53e89b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71e4008550 tx=0x7f71e4008860 comp rx=0 tx=0).stop 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 -- 172.21.15.87:0/3085938782 shutdown_connections 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e8f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e89b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/3085938782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e89210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 -- 172.21.15.87:0/3085938782 >> 172.21.15.87:0/3085938782 conn(0x559b53db6030 msgr2=0x559b53e841d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 -- 172.21.15.87:0/3085938782 shutdown_connections 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.979+0000 7f71fc43fb80 1 -- 172.21.15.87:0/3085938782 wait complete. 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 -- start start 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 Processor -- start 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e94ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559b53d7af70 con 0x559b53e89750 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559b53dd67c0 con 0x559b53e8ef10 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b53e96210 con 0x559b53e8f830 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71fc43fb80 10 client.0 _ll_register_callbacks cb 0x559b53d70560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37404/0 (socket says 172.21.15.87:37404) 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.980+0000 7f71eb7fe700 1 -- 172.21.15.87:0/816993927 learned_addr learned my addr 172.21.15.87:0/816993927 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eaffd700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71ebfff700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e94ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 -- 172.21.15.87:0/816993927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 msgr2=0x559b53e94ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e94ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 -- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 msgr2=0x559b53e9b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 -- 172.21.15.87:0/816993927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b53da0700 con 0x559b53e89750 2024-06-06T07:44:35.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eaffd700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71e00ccb20 tx=0x7f71e009b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b20b49fd3e2d633 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e009fbe0 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f71e000f740 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4003da0 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f71e00a44a0 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71fc43fb80 10 client.14715 fetch_fsmap learned FSMap version 17 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x559b53ea2c40 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003da0 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.981+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f71e00a4010 con 0x559b53e89750 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 20 client.14715 populate_metadata read hostname 'smithi087' 2024-06-06T07:44:35.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53ea52b0 0x559b53ea6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 4 client.14715 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b53de7d00 con 0x559b53ea52b0 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71eaffd700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53ea52b0 0x559b53ea6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71caffd700 10 client.14715 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.982+0000 7f71eaffd700 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53ea52b0 0x559b53ea6720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb68ce7ece0d7bc server_cookie=5c1ccfbfdea4b295 in_seq=0 out_seq=0 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.983+0000 7f71caffd700 1 -- 172.21.15.87:0/816993927 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b53de7d00 con 0x559b53ea52b0 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.983+0000 7f71caffd700 10 client.14715 handle_command_reply: tid=0 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.983+0000 7f71fc43fb80 1 client.14715 shutdown 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.983+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 msgr2=0x559b53e9aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:35.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.983+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71e00ccb20 tx=0x7f71e009b240 comp rx=0 tx=0).stop 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 shutdown_connections 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53ea52b0 0x559b53ea6720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e94ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e89750 0x559b53e9aed0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/816993927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 >> 172.21.15.87:0/816993927 conn(0x559b53db6030 msgr2=0x559b53e836b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71caffd700 0 client.14715 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 shutdown_connections 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 1 -- 172.21.15.87:0/816993927 wait complete. 2024-06-06T07:44:35.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:35.984+0000 7f71fc43fb80 20 client.14715 trim_cache size 0 max 16384 2024-06-06T07:44:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 Processor -- start 2024-06-06T07:44:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 -- start start 2024-06-06T07:44:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 0x559b53e90280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea4a60 con 0x559b53e8f830 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea5130 con 0x559b53e8ef10 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea5800 con 0x559b53e89750 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37418/0 (socket says 172.21.15.87:37418) 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.005+0000 7f71ebfff700 1 -- 172.21.15.87:0/232230226 learned_addr learned my addr 172.21.15.87:0/232230226 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 0x559b53e90280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71eaffd700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 -- 172.21.15.87:0/232230226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 msgr2=0x559b53e90280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 0x559b53e90280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 -- 172.21.15.87:0/232230226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 msgr2=0x559b53e9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 -- 172.21.15.87:0/232230226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b53e9b780 con 0x559b53e8f830 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 0x559b53e90280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71eaffd700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71ebfff700 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f71e4015000 tx=0x7f71e40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad0937a7be7183 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71d17fa700 1 -- 172.21.15.87:0/232230226 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4021030 con 0x559b53e8f830 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71d17fa700 1 -- 172.21.15.87:0/232230226 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003a00 con 0x559b53e8f830 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.006+0000 7f71d17fa700 1 -- 172.21.15.87:0/232230226 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004470 con 0x559b53e8f830 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- 172.21.15.87:0/232230226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 msgr2=0x559b53eafb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f71e4015000 tx=0x7f71e40077a0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- 172.21.15.87:0/232230226 shutdown_connections 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8f830 0x559b53eafb80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e89750 0x559b53e90280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/232230226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e8ef10 0x559b53e9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- 172.21.15.87:0/232230226 >> 172.21.15.87:0/232230226 conn(0x559b53db6030 msgr2=0x559b53e9ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- 172.21.15.87:0/232230226 shutdown_connections 2024-06-06T07:44:36.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- 172.21.15.87:0/232230226 wait complete. 2024-06-06T07:44:36.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 -- start start 2024-06-06T07:44:36.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.007+0000 7f71fc43fb80 1 Processor -- start 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 0x559b53e867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea65a0 con 0x559b53e8ef10 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea5ed0 con 0x559b53e89750 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b53ea6c70 con 0x559b53e8f830 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 10 client.0 _ll_register_callbacks cb 0x559b53d70560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37434/0 (socket says 172.21.15.87:37434) 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 -- 172.21.15.87:0/2590989380 learned_addr learned my addr 172.21.15.87:0/2590989380 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 0x559b53e867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71ebfff700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 -- 172.21.15.87:0/2590989380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 msgr2=0x559b53e87270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e87270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 -- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 msgr2=0x559b53e867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 0x559b53e867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 -- 172.21.15.87:0/2590989380 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b53f11dc0 con 0x559b53e8ef10 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 0x559b53e867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71eaffd700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009990 tx=0x7f71dc019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942e32ef796dab45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71caffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00b9d0 con 0x559b53e8ef10 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f71dc00bb70 con 0x559b53e8ef10 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.008+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4003a00 con 0x559b53e8ef10 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f71dc041050 con 0x559b53e8ef10 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 10 client.14721 fetch_fsmap learned FSMap version 17 2024-06-06T07:44:36.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x559b53f13070 con 0x559b53e8ef10 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003a00 con 0x559b53e8ef10 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f71dc003ce0 con 0x559b53e8ef10 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 20 client.14721 populate_metadata read hostname 'smithi087' 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53f17c60 0x559b53f16970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 4 client.14721 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "status"}) -- 0x559b53de2280 con 0x559b53f17c60 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.009+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53f17c60 0x559b53f16970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71eb7fe700 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53f17c60 0x559b53f16970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d9a885fe2aa21e server_cookie=794a3cd48c981795 in_seq=0 out_seq=0 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71caffd700 10 client.14721 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71caffd700 1 -- 172.21.15.87:0/2590989380 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559b53de2280 con 0x559b53f17c60 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71caffd700 10 client.14721 handle_command_reply: tid=0 2024-06-06T07:44:36.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 client.14721 shutdown 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 msgr2=0x559b53e86d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009990 tx=0x7f71dc019050 comp rx=0 tx=0).stop 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 shutdown_connections 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559b53f17c60 0x559b53f16970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b53e89750 0x559b53e867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b53e8f830 0x559b53e87270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 --2- 172.21.15.87:0/2590989380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b53e8ef10 0x559b53e86d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71caffd700 0 client.14721 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 >> 172.21.15.87:0/2590989380 conn(0x559b53db6030 msgr2=0x559b53e82e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 shutdown_connections 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 1 -- 172.21.15.87:0/2590989380 wait complete. 2024-06-06T07:44:36.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.010+0000 7f71fc43fb80 20 client.14721 trim_cache size 0 max 16384 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24467, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 33.679718665999999, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 17, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 95, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 95, 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 50.438636516000003 2024-06-06T07:44:36.017 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 msgr2=0x7f71d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 0x7f71d40a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b5c0 tx=0x7f71dc005fb0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 msgr2=0x7f71ec19d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c9b0 tx=0x7f71e000cd70 comp rx=0 tx=0).stop 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 shutdown_connections 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f71d40a2ca0 0x7f71d40a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ec1073f0 0x7f71ec197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71ec101380 0x7f71ec19d790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 --2- 172.21.15.87:0/649733677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71ec101d40 0x7f71ec19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 >> 172.21.15.87:0/649733677 conn(0x7f71ec078300 msgr2=0x7f71ec0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 shutdown_connections 2024-06-06T07:44:36.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:36.011+0000 7f71f1bbc700 1 -- 172.21.15.87:0/649733677 wait complete. 2024-06-06T07:44:36.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T07:44:36.703 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:36.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T07:44:36.943 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:37.318 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:37 smithi087 ceph-mon[32276]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:37 smithi195 ceph-mon[37040]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:37 smithi181 ceph-mon[36790]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 -- 172.21.15.87:0/3574349965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2230107840 msgr2=0x7f223010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 --2- 172.21.15.87:0/3574349965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2230107840 0x7f223010f3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f222800b0d0 tx=0x7f222800b3e0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 -- 172.21.15.87:0/3574349965 shutdown_connections 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 --2- 172.21.15.87:0/3574349965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230102740 0x7f2230102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 --2- 172.21.15.87:0/3574349965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2230107840 0x7f223010f3c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 --2- 172.21.15.87:0/3574349965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22301030f0 0x7f2230107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 -- 172.21.15.87:0/3574349965 >> 172.21.15.87:0/3574349965 conn(0x7f22300795f0 msgr2=0x7f2230079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 -- 172.21.15.87:0/3574349965 shutdown_connections 2024-06-06T07:44:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 -- 172.21.15.87:0/3574349965 wait complete. 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.618+0000 7f223796d700 1 Processor -- start 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 -- start start 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2230102740 0x7f223019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 0x7f22301988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2230112780 con 0x7f22301030f0 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2230112600 con 0x7f2230107840 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2230112900 con 0x7f2230102740 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36704/0 (socket says 172.21.15.87:36704) 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 -- 172.21.15.87:0/3581923445 learned_addr learned my addr 172.21.15.87:0/3581923445 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2235709700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2230102740 0x7f223019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2235f0a700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 0x7f22301988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 -- 172.21.15.87:0/3581923445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2230102740 msgr2=0x7f223019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2230102740 0x7f223019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 -- 172.21.15.87:0/3581923445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 msgr2=0x7f22301988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 0x7f22301988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 -- 172.21.15.87:0/3581923445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2230199080 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2235f0a700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 0x7f22301988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f2234f08700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f222000e9c0 tx=0x7f222000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222000cbd0 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2230199370 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222000cd70 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.619+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220010430 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.620+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22301a5530 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.620+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2230103bd0 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.622+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2220004750 con 0x7f22301030f0 2024-06-06T07:44:38.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.622+0000 7f22267fc700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 0x7f221c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.622+0000 7f2235709700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 0x7f221c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.622+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f22200cf520 con 0x7f22301030f0 2024-06-06T07:44:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.623+0000 7f2235709700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 0x7f221c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c00ac50 tx=0x7f222c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:38.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.627+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f222009c240 con 0x7f22301030f0 2024-06-06T07:44:38.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.868+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22301a5790 con 0x7f22301030f0 2024-06-06T07:44:38.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.869+0000 7f22267fc700 1 -- 172.21.15.87:0/3581923445 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f222009bb40 con 0x7f22301030f0 2024-06-06T07:44:38.869 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T07:44:38.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 msgr2=0x7f221c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:38.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 0x7f221c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c00ac50 tx=0x7f222c00a380 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 msgr2=0x7f223019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f222000e9c0 tx=0x7f222000ecd0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 shutdown_connections 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f221c0a2c20 0x7f221c0a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2230107840 0x7f22301988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2230102740 0x7f223019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 --2- 172.21.15.87:0/3581923445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22301030f0 0x7f223019ec10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 >> 172.21.15.87:0/3581923445 conn(0x7f22300795f0 msgr2=0x7f22301003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 shutdown_connections 2024-06-06T07:44:38.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:38.871+0000 7f223796d700 1 -- 172.21.15.87:0/3581923445 wait complete. 2024-06-06T07:44:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:39 smithi195 ceph-mon[37040]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:39 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3581923445' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:44:39.538 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:39 smithi087 ceph-mon[32276]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:39.538 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:39 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3581923445' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:44:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:39 smithi181 ceph-mon[36790]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:39 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3581923445' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:44:39.569 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T07:44:39.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T07:44:39.571 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T07:44:39.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:39.574 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-06-06T07:44:39.592 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T07:44:39.592 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T07:44:39.592 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T07:44:39.592 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T07:44:39.593 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-06-06T07:44:39.594 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-06T07:44:39.594 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42227sec preferred_lft 42227sec 2024-06-06T07:44:39.594 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-06-06T07:44:39.594 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:44:39.594 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T07:44:39.595 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:44:39.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:44:39.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:39.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:39.838 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T07:44:39.863 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-06-06T07:44:39.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:39.865 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-06-06T07:44:39.938 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-06-06T07:44:39.938 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T07:44:39.938 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-06T07:44:39.939 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-06T07:44:39.939 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-06T07:44:39.939 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:44:40.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:44:40.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:40.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:40.259 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-06-06T07:44:40.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:40.275 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-06-06T07:44:40.329 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:44:40.330 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:44:40.330 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:40.330 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-06T07:44:40.330 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:44:40.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:44:40.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:40.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-06T07:44:40.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-06T07:44:40.450 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:44:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:44:40.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:40.574 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:44:40.574 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:44:40.574 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-06-06T07:44:40.574 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T07:44:40.574 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:44:40.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:44:40.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:40.651 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-06T07:44:40.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:44:40.652 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:40.667 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T07:44:40.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:44:40.667 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:40.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-06-06T07:44:40.787 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T07:44:40.844 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-06-06T07:44:40.844 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-06-06T07:44:40.845 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/merged 2024-06-06T07:44:40.846 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/0 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/merged 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:40.847 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:40.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:40.848 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-06-06T07:44:40.900 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T07:44:40.900 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-06T07:44:40.942 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-06-06T07:44:40.966 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T07:44:41.022 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-06-06T07:44:41.022 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-06-06T07:44:41.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/merged 2024-06-06T07:44:41.024 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/0 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/merged 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:41.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:41.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:41.026 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-06-06T07:44:41.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:2024-06-06T07:44:41.081+0000 7f4a0ca62480 -1 init, newargv = 0x562ed6af7860 newargc=15 2024-06-06T07:44:41.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[75552]: starting ceph client 2024-06-06T07:44:41.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[75552]: starting fuse 2024-06-06T07:44:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:41 smithi195 ceph-mon[37040]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:41 smithi181 ceph-mon[36790]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:41.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:41 smithi087 ceph-mon[32276]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:44:42.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-06-06T07:44:42.211 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T07:44:42.269 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-06-06T07:44:42.270 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T07:44:42.271 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/merged 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/merged 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/0 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/merged 2024-06-06T07:44:42.272 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/merged 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:42.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:42.274 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-06-06T07:44:42.327 INFO:teuthology.orchestra.run.smithi087.stdout:119 2024-06-06T07:44:42.327 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-06T07:44:42.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-06-06T07:44:42.327 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-06-06T07:44:42.327 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-06-06T07:44:42.327 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.admin")) 2024-06-06T07:44:42.329 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-06-06T07:44:42.450 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.admin.75552.asok 2024-06-06T07:44:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:42.460 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75552.asok 2024-06-06T07:44:42.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:42.461 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75552.asok status 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "admin", 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-06-06T07:44:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "75552", 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 14733, 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 14733 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-06-06T07:44:42.597 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1715233861 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1715233861 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.14733 192.168.0.1:0/1715233861", 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/1715233861", 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 17, 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 95, 2024-06-06T07:44:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-06-06T07:44:42.599 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-06-06T07:44:42.599 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-06-06T07:44:42.599 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:44:42.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:42.605 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-06-06T07:44:42.605 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:42.620 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-06-06T07:44:42.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:42.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:42.621 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:42.687 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-06T07:44:42.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T07:44:42.817 INFO:teuthology.orchestra.run.smithi087.stdout:# file: . 2024-06-06T07:44:42.817 INFO:teuthology.orchestra.run.smithi087.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-06T07:44:42.818 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:42.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:44:42.824 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:44:42.824 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:44:42.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T07:44:43.119 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:43 smithi195 ceph-mon[37040]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:43.519 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:43 smithi087 ceph-mon[32276]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:43 smithi181 ceph-mon[36790]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:44.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 -- 172.21.15.87:0/661007688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc107570 msgr2=0x7f1acc107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:44.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 --2- 172.21.15.87:0/661007688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc107570 0x7f1acc107950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009990 tx=0x7f1abc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 -- 172.21.15.87:0/661007688 shutdown_connections 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 --2- 172.21.15.87:0/661007688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc101810 0x7f1acc10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 --2- 172.21.15.87:0/661007688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc107570 0x7f1acc107950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 --2- 172.21.15.87:0/661007688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 0x7f1acc1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 -- 172.21.15.87:0/661007688 >> 172.21.15.87:0/661007688 conn(0x7f1acc0766b0 msgr2=0x7f1acc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 -- 172.21.15.87:0/661007688 shutdown_connections 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.854+0000 7f1ad102e700 1 -- 172.21.15.87:0/661007688 wait complete. 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 Processor -- start 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 -- start start 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 0x7f1acc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc107570 0x7f1acc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:44.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1127d0 con 0x7f1acc101810 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112650 con 0x7f1acc100e50 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.855+0000 7f1ad102e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112950 con 0x7f1acc107570 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36728/0 (socket says 172.21.15.87:36728) 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 -- 172.21.15.87:0/1140318254 learned_addr learned my addr 172.21.15.87:0/1140318254 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1acad9d700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 0x7f1acc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1acb59e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc107570 0x7f1acc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 -- 172.21.15.87:0/1140318254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc107570 msgr2=0x7f1acc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc107570 0x7f1acc198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 -- 172.21.15.87:0/1140318254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 msgr2=0x7f1acc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 0x7f1acc19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:44.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 -- 172.21.15.87:0/1140318254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc1993f0 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1aca59c700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000d950 tx=0x7f1ac000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0016070 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc1996e0 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac000fd70 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000e610 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.856+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a59c0 con 0x7f1acc101810 2024-06-06T07:44:44.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.857+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc06a680 con 0x7f1acc101810 2024-06-06T07:44:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.859+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1ac0019070 con 0x7f1acc101810 2024-06-06T07:44:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.859+0000 7f1ab3fff700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 0x7f1ab40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.859+0000 7f1acad9d700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 0x7f1ab40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.859+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f1ac00ced10 con 0x7f1acc101810 2024-06-06T07:44:44.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.860+0000 7f1acad9d700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 0x7f1ab40a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006010 tx=0x7f1abc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:44.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:44.863+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ac009ba30 con 0x7f1acc101810 2024-06-06T07:44:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.071+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1acc199fd0 con 0x7f1acc101810 2024-06-06T07:44:45.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.072+0000 7f1ab3fff700 1 -- 172.21.15.87:0/1140318254 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+592 (secure 0 0 0) 0x7f1ac009b330 con 0x7f1acc101810 2024-06-06T07:44:45.072 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:45.072 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3853061787 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:45.072 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3813763622 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/1264886975 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2314391466 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3870018525 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:45.073 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2887993384 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 msgr2=0x7f1ab40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 0x7f1ab40a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006010 tx=0x7f1abc005960 comp rx=0 tx=0).stop 2024-06-06T07:44:45.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 msgr2=0x7f1acc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000d950 tx=0x7f1ac000dc60 comp rx=0 tx=0).stop 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 shutdown_connections 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1ab40a2c20 0x7f1ab40a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100e50 0x7f1acc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1acc107570 0x7f1acc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 --2- 172.21.15.87:0/1140318254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1acc101810 0x7f1acc19ef80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 >> 172.21.15.87:0/1140318254 conn(0x7f1acc0766b0 msgr2=0x7f1acc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 shutdown_connections 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:45.074+0000 7f1ad102e700 1 -- 172.21.15.87:0/1140318254 wait complete. 2024-06-06T07:44:45.075 INFO:teuthology.orchestra.run.smithi087.stderr:listed 10 entries 2024-06-06T07:44:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:45 smithi195 ceph-mon[37040]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1140318254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:45 smithi181 ceph-mon[36790]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1140318254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:45.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:45 smithi087 ceph-mon[32276]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:45.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1140318254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:45.807 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:44:45.807 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:44:45.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T07:44:46.102 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-06-06T07:44:46.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-06T07:44:46.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-06T07:44:46.494 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-06T07:44:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-06T07:44:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T07:44:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-06-06T07:44:46.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-06T07:44:46.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-06T07:44:46.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-06T07:44:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-06-06T07:44:46.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-06T07:44:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-06T07:44:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:47 smithi195 ceph-mon[37040]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T07:44:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-06T07:44:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:47 smithi181 ceph-mon[36790]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T07:44:47.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-06T07:44:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:47 smithi087 ceph-mon[32276]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 -- 172.21.15.87:0/4169894368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c101d80 msgr2=0x7f5b2c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 --2- 172.21.15.87:0/4169894368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c101d80 0x7f5b2c10e240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b0a0 tx=0x7f5b2400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 -- 172.21.15.87:0/4169894368 shutdown_connections 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 --2- 172.21.15.87:0/4169894368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c107440 0x7f5b2c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 --2- 172.21.15.87:0/4169894368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c101d80 0x7f5b2c10e240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 --2- 172.21.15.87:0/4169894368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 -- 172.21.15.87:0/4169894368 >> 172.21.15.87:0/4169894368 conn(0x7f5b2c077400 msgr2=0x7f5b2c077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:47.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 -- 172.21.15.87:0/4169894368 shutdown_connections 2024-06-06T07:44:47.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.818+0000 7f5b31158700 1 -- 172.21.15.87:0/4169894368 wait complete. 2024-06-06T07:44:47.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.819+0000 7f5b31158700 1 Processor -- start 2024-06-06T07:44:47.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.819+0000 7f5b31158700 1 -- start start 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 0x7f5b2c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c107440 0x7f5b2c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c111680 con 0x7f5b2c1013c0 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c111500 con 0x7f5b2c101d80 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c111800 con 0x7f5b2c107440 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54058/0 (socket says 172.21.15.87:54058) 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 -- 172.21.15.87:0/2973471547 learned_addr learned my addr 172.21.15.87:0/2973471547 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2b59e700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c107440 0x7f5b2c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2a59c700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 0x7f5b2c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:47.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 -- 172.21.15.87:0/2973471547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c107440 msgr2=0x7f5b2c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c107440 0x7f5b2c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 -- 172.21.15.87:0/2973471547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 msgr2=0x7f5b2c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 0x7f5b2c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 -- 172.21.15.87:0/2973471547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c198060 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2a59c700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 0x7f5b2c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b2ad9d700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00d950 tx=0x7f5b1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c016070 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c198350 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b1c009e20 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.820+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00f620 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.821+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c1a4610 con 0x7f5b2c1013c0 2024-06-06T07:44:47.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.821+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c102ab0 con 0x7f5b2c1013c0 2024-06-06T07:44:47.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.823+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b1c019070 con 0x7f5b2c1013c0 2024-06-06T07:44:47.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.823+0000 7f5b13fff700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 0x7f5b140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:47.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.823+0000 7f5b2a59c700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 0x7f5b140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:47.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.823+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f5b1c0cef00 con 0x7f5b2c1013c0 2024-06-06T07:44:47.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.824+0000 7f5b2a59c700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 0x7f5b140a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c077b40 tx=0x7f5b20005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:47.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:47.827+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b1c09bb00 con 0x7f5b2c1013c0 2024-06-06T07:44:48.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.034+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b2c198e60 con 0x7f5b2c1013c0 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.035+0000 7f5b13fff700 1 -- 172.21.15.87:0/2973471547 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+592 (secure 0 0 0) 0x7f5b2c198e60 con 0x7f5b2c1013c0 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3853061787 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3813763622 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/1264886975 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2314391466 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3870018525 2024-06-07T07:36:20.511068+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2887993384 2024-06-07T07:36:46.051849+0000 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 msgr2=0x7f5b140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 0x7f5b140a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c077b40 tx=0x7f5b20005f30 comp rx=0 tx=0).stop 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 msgr2=0x7f5b2c19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00d950 tx=0x7f5b1c00dc60 comp rx=0 tx=0).stop 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 shutdown_connections 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5b140a2c20 0x7f5b140a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b2c107440 0x7f5b2c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2c1013c0 0x7f5b2c19d740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 --2- 172.21.15.87:0/2973471547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b2c101d80 0x7f5b2c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 >> 172.21.15.87:0/2973471547 conn(0x7f5b2c077400 msgr2=0x7f5b2c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 shutdown_connections 2024-06-06T07:44:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:48.037+0000 7f5b31158700 1 -- 172.21.15.87:0/2973471547 wait complete. 2024-06-06T07:44:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:listed 10 entries 2024-06-06T07:44:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:48 smithi087 ceph-mon[32276]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T07:44:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:48 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2973471547' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:48.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:48 smithi087 ceph-mon[32276]: pgmap v328: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T07:44:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:48 smithi195 ceph-mon[37040]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T07:44:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:48 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2973471547' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:48.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:48 smithi195 ceph-mon[37040]: pgmap v328: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T07:44:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:48 smithi181 ceph-mon[36790]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T07:44:48.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:48 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2973471547' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:44:48.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:48 smithi181 ceph-mon[36790]: pgmap v328: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T07:44:48.791 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-06-06T07:44:48.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:48.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-06T07:44:48.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi087.stderr:ceph-fuse[75552]: fuse finished with error 0 and tester_r 0 2024-06-06T07:44:48.819 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T07:44:48.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:44:48.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:44:48.866 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-06T07:44:48.888 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:44:49.165 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:50.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- 172.21.15.87:0/3256775447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 msgr2=0x7f88981077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:50.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 --2- 172.21.15.87:0/3256775447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f88981077d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8888009990 tx=0x7f8888009ca0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- 172.21.15.87:0/3256775447 shutdown_connections 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 --2- 172.21.15.87:0/3256775447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101d40 0x7f889810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 --2- 172.21.15.87:0/3256775447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f88981077d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 --2- 172.21.15.87:0/3256775447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101380 0x7f8898101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- 172.21.15.87:0/3256775447 >> 172.21.15.87:0/3256775447 conn(0x7f8898078300 msgr2=0x7f8898078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- 172.21.15.87:0/3256775447 shutdown_connections 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- 172.21.15.87:0/3256775447 wait complete. 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 Processor -- start 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.860+0000 7f889e365700 1 -- start start 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101380 0x7f88981a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:50.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101d40 0x7f88981a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8898110600 con 0x7f88981073f0 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8898110480 con 0x7f8898101380 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889e365700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8898110780 con 0x7f8898101d40 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54066/0 (socket says 172.21.15.87:54066) 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 -- 172.21.15.87:0/3989215469 learned_addr learned my addr 172.21.15.87:0/3989215469 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f8897fff700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101380 0x7f88981a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f88977fe700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101d40 0x7f88981a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 -- 172.21.15.87:0/3989215469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101d40 msgr2=0x7f88981a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101d40 0x7f88981a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 -- 172.21.15.87:0/3989215469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101380 msgr2=0x7f88981a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101380 0x7f88981a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:50.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 -- 172.21.15.87:0/3989215469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889819b850 con 0x7f88981073f0 2024-06-06T07:44:50.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.861+0000 7f889c902700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f889000b970 tx=0x7f889000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.862+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000d860 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.862+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889000e430 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.862+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890017660 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.862+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889819bb40 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.862+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981a7aa0 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.863+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8898069180 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.864+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f889000e5e0 con 0x7f88981073f0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.864+0000 7f88957fa700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 0x7f88800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.865+0000 7f8897fff700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 0x7f88800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:50.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.865+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f88900cf170 con 0x7f88981073f0 2024-06-06T07:44:50.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.865+0000 7f8897fff700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 0x7f88800a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888800b5c0 tx=0x7f8888005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:50.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:50.868+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8890013030 con 0x7f88981073f0 2024-06-06T07:44:51.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.064+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88981a7d30 con 0x7f88981073f0 2024-06-06T07:44:51.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.066+0000 7f88957fa700 1 -- 172.21.15.87:0/3989215469 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24625 (secure 0 0 0) 0x7f889009bd70 con 0x7f88981073f0 2024-06-06T07:44:51.066 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:51.066 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":97,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:47.243465+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:44:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 msgr2=0x7f88800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 0x7f88800a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888800b5c0 tx=0x7f8888005960 comp rx=0 tx=0).stop 2024-06-06T07:44:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 msgr2=0x7f889819b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f889000b970 tx=0x7f889000bc80 comp rx=0 tx=0).stop 2024-06-06T07:44:51.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 shutdown_connections 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f88800a2c20 0x7f88800a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8898101380 0x7f88981a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88981073f0 0x7f889819b080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 --2- 172.21.15.87:0/3989215469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898101d40 0x7f88981a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 >> 172.21.15.87:0/3989215469 conn(0x7f8898078300 msgr2=0x7f88980ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 shutdown_connections 2024-06-06T07:44:51.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:51.068+0000 7f889e365700 1 -- 172.21.15.87:0/3989215469 wait complete. 2024-06-06T07:44:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:51 smithi195 ceph-mon[37040]: pgmap v329: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T07:44:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:51 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3989215469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:51 smithi181 ceph-mon[36790]: pgmap v329: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T07:44:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:51 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3989215469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:51.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:51 smithi087 ceph-mon[32276]: pgmap v329: 129 pgs: 1 activating, 2 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T07:44:51.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:51 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3989215469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:51.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:44:51.982 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:53.445 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:53 smithi087 ceph-mon[32276]: pgmap v330: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:53 smithi195 ceph-mon[37040]: pgmap v330: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:53 smithi181 ceph-mon[36790]: pgmap v330: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 -- 172.21.15.87:0/1061298685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c105330 msgr2=0x7feb2c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 --2- 172.21.15.87:0/1061298685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c105330 0x7feb2c105710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0099c0 tx=0x7feb1c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 -- 172.21.15.87:0/1061298685 shutdown_connections 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 --2- 172.21.15.87:0/1061298685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c107b60 0x7feb2c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 --2- 172.21.15.87:0/1061298685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c105330 0x7feb2c105710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 --2- 172.21.15.87:0/1061298685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c1071a0 0x7feb2c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 -- 172.21.15.87:0/1061298685 >> 172.21.15.87:0/1061298685 conn(0x7feb2c0766b0 msgr2=0x7feb2c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 -- 172.21.15.87:0/1061298685 shutdown_connections 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 -- 172.21.15.87:0/1061298685 wait complete. 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.708+0000 7feb31351700 1 Processor -- start 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 -- start start 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c105330 0x7feb2c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c1071a0 0x7feb2c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c111830 con 0x7feb2c107b60 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c1116b0 con 0x7feb2c105330 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb31351700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c1119b0 con 0x7feb2c1071a0 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:53.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54074/0 (socket says 172.21.15.87:54074) 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 -- 172.21.15.87:0/3611450881 learned_addr learned my addr 172.21.15.87:0/3611450881 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2a7fc700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c1071a0 0x7feb2c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2affd700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c105330 0x7feb2c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 -- 172.21.15.87:0/3611450881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c1071a0 msgr2=0x7feb2c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c1071a0 0x7feb2c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 -- 172.21.15.87:0/3611450881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c105330 msgr2=0x7feb2c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c105330 0x7feb2c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.709+0000 7feb2b7fe700 1 -- 172.21.15.87:0/3611450881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c1a7240 con 0x7feb2c107b60 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb2b7fe700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb2400b970 tx=0x7feb2400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400d860 con 0x7feb2c107b60 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2400e430 con 0x7feb2c107b60 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24017660 con 0x7feb2c107b60 2024-06-06T07:44:53.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c1a7530 con 0x7feb2c107b60 2024-06-06T07:44:53.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.710+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c1a7a20 con 0x7feb2c107b60 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.711+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c069180 con 0x7feb2c107b60 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.712+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb2400e5e0 con 0x7feb2c107b60 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.713+0000 7feb13fff700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 0x7feb140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.713+0000 7feb2affd700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 0x7feb140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.713+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7feb240cf170 con 0x7feb2c107b60 2024-06-06T07:44:53.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.713+0000 7feb2affd700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 0x7feb140a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0098e0 tx=0x7feb1c005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:53.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.717+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feb24013030 con 0x7feb2c107b60 2024-06-06T07:44:53.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.913+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb2c1a7d80 con 0x7feb2c107b60 2024-06-06T07:44:53.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.915+0000 7feb13fff700 1 -- 172.21.15.87:0/3611450881 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24625 (secure 0 0 0) 0x7feb2409bd70 con 0x7feb2c107b60 2024-06-06T07:44:53.916 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:44:53.916 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":97,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:47.243465+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:44:53.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 msgr2=0x7feb140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 0x7feb140a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0098e0 tx=0x7feb1c005ab0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 msgr2=0x7feb2c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb2400b970 tx=0x7feb2400bc80 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 shutdown_connections 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7feb140a2c20 0x7feb140a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2c105330 0x7feb2c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb2c107b60 0x7feb2c102ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 --2- 172.21.15.87:0/3611450881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb2c1071a0 0x7feb2c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 >> 172.21.15.87:0/3611450881 conn(0x7feb2c0766b0 msgr2=0x7feb2c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 shutdown_connections 2024-06-06T07:44:53.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:53.917+0000 7feb31351700 1 -- 172.21.15.87:0/3611450881 wait complete. 2024-06-06T07:44:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:54 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3611450881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:54 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:54 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3611450881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:54 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:54 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3611450881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:54 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:44:54.668 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-06T07:44:55.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:55 smithi087 ceph-mon[32276]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:55 smithi195 ceph-mon[37040]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:55.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:55 smithi181 ceph-mon[36790]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-06-06T07:44:55.557 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:57.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:56 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:56 smithi087 ceph-mon[32276]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 310 B/s wr, 0 op/s 2024-06-06T07:44:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:56 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:56 smithi181 ceph-mon[36790]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 310 B/s wr, 0 op/s 2024-06-06T07:44:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:56 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:57.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:56 smithi195 ceph-mon[37040]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 310 B/s wr, 0 op/s 2024-06-06T07:44:57.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 -- 172.21.15.87:0/1929616689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458101440 msgr2=0x7ff45810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:57.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/1929616689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458101440 0x7ff45810f330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff45000b0a0 tx=0x7ff45000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 -- 172.21.15.87:0/1929616689 shutdown_connections 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/1929616689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458108510 0x7ff4581088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/1929616689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458101440 0x7ff45810f330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/1929616689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 0x7ff458100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 -- 172.21.15.87:0/1929616689 >> 172.21.15.87:0/1929616689 conn(0x7ff458077a00 msgr2=0x7ff4580fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 -- 172.21.15.87:0/1929616689 shutdown_connections 2024-06-06T07:44:57.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.297+0000 7ff45cbc7700 1 -- 172.21.15.87:0/1929616689 wait complete. 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 Processor -- start 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 -- start start 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 0x7ff45819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458101440 0x7ff45819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff458112770 con 0x7ff458108510 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581125f0 con 0x7ff458101440 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.298+0000 7ff45cbc7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581128f0 con 0x7ff458100a80 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58218/0 (socket says 172.21.15.87:58218) 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 -- 172.21.15.87:0/3587754536 learned_addr learned my addr 172.21.15.87:0/3587754536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:44:57.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff45659c700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 0x7ff45819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff44dd9b700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458101440 0x7ff45819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 -- 172.21.15.87:0/3587754536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 msgr2=0x7ff45819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 0x7ff45819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 -- 172.21.15.87:0/3587754536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458101440 msgr2=0x7ff45819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458101440 0x7ff45819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 -- 172.21.15.87:0/3587754536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff458199030 con 0x7ff458108510 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff456d9d700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4500062a0 tx=0x7ff45000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450021070 con 0x7ff458108510 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4581992c0 con 0x7ff458108510 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450007c80 con 0x7ff458108510 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450005420 con 0x7ff458108510 2024-06-06T07:44:57.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.299+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4581a55e0 con 0x7ff458108510 2024-06-06T07:44:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.300+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff458069180 con 0x7ff458108510 2024-06-06T07:44:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.301+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff450024070 con 0x7ff458108510 2024-06-06T07:44:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.302+0000 7ff44ffff700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 0x7ff4440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:44:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.302+0000 7ff45659c700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 0x7ff4440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:44:57.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.302+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff45000f020 con 0x7ff458108510 2024-06-06T07:44:57.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.303+0000 7ff45659c700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 0x7ff4440a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff458199a10 tx=0x7ff44800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:44:57.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.307+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff45002b070 con 0x7ff458108510 2024-06-06T07:44:57.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.564+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff458199e00 con 0x7ff458108510 2024-06-06T07:44:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.886+0000 7ff44ffff700 1 -- 172.21.15.87:0/3587754536 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff45009d050 con 0x7ff458108510 2024-06-06T07:44:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 msgr2=0x7ff4440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 0x7ff4440a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff458199a10 tx=0x7ff44800a380 comp rx=0 tx=0).stop 2024-06-06T07:44:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 msgr2=0x7ff4581988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:44:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4500062a0 tx=0x7ff45000be60 comp rx=0 tx=0).stop 2024-06-06T07:44:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 shutdown_connections 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff4440a2c20 0x7ff4440a50e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458108510 0x7ff4581988d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff458100a80 0x7ff45819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 --2- 172.21.15.87:0/3587754536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff458101440 0x7ff45819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 >> 172.21.15.87:0/3587754536 conn(0x7ff458077a00 msgr2=0x7ff4580ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.888+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 shutdown_connections 2024-06-06T07:44:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:44:57.889+0000 7ff45cbc7700 1 -- 172.21.15.87:0/3587754536 wait complete. 2024-06-06T07:44:58.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T07:44:58.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:57 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.271 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T07:44:58.271 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.271 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:57 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T07:44:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:57 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:58.607 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T07:44:58.607 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-06T07:44:58.846 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:44:59.139 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:44:58 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:44:59.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:44:58 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3587754536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:44:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:44:58 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:45:00.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 -- 172.21.15.87:0/1914485635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 msgr2=0x7f85a810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1914485635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a810e280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85a000b0d0 tx=0x7f85a000b3e0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 -- 172.21.15.87:0/1914485635 shutdown_connections 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1914485635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a810e280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1914485635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 0x7f85a8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1914485635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 0x7f85a8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 -- 172.21.15.87:0/1914485635 >> 172.21.15.87:0/1914485635 conn(0x7f85a8078340 msgr2=0x7f85a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 -- 172.21.15.87:0/1914485635 shutdown_connections 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.597+0000 7f85ad59f700 1 -- 172.21.15.87:0/1914485635 wait complete. 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 Processor -- start 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 -- start start 2024-06-06T07:45:00.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 0x7f85a819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 0x7f85a819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81116a0 con 0x7f85a81066b0 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8111520 con 0x7f85a8101fd0 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.598+0000 7f85ad59f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8111820 con 0x7f85a8101620 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58244/0 (socket says 172.21.15.87:58244) 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 -- 172.21.15.87:0/1728985942 learned_addr learned my addr 172.21.15.87:0/1728985942 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a67fc700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 0x7f85a819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a6ffd700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 0x7f85a819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 -- 172.21.15.87:0/1728985942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 msgr2=0x7f85a819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 0x7f85a819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 -- 172.21.15.87:0/1728985942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 msgr2=0x7f85a819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:00.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 0x7f85a819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 -- 172.21.15.87:0/1728985942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a81990b0 con 0x7f85a81066b0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85a77fe700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f85a0006070 tx=0x7f85a0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0021070 con 0x7f85a81066b0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a8199340 con 0x7f85a81066b0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0007c00 con 0x7f85a81066b0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00052a0 con 0x7f85a81066b0 2024-06-06T07:45:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.599+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a5610 con 0x7f85a81066b0 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.600+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8069180 con 0x7f85a81066b0 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.601+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85a0024070 con 0x7f85a81066b0 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.602+0000 7f858ffff700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 0x7f85900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.602+0000 7f85a6ffd700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 0x7f85900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.602+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f85a000f020 con 0x7f85a81066b0 2024-06-06T07:45:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.603+0000 7f85a6ffd700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 0x7f85900a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85a8078a70 tx=0x7f859800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.606+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f85a002b020 con 0x7f85a81066b0 2024-06-06T07:45:00.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:00.844+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f85a8199e30 con 0x7f85a81066b0 2024-06-06T07:45:01.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.205+0000 7f858ffff700 1 -- 172.21.15.87:0/1728985942 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f85a0007db0 con 0x7f85a81066b0 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 msgr2=0x7f85900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 0x7f85900a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85a8078a70 tx=0x7f859800a380 comp rx=0 tx=0).stop 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 msgr2=0x7f85a8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f85a0006070 tx=0x7f85a0004e40 comp rx=0 tx=0).stop 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 shutdown_connections 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f85900a2c20 0x7f85900a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85a81066b0 0x7f85a8198950 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a8101620 0x7f85a819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:01.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 --2- 172.21.15.87:0/1728985942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a8101fd0 0x7f85a819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:01.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 >> 172.21.15.87:0/1728985942 conn(0x7f85a8078340 msgr2=0x7f85a806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:01.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 shutdown_connections 2024-06-06T07:45:01.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:01.208+0000 7f85ad59f700 1 -- 172.21.15.87:0/1728985942 wait complete. 2024-06-06T07:45:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:01 smithi195 ceph-mon[37040]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T07:45:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:01 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T07:45:01.548 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:01 smithi087 ceph-mon[32276]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T07:45:01.548 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:01 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T07:45:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:01 smithi181 ceph-mon[36790]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T07:45:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:01 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T07:45:01.922 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-06T07:45:02.164 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T07:45:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:02 smithi195 ceph-mon[37040]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T07:45:02.550 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T07:45:02.550 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T07:45:02.550 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T07:45:02.551 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:45:02.551 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T07:45:02.551 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:02 smithi087 ceph-mon[32276]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T07:45:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T07:45:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1728985942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T07:45:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T07:45:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T07:45:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T07:45:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:02 smithi181 ceph-mon[36790]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T07:45:03.305 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:45:03.305 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:active 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: Cluster is now healthy 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T07:45:03.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:03 smithi087 ceph-mon[32276]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:active 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: Cluster is now healthy 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T07:45:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:03 smithi195 ceph-mon[37040]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] up:active 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: Cluster is now healthy 2024-06-06T07:45:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T07:45:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T07:45:03.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:03 smithi181 ceph-mon[36790]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T07:45:03.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 -- 172.21.15.87:0/806618212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230107400 msgr2=0x7f92301077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 --2- 172.21.15.87:0/806618212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230107400 0x7f92301077e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f922c009990 tx=0x7f922c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 -- 172.21.15.87:0/806618212 shutdown_connections 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 --2- 172.21.15.87:0/806618212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101d50 0x7f923010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 --2- 172.21.15.87:0/806618212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230107400 0x7f92301077e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 --2- 172.21.15.87:0/806618212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230101390 0x7f9230101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 -- 172.21.15.87:0/806618212 >> 172.21.15.87:0/806618212 conn(0x7f9230078340 msgr2=0x7f9230078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 -- 172.21.15.87:0/806618212 shutdown_connections 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.888+0000 7f92376bf700 1 -- 172.21.15.87:0/806618212 wait complete. 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 Processor -- start 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 -- start start 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101390 0x7f923019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:03.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 0x7f9230197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92301116a0 con 0x7f9230101d50 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111520 con 0x7f9230101390 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f92376bf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111820 con 0x7f9230107400 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f9234c5a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f9234c5a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58266/0 (socket says 172.21.15.87:58266) 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.889+0000 7f9234c5a700 1 -- 172.21.15.87:0/978030567 learned_addr learned my addr 172.21.15.87:0/978030567 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f923545b700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101390 0x7f923019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9235c5c700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 0x7f9230197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 -- 172.21.15.87:0/978030567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 msgr2=0x7f9230197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 0x7f9230197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 -- 172.21.15.87:0/978030567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101390 msgr2=0x7f923019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101390 0x7f923019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:03.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 -- 172.21.15.87:0/978030567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230198080 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9235c5c700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 0x7f9230197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f9234c5a700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f922000d980 tx=0x7f922000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220017070 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9220005020 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9230198370 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220009bc0 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.890+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a45e0 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.891+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9230069180 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.892+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92200051d0 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.893+0000 7f92267fc700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 0x7f921c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.893+0000 7f923545b700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 0x7f921c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.893+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f92200cf260 con 0x7f9230101d50 2024-06-06T07:45:03.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.893+0000 7f923545b700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 0x7f921c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f922c00b5c0 tx=0x7f922c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:03.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:03.897+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9220020020 con 0x7f9230101d50 2024-06-06T07:45:04.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.137+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f923002dca0 con 0x7f9230101d50 2024-06-06T07:45:04.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.230+0000 7f92267fc700 1 -- 172.21.15.87:0/978030567 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f922009be60 con 0x7f9230101d50 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.232+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 msgr2=0x7f921c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.232+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 0x7f921c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f922c00b5c0 tx=0x7f922c005960 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.232+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 msgr2=0x7f923019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.232+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f922000d980 tx=0x7f922000dc90 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 shutdown_connections 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f921c0a2ca0 0x7f921c0a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9230101390 0x7f923019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9230107400 0x7f9230197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 --2- 172.21.15.87:0/978030567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9230101d50 0x7f923019dc50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 >> 172.21.15.87:0/978030567 conn(0x7f9230078340 msgr2=0x7f92300fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:04.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 shutdown_connections 2024-06-06T07:45:04.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:04.233+0000 7f92376bf700 1 -- 172.21.15.87:0/978030567 wait complete. 2024-06-06T07:45:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:04 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:45:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:04 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:04 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T07:45:04.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:04 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:45:04.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:04 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:04.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:04 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T07:45:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:04 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:45:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:04 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:04 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T07:45:05.066 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-06T07:45:05.321 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:05.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:05 smithi087 ceph-mon[32276]: pgmap v336: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-06T07:45:05.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:05 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T07:45:05.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:05 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:05.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:05 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:05 smithi195 ceph-mon[37040]: pgmap v336: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-06T07:45:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:05 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T07:45:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:05 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:05 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:05 smithi181 ceph-mon[36790]: pgmap v336: 129 pgs: 129 active+clean; 582 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-06T07:45:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:05 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/978030567' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T07:45:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:05 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T07:45:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:05 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.103+0000 7f901aa9b700 1 -- 172.21.15.87:0/1466689694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 msgr2=0x7f90141029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.103+0000 7f901aa9b700 1 --2- 172.21.15.87:0/1466689694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f90141029d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90000099c0 tx=0x7f9000009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- 172.21.15.87:0/1466689694 shutdown_connections 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- 172.21.15.87:0/1466689694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90141073a0 0x7f9014107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- 172.21.15.87:0/1466689694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9014102f10 0x7f901410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- 172.21.15.87:0/1466689694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f90141029d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- 172.21.15.87:0/1466689694 >> 172.21.15.87:0/1466689694 conn(0x7f9014079650 msgr2=0x7f9014079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- 172.21.15.87:0/1466689694 shutdown_connections 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- 172.21.15.87:0/1466689694 wait complete. 2024-06-06T07:45:07.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 Processor -- start 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- start start 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9014102f10 0x7f9014074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90141073a0 0x7f9014075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90141127e0 con 0x7f9014102550 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9014112660 con 0x7f9014102f10 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.104+0000 7f901aa9b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9014112960 con 0x7f90141073a0 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53780/0 (socket says 172.21.15.87:53780) 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 -- 172.21.15.87:0/3233610958 learned_addr learned my addr 172.21.15.87:0/3233610958 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9019038700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90141073a0 0x7f9014075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9013fff700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9014102f10 0x7f9014074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:07.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 -- 172.21.15.87:0/3233610958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90141073a0 msgr2=0x7f9014075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90141073a0 0x7f9014075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 -- 172.21.15.87:0/3233610958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9014102f10 msgr2=0x7f9014074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9014102f10 0x7f9014074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 -- 172.21.15.87:0/3233610958 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90141a7130 con 0x7f9014102550 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9018837700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f900800b530 tx=0x7f900800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008015020 con 0x7f9014102550 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9008011d10 con 0x7f9014102550 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900800f2e0 con 0x7f9014102550 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90141a7420 con 0x7f9014102550 2024-06-06T07:45:07.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.105+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90141a7980 con 0x7f9014102550 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.106+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9014103b80 con 0x7f9014102550 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.107+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9008018020 con 0x7f9014102550 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.108+0000 7f9011ffb700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 0x7f90040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.108+0000 7f9013fff700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 0x7f90040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.108+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f90080cebc0 con 0x7f9014102550 2024-06-06T07:45:07.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.109+0000 7f9013fff700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 0x7f90040a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90140761d0 tx=0x7f9000005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:07.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.113+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f900801f020 con 0x7f9014102550 2024-06-06T07:45:07.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:07.357+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9014075c70 con 0x7f9014102550 2024-06-06T07:45:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:07 smithi195 ceph-mon[37040]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-06T07:45:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:07 smithi181 ceph-mon[36790]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-06T07:45:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:07 smithi087 ceph-mon[32276]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-06T07:45:08.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.245+0000 7f9011ffb700 1 -- 172.21.15.87:0/3233610958 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f900809b7c0 con 0x7f9014102550 2024-06-06T07:45:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 msgr2=0x7f90040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 0x7f90040a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90140761d0 tx=0x7f9000005fd0 comp rx=0 tx=0).stop 2024-06-06T07:45:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 msgr2=0x7f9014074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f900800b530 tx=0x7f900800b840 comp rx=0 tx=0).stop 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 shutdown_connections 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f90040a2ca0 0x7f90040a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90141073a0 0x7f9014075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9014102550 0x7f9014074790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 --2- 172.21.15.87:0/3233610958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9014102f10 0x7f9014074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 >> 172.21.15.87:0/3233610958 conn(0x7f9014079650 msgr2=0x7f90141005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 shutdown_connections 2024-06-06T07:45:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:08.248+0000 7f901aa9b700 1 -- 172.21.15.87:0/3233610958 wait complete. 2024-06-06T07:45:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:08 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T07:45:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:08 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T07:45:08.603 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:08 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T07:45:08.946 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-06T07:45:08.946 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-06T07:45:09.192 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:09 smithi195 ceph-mon[37040]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:45:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:09 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T07:45:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:09 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:09 smithi181 ceph-mon[36790]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:45:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:09 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T07:45:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:09 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:09 smithi087 ceph-mon[32276]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:09 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3233610958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:09 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:45:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 -- 172.21.15.87:0/1320423528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c1027a0 msgr2=0x7ff71c106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 --2- 172.21.15.87:0/1320423528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c1027a0 0x7ff71c106020 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7040099c0 tx=0x7ff704009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 -- 172.21.15.87:0/1320423528 shutdown_connections 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 --2- 172.21.15.87:0/1320423528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c106750 0x7ff71c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 --2- 172.21.15.87:0/1320423528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c101df0 0x7ff71c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 --2- 172.21.15.87:0/1320423528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c1027a0 0x7ff71c106020 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 -- 172.21.15.87:0/1320423528 >> 172.21.15.87:0/1320423528 conn(0x7ff71c077a00 msgr2=0x7ff71c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 -- 172.21.15.87:0/1320423528 shutdown_connections 2024-06-06T07:45:11.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.950+0000 7ff721e79700 1 -- 172.21.15.87:0/1320423528 wait complete. 2024-06-06T07:45:11.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 Processor -- start 2024-06-06T07:45:11.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 -- start start 2024-06-06T07:45:11.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c101df0 0x7ff71c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:11.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c1027a0 0x7ff71c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:11.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c111760 con 0x7ff71c106750 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1115e0 con 0x7ff71c101df0 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff721e79700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1118e0 con 0x7ff71c1027a0 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.951+0000 7ff71bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53800/0 (socket says 172.21.15.87:53800) 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 -- 172.21.15.87:0/1042545963 learned_addr learned my addr 172.21.15.87:0/1042545963 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71b7fe700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c101df0 0x7ff71c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff712ffd700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c1027a0 0x7ff71c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 -- 172.21.15.87:0/1042545963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c1027a0 msgr2=0x7ff71c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c1027a0 0x7ff71c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 -- 172.21.15.87:0/1042545963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c101df0 msgr2=0x7ff71c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:11.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c101df0 0x7ff71c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 -- 172.21.15.87:0/1042545963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c199400 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff71bfff700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b970 tx=0x7ff71400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714019070 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c1996f0 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71400ce10 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714007370 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.952+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a59d0 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.953+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c069180 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.954+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff71401c070 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.955+0000 7ff7197fa700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 0x7ff7080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.955+0000 7ff71b7fe700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 0x7ff7080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.955+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff7140cec50 con 0x7ff71c106750 2024-06-06T07:45:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.956+0000 7ff71b7fe700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 0x7ff7080a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009a10 tx=0x7ff70c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:11.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:10.959+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff71409b7f0 con 0x7ff71c106750 2024-06-06T07:45:11.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.184+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff71c102bf0 con 0x7ff7080a2c20 2024-06-06T07:45:11.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.213+0000 7ff7197fa700 1 -- 172.21.15.87:0/1042545963 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff70c009660 con 0x7ff7080a2c20 2024-06-06T07:45:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 msgr2=0x7ff7080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 0x7ff7080a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009a10 tx=0x7ff70c009190 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 msgr2=0x7ff71c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b970 tx=0x7ff71400bc80 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 shutdown_connections 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff7080a2c20 0x7ff7080a50e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff71c101df0 0x7ff71c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff71c106750 0x7ff71c198b20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 --2- 172.21.15.87:0/1042545963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff71c1027a0 0x7ff71c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.215+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 >> 172.21.15.87:0/1042545963 conn(0x7ff71c077a00 msgr2=0x7ff71c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.216+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 shutdown_connections 2024-06-06T07:45:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:11.216+0000 7ff721e79700 1 -- 172.21.15.87:0/1042545963 wait complete. 2024-06-06T07:45:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:11 smithi195 ceph-mon[37040]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:45:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:11 smithi181 ceph-mon[36790]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:45:11.592 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-mon[32276]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:45:11.592 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:45:11.189+0000 7f0ac50b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:45:11.592 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:45:11.189+0000 7f0ac50b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:45:11.593 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:45:11.189+0000 7f0ac50b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:45:11.593 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:45:11.189+0000 7f0ac50b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:45:11.593 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:45:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:45:11.189+0000 7f0ac50b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:45:11.947 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-06T07:45:12.189 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:12 smithi195 ceph-mon[37040]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:12 smithi181 ceph-mon[36790]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:12.587 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:12 smithi087 ceph-mon[32276]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:13 smithi195 ceph-mon[37040]: pgmap v340: 129 pgs: 129 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T07:45:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:13 smithi195 ceph-mon[37040]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T07:45:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:13 smithi181 ceph-mon[36790]: pgmap v340: 129 pgs: 129 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T07:45:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:13 smithi181 ceph-mon[36790]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T07:45:13.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:13 smithi087 ceph-mon[32276]: pgmap v340: 129 pgs: 129 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T07:45:13.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:13 smithi087 ceph-mon[32276]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 -- 172.21.15.87:0/2678626576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 msgr2=0x7f2ad810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/2678626576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad810e240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000b0a0 tx=0x7f2ad000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 -- 172.21.15.87:0/2678626576 shutdown_connections 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/2678626576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/2678626576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad810e240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/2678626576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 -- 172.21.15.87:0/2678626576 >> 172.21.15.87:0/2678626576 conn(0x7f2ad8078340 msgr2=0x7f2ad8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 -- 172.21.15.87:0/2678626576 shutdown_connections 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.958+0000 7f2ae06a2700 1 -- 172.21.15.87:0/2678626576 wait complete. 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 Processor -- start 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 -- start start 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81116f0 con 0x7f2ad8101d50 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8111570 con 0x7f2ad8107400 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ae06a2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8111870 con 0x7f2ad8101390 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2addc3d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2addc3d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53842/0 (socket says 172.21.15.87:53842) 2024-06-06T07:45:13.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2addc3d700 1 -- 172.21.15.87:0/3146120944 learned_addr learned my addr 172.21.15.87:0/3146120944 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2ade43e700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.959+0000 7f2adec3f700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 -- 172.21.15.87:0/3146120944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 msgr2=0x7f2ad819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 -- 172.21.15.87:0/3146120944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 msgr2=0x7f2ad8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 -- 172.21.15.87:0/3146120944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad81991c0 con 0x7f2ad8101d50 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2ade43e700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2adec3f700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2addc3d700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b530 tx=0x7f2ac800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8015020 con 0x7f2ad8101d50 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad81994b0 con 0x7f2ad8101d50 2024-06-06T07:45:13.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac8011d10 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8010e50 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.960+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81a5770 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.961+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad8069180 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.962+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2ac8018020 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.963+0000 7f2acf7fe700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 0x7f2ac40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.963+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2ac80cf150 con 0x7f2ad8101d50 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.963+0000 7f2ade43e700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 0x7f2ac40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.963+0000 7f2ade43e700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 0x7f2ac40a5310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4005fd0 tx=0x7f2ad4005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:13.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:13.967+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ac801f020 con 0x7f2ad8101d50 2024-06-06T07:45:14.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.195+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2ad806a170 con 0x7f2ac40a2e50 2024-06-06T07:45:14.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.222+0000 7f2acf7fe700 1 -- 172.21.15.87:0/3146120944 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ad806a170 con 0x7f2ac40a2e50 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 msgr2=0x7f2ac40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 0x7f2ac40a5310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4005fd0 tx=0x7f2ad4005e40 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 msgr2=0x7f2ad819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b530 tx=0x7f2ac800b8f0 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 shutdown_connections 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ac40a2e50 0x7f2ac40a5310 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ad8107400 0x7f2ad8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad8101390 0x7f2ad819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 --2- 172.21.15.87:0/3146120944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad8101d50 0x7f2ad819ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 >> 172.21.15.87:0/3146120944 conn(0x7f2ad8078340 msgr2=0x7f2ad81053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:14.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.224+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 shutdown_connections 2024-06-06T07:45:14.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:14.225+0000 7f2ae06a2700 1 -- 172.21.15.87:0/3146120944 wait complete. 2024-06-06T07:45:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:14 smithi181 ceph-mon[36790]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-06-06T07:45:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:14 smithi181 ceph-mon[36790]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:14.591 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:14 smithi087 ceph-mon[32276]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-06-06T07:45:14.591 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:14 smithi087 ceph-mon[32276]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:14 smithi195 ceph-mon[37040]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-06-06T07:45:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:14 smithi195 ceph-mon[37040]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:14.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-06T07:45:15.203 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- 172.21.15.87:0/1764319756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac60107550 msgr2=0x7fac60107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:16.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 --2- 172.21.15.87:0/1764319756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac60107550 0x7fac60107930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fac50009990 tx=0x7fac50009ca0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- 172.21.15.87:0/1764319756 shutdown_connections 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 --2- 172.21.15.87:0/1764319756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac601017f0 0x7fac6010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 --2- 172.21.15.87:0/1764319756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac60107550 0x7fac60107930 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 --2- 172.21.15.87:0/1764319756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60100e30 0x7fac601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- 172.21.15.87:0/1764319756 >> 172.21.15.87:0/1764319756 conn(0x7fac600766b0 msgr2=0x7fac60076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- 172.21.15.87:0/1764319756 shutdown_connections 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- 172.21.15.87:0/1764319756 wait complete. 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 Processor -- start 2024-06-06T07:45:16.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.851+0000 7fac66619700 1 -- start start 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac60100e30 0x7fac60198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 0x7fac6019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac60112890 con 0x7fac601017f0 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac60112710 con 0x7fac60107550 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac66619700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac60112a10 con 0x7fac60100e30 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53860/0 (socket says 172.21.15.87:53860) 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 -- 172.21.15.87:0/2477789136 learned_addr learned my addr 172.21.15.87:0/2477789136 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:16.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac64bb6700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 0x7fac6019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5ffff700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac60100e30 0x7fac60198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 -- 172.21.15.87:0/2477789136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac60100e30 msgr2=0x7fac60198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac60100e30 0x7fac60198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 -- 172.21.15.87:0/2477789136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 msgr2=0x7fac6019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 0x7fac6019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 -- 172.21.15.87:0/2477789136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6019dd80 con 0x7fac601017f0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac64bb6700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 0x7fac6019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.852+0000 7fac5f7fe700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b530 tx=0x7fac5400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.853+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54015020 con 0x7fac601017f0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.853+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac54011d10 con 0x7fac601017f0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.853+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6019e070 con 0x7fac601017f0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.853+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54010e50 con 0x7fac601017f0 2024-06-06T07:45:16.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.853+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac601a58f0 con 0x7fac601017f0 2024-06-06T07:45:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.854+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac60069180 con 0x7fac601017f0 2024-06-06T07:45:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.855+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac54018020 con 0x7fac601017f0 2024-06-06T07:45:16.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.856+0000 7fac5d7fa700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 0x7fac480a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.856+0000 7fac5ffff700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 0x7fac480a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.857+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fac540cf220 con 0x7fac601017f0 2024-06-06T07:45:16.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.857+0000 7fac5ffff700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 0x7fac480a5310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac50005f50 tx=0x7fac50005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:16.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:16.860+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac5401f020 con 0x7fac601017f0 2024-06-06T07:45:17.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.085+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fac600628d0 con 0x7fac480a2e50 2024-06-06T07:45:17.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.090+0000 7fac5d7fa700 1 -- 172.21.15.87:0/2477789136 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fac600628d0 con 0x7fac480a2e50 2024-06-06T07:45:17.091 INFO:teuthology.orchestra.run.smithi087.stdout:/volumes/qa/sv_0/ce1dc7b8-63d2-44aa-acca-7a99706b2068 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 msgr2=0x7fac480a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 0x7fac480a5310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac50005f50 tx=0x7fac50005e80 comp rx=0 tx=0).stop 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 msgr2=0x7fac60199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b530 tx=0x7fac5400b8f0 comp rx=0 tx=0).stop 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 shutdown_connections 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fac480a2e50 0x7fac480a5310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac60107550 0x7fac6019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:17.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac60100e30 0x7fac60198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:17.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 --2- 172.21.15.87:0/2477789136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac601017f0 0x7fac60199300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:17.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 >> 172.21.15.87:0/2477789136 conn(0x7fac600766b0 msgr2=0x7fac600ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:17.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 shutdown_connections 2024-06-06T07:45:17.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:17.093+0000 7fac66619700 1 -- 172.21.15.87:0/2477789136 wait complete. 2024-06-06T07:45:17.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:17 smithi087 ceph-mon[32276]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T07:45:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:17 smithi195 ceph-mon[37040]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T07:45:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:17 smithi181 ceph-mon[36790]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T07:45:17.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-06T07:45:18.060 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:18 smithi195 ceph-mon[37040]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:18.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:18 smithi087 ceph-mon[32276]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:18 smithi181 ceph-mon[36790]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:19.499 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:19 smithi087 ceph-mon[32276]: pgmap v343: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-06T07:45:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:19 smithi195 ceph-mon[37040]: pgmap v343: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-06T07:45:19.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:19 smithi181 ceph-mon[36790]: pgmap v343: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 -- 172.21.15.87:0/900979801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c102900 msgr2=0x7ff61c10e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 --2- 172.21.15.87:0/900979801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c102900 0x7ff61c10e260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff61400b0a0 tx=0x7ff61400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 -- 172.21.15.87:0/900979801 shutdown_connections 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 --2- 172.21.15.87:0/900979801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c1063c0 0x7ff61c1067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 --2- 172.21.15.87:0/900979801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c102900 0x7ff61c10e260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 --2- 172.21.15.87:0/900979801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c101f40 0x7ff61c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 -- 172.21.15.87:0/900979801 >> 172.21.15.87:0/900979801 conn(0x7ff61c076730 msgr2=0x7ff61c076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 -- 172.21.15.87:0/900979801 shutdown_connections 2024-06-06T07:45:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.764+0000 7ff621487700 1 -- 172.21.15.87:0/900979801 wait complete. 2024-06-06T07:45:19.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.765+0000 7ff621487700 1 Processor -- start 2024-06-06T07:45:19.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.765+0000 7ff621487700 1 -- start start 2024-06-06T07:45:19.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.765+0000 7ff621487700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.765+0000 7ff621487700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 0x7ff61c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.765+0000 7ff621487700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c1063c0 0x7ff61c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff621487700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c111740 con 0x7ff61c101f40 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff621487700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1115c0 con 0x7ff61c102900 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff621487700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1118c0 con 0x7ff61c1063c0 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45248/0 (socket says 172.21.15.87:45248) 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 -- 172.21.15.87:0/2609758731 learned_addr learned my addr 172.21.15.87:0/2609758731 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61b7fe700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c1063c0 0x7ff61c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61a7fc700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 0x7ff61c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 -- 172.21.15.87:0/2609758731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c1063c0 msgr2=0x7ff61c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c1063c0 0x7ff61c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 -- 172.21.15.87:0/2609758731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 msgr2=0x7ff61c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 0x7ff61c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 -- 172.21.15.87:0/2609758731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c198100 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61a7fc700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 0x7ff61c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff61affd700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c9b0 tx=0x7ff60c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c007b00 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c1983f0 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60c007ca0 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.766+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c018730 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.767+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a46b0 con 0x7ff61c101f40 2024-06-06T07:45:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.768+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c06a6f0 con 0x7ff61c101f40 2024-06-06T07:45:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.769+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff60c0188d0 con 0x7ff61c101f40 2024-06-06T07:45:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.769+0000 7ff603fff700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 0x7ff6040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.769+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff60c0cf4c0 con 0x7ff61c101f40 2024-06-06T07:45:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.769+0000 7ff61a7fc700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 0x7ff6040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.770+0000 7ff61a7fc700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 0x7ff6040a51f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff61c076c90 tx=0x7ff610008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:19.774+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff60c014030 con 0x7ff61c101f40 2024-06-06T07:45:20.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.000+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff61c0628d0 con 0x7ff6040a2d30 2024-06-06T07:45:20.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.026+0000 7ff603fff700 1 -- 172.21.15.87:0/2609758731 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff61c0628d0 con 0x7ff6040a2d30 2024-06-06T07:45:20.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 msgr2=0x7ff6040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 0x7ff6040a51f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff61c076c90 tx=0x7ff610008040 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 msgr2=0x7ff61c19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c9b0 tx=0x7ff60c00ccc0 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 shutdown_connections 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff6040a2d30 0x7ff6040a51f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff61c1063c0 0x7ff61c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff61c101f40 0x7ff61c19d7e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 --2- 172.21.15.87:0/2609758731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61c102900 0x7ff61c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.028+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 >> 172.21.15.87:0/2609758731 conn(0x7ff61c076730 msgr2=0x7ff61c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.029+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 shutdown_connections 2024-06-06T07:45:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:20.029+0000 7ff621487700 1 -- 172.21.15.87:0/2609758731 wait complete. 2024-06-06T07:45:20.758 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-06T07:45:20.996 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:21.458 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:21 smithi087 ceph-mon[32276]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:21.459 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:21 smithi087 ceph-mon[32276]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T07:45:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:21 smithi195 ceph-mon[37040]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:21 smithi195 ceph-mon[37040]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T07:45:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:21 smithi181 ceph-mon[36790]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:21 smithi181 ceph-mon[36790]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 -- 172.21.15.87:0/3623466027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 msgr2=0x7f4910107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 --2- 172.21.15.87:0/3623466027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f4910107110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49040099c0 tx=0x7f4904009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 -- 172.21.15.87:0/3623466027 shutdown_connections 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 --2- 172.21.15.87:0/3623466027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f4910102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 --2- 172.21.15.87:0/3623466027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 0x7f491010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 --2- 172.21.15.87:0/3623466027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f4910107110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 -- 172.21.15.87:0/3623466027 >> 172.21.15.87:0/3623466027 conn(0x7f49100795f0 msgr2=0x7f4910079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 -- 172.21.15.87:0/3623466027 shutdown_connections 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.758+0000 7f49184b4700 1 -- 172.21.15.87:0/3623466027 wait complete. 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 Processor -- start 2024-06-06T07:45:22.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 -- start start 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f491019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 0x7f4910198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49101127d0 con 0x7f49101030f0 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4910112650 con 0x7f4910102740 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f49184b4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4910112950 con 0x7f4910107840 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45262/0 (socket says 172.21.15.87:45262) 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 -- 172.21.15.87:0/3135352193 learned_addr learned my addr 172.21.15.87:0/3135352193 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4916250700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f491019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4916a51700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 0x7f4910198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 -- 172.21.15.87:0/3135352193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 msgr2=0x7f4910198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 0x7f4910198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 -- 172.21.15.87:0/3135352193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 msgr2=0x7f491019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:22.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f491019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4915a4f700 1 -- 172.21.15.87:0/3135352193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101991d0 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.759+0000 7f4916250700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f491019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f4915a4f700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49040059d0 tx=0x7f49040038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904027070 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4904004050 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490400f460 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4910199460 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.760+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a5730 con 0x7f49101030f0 2024-06-06T07:45:22.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.761+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4910069180 con 0x7f49101030f0 2024-06-06T07:45:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.762+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f490402a070 con 0x7f49101030f0 2024-06-06T07:45:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.763+0000 7f490f7fe700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 0x7f48f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.763+0000 7f4916250700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 0x7f48f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.763+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f49040d0b40 con 0x7f49101030f0 2024-06-06T07:45:22.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.763+0000 7f4916250700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 0x7f48f80a51f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4910107400 tx=0x7f4908009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:22.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.766+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4904031020 con 0x7f49101030f0 2024-06-06T07:45:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.993+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f49100628d0 con 0x7f48f80a2d30 2024-06-06T07:45:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:22.998+0000 7f490f7fe700 1 -- 172.21.15.87:0/3135352193 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f49100628d0 con 0x7f48f80a2d30 2024-06-06T07:45:22.998 INFO:teuthology.orchestra.run.smithi087.stdout:/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 2024-06-06T07:45:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 msgr2=0x7f48f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 0x7f48f80a51f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4910107400 tx=0x7f4908009040 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 msgr2=0x7f491019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49040059d0 tx=0x7f49040038b0 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 shutdown_connections 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f48f80a2d30 0x7f48f80a51f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910102740 0x7f491019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4910107840 0x7f4910198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 --2- 172.21.15.87:0/3135352193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49101030f0 0x7f491019eda0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 >> 172.21.15.87:0/3135352193 conn(0x7f49100795f0 msgr2=0x7f4910100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 shutdown_connections 2024-06-06T07:45:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:23.000+0000 7f49184b4700 1 -- 172.21.15.87:0/3135352193 wait complete. 2024-06-06T07:45:23.311 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:23 smithi087 ceph-mon[32276]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-06T07:45:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:23 smithi195 ceph-mon[37040]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-06T07:45:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:23 smithi181 ceph-mon[36790]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-06T07:45:23.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:45:24.004 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:24.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:24 smithi087 ceph-mon[32276]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:24 smithi195 ceph-mon[37040]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:24 smithi181 ceph-mon[36790]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:45:25.482 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:25 smithi087 ceph-mon[32276]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-06-06T07:45:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:25 smithi195 ceph-mon[37040]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-06-06T07:45:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:25 smithi181 ceph-mon[36790]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 -- 172.21.15.87:0/4281935551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 msgr2=0x7fc9d0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 --2- 172.21.15.87:0/4281935551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d0107110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80099c0 tx=0x7fc9b8009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 -- 172.21.15.87:0/4281935551 shutdown_connections 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 --2- 172.21.15.87:0/4281935551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0107840 0x7fc9d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 --2- 172.21.15.87:0/4281935551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0102740 0x7fc9d0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 --2- 172.21.15.87:0/4281935551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d0107110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.715+0000 7fc9d5663700 1 -- 172.21.15.87:0/4281935551 >> 172.21.15.87:0/4281935551 conn(0x7fc9d00795f0 msgr2=0x7fc9d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:25.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.716+0000 7fc9d5663700 1 -- 172.21.15.87:0/4281935551 shutdown_connections 2024-06-06T07:45:25.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.716+0000 7fc9d5663700 1 -- 172.21.15.87:0/4281935551 wait complete. 2024-06-06T07:45:25.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.716+0000 7fc9d5663700 1 Processor -- start 2024-06-06T07:45:25.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.716+0000 7fc9d5663700 1 -- start start 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0102740 0x7fc9d019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 0x7fc9d0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0112750 con 0x7fc9d01030f0 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01125d0 con 0x7fc9d0102740 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01128d0 con 0x7fc9d0107840 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45280/0 (socket says 172.21.15.87:45280) 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 -- 172.21.15.87:0/3407797897 learned_addr learned my addr 172.21.15.87:0/3407797897 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ceffd700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0102740 0x7fc9d019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9cf7fe700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 0x7fc9d0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 -- 172.21.15.87:0/3407797897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 msgr2=0x7fc9d0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:25.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 0x7fc9d0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 -- 172.21.15.87:0/3407797897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0102740 msgr2=0x7fc9d019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0102740 0x7fc9d019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 -- 172.21.15.87:0/3407797897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01990b0 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9cf7fe700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 0x7fc9d0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9ce7fc700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006010 tx=0x7fc9b8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8027070 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d0199340 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b80040d0 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800f460 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.717+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a5660 con 0x7fc9d01030f0 2024-06-06T07:45:25.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.718+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d0103bd0 con 0x7fc9d01030f0 2024-06-06T07:45:25.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.720+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc9b802a070 con 0x7fc9d01030f0 2024-06-06T07:45:25.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.720+0000 7fc9c7fff700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 0x7fc9bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:25.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.720+0000 7fc9ceffd700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 0x7fc9bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:25.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.720+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fc9b80d0bd0 con 0x7fc9d01030f0 2024-06-06T07:45:25.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.721+0000 7fc9ceffd700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 0x7fc9bc0a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0005ea0 tx=0x7fc9c0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:25.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.724+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc9b8031020 con 0x7fc9d01030f0 2024-06-06T07:45:25.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.964+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9d0199e80 con 0x7fc9d01030f0 2024-06-06T07:45:25.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.965+0000 7fc9c7fff700 1 -- 172.21.15.87:0/3407797897 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7fc9b8013070 con 0x7fc9d01030f0 2024-06-06T07:45:25.965 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:25.965 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 msgr2=0x7fc9bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 0x7fc9bc0a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0005ea0 tx=0x7fc9c0005e30 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 msgr2=0x7fc9d019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006010 tx=0x7fc9b8005640 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 shutdown_connections 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc9bc0a2cb0 0x7fc9bc0a5170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d0102740 0x7fc9d019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0107840 0x7fc9d0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 --2- 172.21.15.87:0/3407797897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9d01030f0 0x7fc9d019ecd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.967+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 >> 172.21.15.87:0/3407797897 conn(0x7fc9d00795f0 msgr2=0x7fc9d01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:25.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.968+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 shutdown_connections 2024-06-06T07:45:25.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:25.968+0000 7fc9d5663700 1 -- 172.21.15.87:0/3407797897 wait complete. 2024-06-06T07:45:25.969 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:45:26.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:26 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3407797897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:45:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:26 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3407797897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:45:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:26 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3407797897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:45:26.801 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:45:27.043 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:27.430 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:27 smithi087 ceph-mon[32276]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-06-06T07:45:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:27 smithi195 ceph-mon[37040]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-06-06T07:45:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:27 smithi181 ceph-mon[36790]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-06-06T07:45:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 -- 172.21.15.87:0/2728378479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 msgr2=0x7fc164102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:28.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 --2- 172.21.15.87:0/2728378479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc164102b80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1540099c0 tx=0x7fc154009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 -- 172.21.15.87:0/2728378479 shutdown_connections 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 --2- 172.21.15.87:0/2728378479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 0x7fc16410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 --2- 172.21.15.87:0/2728378479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc164102b80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 --2- 172.21.15.87:0/2728378479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 0x7fc164107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.873+0000 7fc16b950700 1 -- 172.21.15.87:0/2728378479 >> 172.21.15.87:0/2728378479 conn(0x7fc164079520 msgr2=0x7fc164079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 -- 172.21.15.87:0/2728378479 shutdown_connections 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 -- 172.21.15.87:0/2728378479 wait complete. 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 Processor -- start 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 -- start start 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 0x7fc16419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:28.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.874+0000 7fc16b950700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 0x7fc164198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc16b950700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1641127b0 con 0x7fc1641027a0 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc16b950700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc164112630 con 0x7fc164103150 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc16b950700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc164112930 con 0x7fc164107870 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33876/0 (socket says 172.21.15.87:33876) 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 -- 172.21.15.87:0/1299452117 learned_addr learned my addr 172.21.15.87:0/1299452117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc168eeb700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 0x7fc16419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc169eed700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 0x7fc164198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 -- 172.21.15.87:0/1299452117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 msgr2=0x7fc164198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 0x7fc164198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 -- 172.21.15.87:0/1299452117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 msgr2=0x7fc16419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 0x7fc16419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 -- 172.21.15.87:0/1299452117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc164199170 con 0x7fc1641027a0 2024-06-06T07:45:28.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc1696ec700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154026070 con 0x7fc1641027a0 2024-06-06T07:45:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc154004cd0 con 0x7fc1641027a0 2024-06-06T07:45:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc164199400 con 0x7fc1641027a0 2024-06-06T07:45:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154003ba0 con 0x7fc1641027a0 2024-06-06T07:45:28.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.875+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1641a56d0 con 0x7fc1641027a0 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.876+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc164069180 con 0x7fc1641027a0 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.878+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc154029070 con 0x7fc1641027a0 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.878+0000 7fc15a7fc700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 0x7fc1500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.878+0000 7fc168eeb700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 0x7fc1500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.878+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fc1540d01d0 con 0x7fc1641027a0 2024-06-06T07:45:28.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.879+0000 7fc168eeb700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 0x7fc1500a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc164108c90 tx=0x7fc16000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:28.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:28.882+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc154030020 con 0x7fc1641027a0 2024-06-06T07:45:29.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.076+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc164199ef0 con 0x7fc1641027a0 2024-06-06T07:45:29.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.078+0000 7fc15a7fc700 1 -- 172.21.15.87:0/1299452117 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24625 (secure 0 0 0) 0x7fc1540bc070 con 0x7fc1641027a0 2024-06-06T07:45:29.078 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:29.079 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":97,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:47.243465+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 msgr2=0x7fc1500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 0x7fc1500a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc164108c90 tx=0x7fc16000b410 comp rx=0 tx=0).stop 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 msgr2=0x7fc16419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154009f80 comp rx=0 tx=0).stop 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 shutdown_connections 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1500a2d30 0x7fc1500a51f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:29.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc164107870 0x7fc164198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1641027a0 0x7fc16419e800 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 --2- 172.21.15.87:0/1299452117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc164103150 0x7fc16419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 >> 172.21.15.87:0/1299452117 conn(0x7fc164079520 msgr2=0x7fc1641004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 shutdown_connections 2024-06-06T07:45:29.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:29.080+0000 7fc16b950700 1 -- 172.21.15.87:0/1299452117 wait complete. 2024-06-06T07:45:29.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:29 smithi087 ceph-mon[32276]: pgmap v348: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-06-06T07:45:29.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1299452117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:29.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:29 smithi195 ceph-mon[37040]: pgmap v348: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-06-06T07:45:29.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1299452117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:29 smithi181 ceph-mon[36790]: pgmap v348: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-06-06T07:45:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1299452117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:29.873 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T07:45:29.873 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:45:30.114 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:31.292 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:31 smithi087 ceph-mon[32276]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-06T07:45:31.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:31 smithi195 ceph-mon[37040]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-06T07:45:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:31 smithi181 ceph-mon[36790]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 -- 172.21.15.87:0/2647713205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 msgr2=0x7f4cf010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2647713205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf010f320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800b0a0 tx=0x7f4ce800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 -- 172.21.15.87:0/2647713205 shutdown_connections 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2647713205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0108500 0x7f4cf01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2647713205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf010f320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2647713205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf006a170 0x7f4cf006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 -- 172.21.15.87:0/2647713205 >> 172.21.15.87:0/2647713205 conn(0x7f4cf0078340 msgr2=0x7f4cf0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 -- 172.21.15.87:0/2647713205 shutdown_connections 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.854+0000 7f4cf5906700 1 -- 172.21.15.87:0/2647713205 wait complete. 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.855+0000 7f4cf5906700 1 Processor -- start 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 -- start start 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf006a170 0x7f4cf019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf0108500 0x7f4cf01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01127e0 con 0x7f4cf006ab30 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0112660 con 0x7f4cf006a170 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cf5906700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0112960 con 0x7f4cf0108500 2024-06-06T07:45:31.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33906/0 (socket says 172.21.15.87:33906) 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 -- 172.21.15.87:0/2838216981 learned_addr learned my addr 172.21.15.87:0/2838216981 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4ceeffd700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf006a170 0x7f4cf019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 -- 172.21.15.87:0/2838216981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf0108500 msgr2=0x7f4cf01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf0108500 0x7f4cf01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 -- 172.21.15.87:0/2838216981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf006a170 msgr2=0x7f4cf019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf006a170 0x7f4cf019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 -- 172.21.15.87:0/2838216981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf0199140 con 0x7f4cf006ab30 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.856+0000 7f4cee7fc700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b530 tx=0x7f4ce400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.857+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4015020 con 0x7f4cf006ab30 2024-06-06T07:45:31.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.857+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf0199430 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.857+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4011d10 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.857+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400f2e0 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.857+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01a56a0 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.858+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf0069180 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.859+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4ce4018020 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.859+0000 7f4cf4904700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 0x7f4cd80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.859+0000 7f4ceeffd700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 0x7f4cd80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.859+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f4ce40cece0 con 0x7f4cf006ab30 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.860+0000 7f4ceeffd700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 0x7f4cd80a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0107e00 tx=0x7f4ce000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:31.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:31.864+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ce401f020 con 0x7f4cf006ab30 2024-06-06T07:45:32.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.056+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cf00628d0 con 0x7f4cd80a2cb0 2024-06-06T07:45:32.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.066+0000 7f4cf4904700 1 -- 172.21.15.87:0/2838216981 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f4cf00628d0 con 0x7f4cd80a2cb0 2024-06-06T07:45:32.067 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 msgr2=0x7f4cd80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 0x7f4cd80a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0107e00 tx=0x7f4ce000a380 comp rx=0 tx=0).stop 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 msgr2=0x7f4cf019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b530 tx=0x7f4ce400b840 comp rx=0 tx=0).stop 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 shutdown_connections 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4cd80a2cb0 0x7f4cd80a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf006a170 0x7f4cf019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:32.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cf0108500 0x7f4cf01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 --2- 172.21.15.87:0/2838216981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf006ab30 0x7f4cf019ed10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.069+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 >> 172.21.15.87:0/2838216981 conn(0x7f4cf0078340 msgr2=0x7f4cf00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.070+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 shutdown_connections 2024-06-06T07:45:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:32.070+0000 7f4cf5906700 1 -- 172.21.15.87:0/2838216981 wait complete. 2024-06-06T07:45:32.076 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:45:32.903 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-06-06T07:45:30.160749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651659,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":305,"num_read_kb":265,"num_write":347,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384980,"kb_used_data":15028,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556844,"statfs":{"total":1151940427776,"available":1151546208256,"internally_reserved":0,"allocated":15388672,"data_stored":8538630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27696,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":58,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003557"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253895+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:44:47.253895+0000","last_peered":"2024-06-06T07:44:47.253895+0000","last_clean":"2024-06-06T07:44:47.253895+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:44:47.253895+0000","last_undegraded":"2024-06-06T07:44:47.253895+0000","last_fullsized":"2024-06-06T07:44:47.253895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249504+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:44:47.249504+0000","last_peered":"2024-06-06T07:44:47.249504+0000","last_clean":"2024-06-06T07:44:47.249504+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:44:47.249504+0000","last_undegraded":"2024-06-06T07:44:47.249504+0000","last_fullsized":"2024-06-06T07:44:47.249504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249348+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:44:47.249348+0000","last_peered":"2024-06-06T07:44:47.249348+0000","last_clean":"2024-06-06T07:44:47.249348+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:44:47.249348+0000","last_undegraded":"2024-06-06T07:44:47.249348+0000","last_fullsized":"2024-06-06T07:44:47.249348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200667+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:44:48.200667+0000","last_peered":"2024-06-06T07:44:48.200667+0000","last_clean":"2024-06-06T07:44:48.200667+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:44:48.200667+0000","last_undegraded":"2024-06-06T07:44:48.200667+0000","last_fullsized":"2024-06-06T07:44:48.200667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249006+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:44:47.249006+0000","last_peered":"2024-06-06T07:44:47.249006+0000","last_clean":"2024-06-06T07:44:47.249006+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:44:47.249006+0000","last_undegraded":"2024-06-06T07:44:47.249006+0000","last_fullsized":"2024-06-06T07:44:47.249006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251412+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:44:47.251412+0000","last_peered":"2024-06-06T07:44:47.251412+0000","last_clean":"2024-06-06T07:44:47.251412+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:44:47.251412+0000","last_undegraded":"2024-06-06T07:44:47.251412+0000","last_fullsized":"2024-06-06T07:44:47.251412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249222+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:44:47.249222+0000","last_peered":"2024-06-06T07:44:47.249222+0000","last_clean":"2024-06-06T07:44:47.249222+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:44:47.249222+0000","last_undegraded":"2024-06-06T07:44:47.249222+0000","last_fullsized":"2024-06-06T07:44:47.249222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305125+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:44:47.305125+0000","last_peered":"2024-06-06T07:44:47.305125+0000","last_clean":"2024-06-06T07:44:47.305125+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:44:47.305125+0000","last_undegraded":"2024-06-06T07:44:47.305125+0000","last_fullsized":"2024-06-06T07:44:47.305125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.256770+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:44:47.256770+0000","last_peered":"2024-06-06T07:44:47.256770+0000","last_clean":"2024-06-06T07:44:47.256770+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:44:47.256770+0000","last_undegraded":"2024-06-06T07:44:47.256770+0000","last_fullsized":"2024-06-06T07:44:47.256770+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251703+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:44:47.251703+0000","last_peered":"2024-06-06T07:44:47.251703+0000","last_clean":"2024-06-06T07:44:47.251703+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:44:47.251703+0000","last_undegraded":"2024-06-06T07:44:47.251703+0000","last_fullsized":"2024-06-06T07:44:47.251703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249755+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:44:47.249755+0000","last_peered":"2024-06-06T07:44:47.249755+0000","last_clean":"2024-06-06T07:44:47.249755+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:44:47.249755+0000","last_undegraded":"2024-06-06T07:44:47.249755+0000","last_fullsized":"2024-06-06T07:44:47.249755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200653+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:44:48.200653+0000","last_peered":"2024-06-06T07:44:48.200653+0000","last_clean":"2024-06-06T07:44:48.200653+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:44:48.200653+0000","last_undegraded":"2024-06-06T07:44:48.200653+0000","last_fullsized":"2024-06-06T07:44:48.200653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259765+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:44:47.259765+0000","last_peered":"2024-06-06T07:44:47.259765+0000","last_clean":"2024-06-06T07:44:47.259765+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:44:47.259765+0000","last_undegraded":"2024-06-06T07:44:47.259765+0000","last_fullsized":"2024-06-06T07:44:47.259765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249177+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:44:47.249177+0000","last_peered":"2024-06-06T07:44:47.249177+0000","last_clean":"2024-06-06T07:44:47.249177+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:44:47.249177+0000","last_undegraded":"2024-06-06T07:44:47.249177+0000","last_fullsized":"2024-06-06T07:44:47.249177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251796+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:44:47.251796+0000","last_peered":"2024-06-06T07:44:47.251796+0000","last_clean":"2024-06-06T07:44:47.251796+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:44:47.251796+0000","last_undegraded":"2024-06-06T07:44:47.251796+0000","last_fullsized":"2024-06-06T07:44:47.251796+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253426+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:44:47.253426+0000","last_peered":"2024-06-06T07:44:47.253426+0000","last_clean":"2024-06-06T07:44:47.253426+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:44:47.253426+0000","last_undegraded":"2024-06-06T07:44:47.253426+0000","last_fullsized":"2024-06-06T07:44:47.253426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253668+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:44:47.253668+0000","last_peered":"2024-06-06T07:44:47.253668+0000","last_clean":"2024-06-06T07:44:47.253668+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:44:47.253668+0000","last_undegraded":"2024-06-06T07:44:47.253668+0000","last_fullsized":"2024-06-06T07:44:47.253668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246765+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:44:47.246765+0000","last_peered":"2024-06-06T07:44:47.246765+0000","last_clean":"2024-06-06T07:44:47.246765+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:44:47.246765+0000","last_undegraded":"2024-06-06T07:44:47.246765+0000","last_fullsized":"2024-06-06T07:44:47.246765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248017+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:44:47.248017+0000","last_peered":"2024-06-06T07:44:47.248017+0000","last_clean":"2024-06-06T07:44:47.248017+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:44:47.248017+0000","last_undegraded":"2024-06-06T07:44:47.248017+0000","last_fullsized":"2024-06-06T07:44:47.248017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248143+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:44:47.248143+0000","last_peered":"2024-06-06T07:44:47.248143+0000","last_clean":"2024-06-06T07:44:47.248143+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:44:47.248143+0000","last_undegraded":"2024-06-06T07:44:47.248143+0000","last_fullsized":"2024-06-06T07:44:47.248143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.233329+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:45:02.233329+0000","last_peered":"2024-06-06T07:45:02.233329+0000","last_clean":"2024-06-06T07:45:02.233329+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:45:02.233329+0000","last_undegraded":"2024-06-06T07:45:02.233329+0000","last_fullsized":"2024-06-06T07:45:02.233329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305425+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:44:47.305425+0000","last_peered":"2024-06-06T07:44:47.305425+0000","last_clean":"2024-06-06T07:44:47.305425+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:44:47.305425+0000","last_undegraded":"2024-06-06T07:44:47.305425+0000","last_fullsized":"2024-06-06T07:44:47.305425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200270+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:44:48.200270+0000","last_peered":"2024-06-06T07:44:48.200270+0000","last_clean":"2024-06-06T07:44:48.200270+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:44:48.200270+0000","last_undegraded":"2024-06-06T07:44:48.200270+0000","last_fullsized":"2024-06-06T07:44:48.200270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305743+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:44:47.305743+0000","last_peered":"2024-06-06T07:44:47.305743+0000","last_clean":"2024-06-06T07:44:47.305743+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:44:47.305743+0000","last_undegraded":"2024-06-06T07:44:47.305743+0000","last_fullsized":"2024-06-06T07:44:47.305743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249423+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:44:47.249423+0000","last_peered":"2024-06-06T07:44:47.249423+0000","last_clean":"2024-06-06T07:44:47.249423+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:44:47.249423+0000","last_undegraded":"2024-06-06T07:44:47.249423+0000","last_fullsized":"2024-06-06T07:44:47.249423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200244+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:44:48.200244+0000","last_peered":"2024-06-06T07:44:48.200244+0000","last_clean":"2024-06-06T07:44:48.200244+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:44:48.200244+0000","last_undegraded":"2024-06-06T07:44:48.200244+0000","last_fullsized":"2024-06-06T07:44:48.200244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232105+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:45:02.232105+0000","last_peered":"2024-06-06T07:45:02.232105+0000","last_clean":"2024-06-06T07:45:02.232105+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:45:02.232105+0000","last_undegraded":"2024-06-06T07:45:02.232105+0000","last_fullsized":"2024-06-06T07:45:02.232105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253227+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:44:47.253227+0000","last_peered":"2024-06-06T07:44:47.253227+0000","last_clean":"2024-06-06T07:44:47.253227+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:44:47.253227+0000","last_undegraded":"2024-06-06T07:44:47.253227+0000","last_fullsized":"2024-06-06T07:44:47.253227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.199895+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:44:48.199895+0000","last_peered":"2024-06-06T07:44:48.199895+0000","last_clean":"2024-06-06T07:44:48.199895+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:44:48.199895+0000","last_undegraded":"2024-06-06T07:44:48.199895+0000","last_fullsized":"2024-06-06T07:44:48.199895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249592+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:44:47.249592+0000","last_peered":"2024-06-06T07:44:47.249592+0000","last_clean":"2024-06-06T07:44:47.249592+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:44:47.249592+0000","last_undegraded":"2024-06-06T07:44:47.249592+0000","last_fullsized":"2024-06-06T07:44:47.249592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.257530+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:44:47.257530+0000","last_peered":"2024-06-06T07:44:47.257530+0000","last_clean":"2024-06-06T07:44:47.257530+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:44:47.257530+0000","last_undegraded":"2024-06-06T07:44:47.257530+0000","last_fullsized":"2024-06-06T07:44:47.257530+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247748+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:44:47.247748+0000","last_peered":"2024-06-06T07:44:47.247748+0000","last_clean":"2024-06-06T07:44:47.247748+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:44:47.247748+0000","last_undegraded":"2024-06-06T07:44:47.247748+0000","last_fullsized":"2024-06-06T07:44:47.247748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":74,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:27.281263+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:45:27.281263+0000","last_peered":"2024-06-06T07:45:27.281263+0000","last_clean":"2024-06-06T07:45:27.281263+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:45:27.281263+0000","last_undegraded":"2024-06-06T07:45:27.281263+0000","last_fullsized":"2024-06-06T07:45:27.281263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249256+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:44:47.249256+0000","last_peered":"2024-06-06T07:44:47.249256+0000","last_clean":"2024-06-06T07:44:47.249256+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:44:47.249256+0000","last_undegraded":"2024-06-06T07:44:47.249256+0000","last_fullsized":"2024-06-06T07:44:47.249256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248099+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:44:47.248099+0000","last_peered":"2024-06-06T07:44:47.248099+0000","last_clean":"2024-06-06T07:44:47.248099+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:44:47.248099+0000","last_undegraded":"2024-06-06T07:44:47.248099+0000","last_fullsized":"2024-06-06T07:44:47.248099+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231548+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:45:02.231548+0000","last_peered":"2024-06-06T07:45:02.231548+0000","last_clean":"2024-06-06T07:45:02.231548+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:45:02.231548+0000","last_undegraded":"2024-06-06T07:45:02.231548+0000","last_fullsized":"2024-06-06T07:45:02.231548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252280+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:44:47.252280+0000","last_peered":"2024-06-06T07:44:47.252280+0000","last_clean":"2024-06-06T07:44:47.252280+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:44:47.252280+0000","last_undegraded":"2024-06-06T07:44:47.252280+0000","last_fullsized":"2024-06-06T07:44:47.252280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253533+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:44:47.253533+0000","last_peered":"2024-06-06T07:44:47.253533+0000","last_clean":"2024-06-06T07:44:47.253533+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:44:47.253533+0000","last_undegraded":"2024-06-06T07:44:47.253533+0000","last_fullsized":"2024-06-06T07:44:47.253533+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231456+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:45:02.231456+0000","last_peered":"2024-06-06T07:45:02.231456+0000","last_clean":"2024-06-06T07:45:02.231456+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:45:02.231456+0000","last_undegraded":"2024-06-06T07:45:02.231456+0000","last_fullsized":"2024-06-06T07:45:02.231456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305230+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:44:47.305230+0000","last_peered":"2024-06-06T07:44:47.305230+0000","last_clean":"2024-06-06T07:44:47.305230+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:44:47.305230+0000","last_undegraded":"2024-06-06T07:44:47.305230+0000","last_fullsized":"2024-06-06T07:44:47.305230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249461+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:44:47.249461+0000","last_peered":"2024-06-06T07:44:47.249461+0000","last_clean":"2024-06-06T07:44:47.249461+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:44:47.249461+0000","last_undegraded":"2024-06-06T07:44:47.249461+0000","last_fullsized":"2024-06-06T07:44:47.249461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252930+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:44:47.252930+0000","last_peered":"2024-06-06T07:44:47.252930+0000","last_clean":"2024-06-06T07:44:47.252930+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:44:47.252930+0000","last_undegraded":"2024-06-06T07:44:47.252930+0000","last_fullsized":"2024-06-06T07:44:47.252930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246470+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:44:47.246470+0000","last_peered":"2024-06-06T07:44:47.246470+0000","last_clean":"2024-06-06T07:44:47.246470+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:44:47.246470+0000","last_undegraded":"2024-06-06T07:44:47.246470+0000","last_fullsized":"2024-06-06T07:44:47.246470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305217+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:44:47.305217+0000","last_peered":"2024-06-06T07:44:47.305217+0000","last_clean":"2024-06-06T07:44:47.305217+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:44:47.305217+0000","last_undegraded":"2024-06-06T07:44:47.305217+0000","last_fullsized":"2024-06-06T07:44:47.305217+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":65,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:25.277268+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:45:25.277268+0000","last_peered":"2024-06-06T07:45:25.277268+0000","last_clean":"2024-06-06T07:45:25.277268+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:45:25.277268+0000","last_undegraded":"2024-06-06T07:45:25.277268+0000","last_fullsized":"2024-06-06T07:45:25.277268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254337+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:44:47.254337+0000","last_peered":"2024-06-06T07:44:47.254337+0000","last_clean":"2024-06-06T07:44:47.254337+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:44:47.254337+0000","last_undegraded":"2024-06-06T07:44:47.254337+0000","last_fullsized":"2024-06-06T07:44:47.254337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.202849+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:44:48.202849+0000","last_peered":"2024-06-06T07:44:48.202849+0000","last_clean":"2024-06-06T07:44:48.202849+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:44:48.202849+0000","last_undegraded":"2024-06-06T07:44:48.202849+0000","last_fullsized":"2024-06-06T07:44:48.202849+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.224628+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:45:01.224628+0000","last_peered":"2024-06-06T07:45:01.224628+0000","last_clean":"2024-06-06T07:45:01.224628+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:45:01.224628+0000","last_undegraded":"2024-06-06T07:45:01.224628+0000","last_fullsized":"2024-06-06T07:45:01.224628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249401+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:44:47.249401+0000","last_peered":"2024-06-06T07:44:47.249401+0000","last_clean":"2024-06-06T07:44:47.249401+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:44:47.249401+0000","last_undegraded":"2024-06-06T07:44:47.249401+0000","last_fullsized":"2024-06-06T07:44:47.249401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200247+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:44:48.200247+0000","last_peered":"2024-06-06T07:44:48.200247+0000","last_clean":"2024-06-06T07:44:48.200247+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:44:48.200247+0000","last_undegraded":"2024-06-06T07:44:48.200247+0000","last_fullsized":"2024-06-06T07:44:48.200247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.226314+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:45:01.226314+0000","last_peered":"2024-06-06T07:45:01.226314+0000","last_clean":"2024-06-06T07:45:01.226314+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:45:01.226314+0000","last_undegraded":"2024-06-06T07:45:01.226314+0000","last_fullsized":"2024-06-06T07:45:01.226314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.261019+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:44:47.261019+0000","last_peered":"2024-06-06T07:44:47.261019+0000","last_clean":"2024-06-06T07:44:47.261019+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:44:47.261019+0000","last_undegraded":"2024-06-06T07:44:47.261019+0000","last_fullsized":"2024-06-06T07:44:47.261019+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246498+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:44:47.246498+0000","last_peered":"2024-06-06T07:44:47.246498+0000","last_clean":"2024-06-06T07:44:47.246498+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:44:47.246498+0000","last_undegraded":"2024-06-06T07:44:47.246498+0000","last_fullsized":"2024-06-06T07:44:47.246498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'30","reported_seq":123,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:26.280219+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:45:26.280219+0000","last_peered":"2024-06-06T07:45:26.280219+0000","last_clean":"2024-06-06T07:45:26.280219+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:45:26.280219+0000","last_undegraded":"2024-06-06T07:45:26.280219+0000","last_fullsized":"2024-06-06T07:45:26.280219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56667,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":82,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250462+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:44:47.250462+0000","last_peered":"2024-06-06T07:44:47.250462+0000","last_clean":"2024-06-06T07:44:47.250462+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:44:47.250462+0000","last_undegraded":"2024-06-06T07:44:47.250462+0000","last_fullsized":"2024-06-06T07:44:47.250462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253000+0000","last_change":"2024-06-06T07:44:11.398614+0000","last_active":"2024-06-06T07:44:47.253000+0000","last_peered":"2024-06-06T07:44:47.253000+0000","last_clean":"2024-06-06T07:44:47.253000+0000","last_became_active":"2024-06-06T07:44:11.396846+0000","last_became_peered":"2024-06-06T07:44:11.396846+0000","last_unstale":"2024-06-06T07:44:47.253000+0000","last_undegraded":"2024-06-06T07:44:47.253000+0000","last_fullsized":"2024-06-06T07:44:47.253000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:45:49.835897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231884+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:45:02.231884+0000","last_peered":"2024-06-06T07:45:02.231884+0000","last_clean":"2024-06-06T07:45:02.231884+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:45:02.231884+0000","last_undegraded":"2024-06-06T07:45:02.231884+0000","last_fullsized":"2024-06-06T07:45:02.231884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259507+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:44:47.259507+0000","last_peered":"2024-06-06T07:44:47.259507+0000","last_clean":"2024-06-06T07:44:47.259507+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:44:47.259507+0000","last_undegraded":"2024-06-06T07:44:47.259507+0000","last_fullsized":"2024-06-06T07:44:47.259507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252504+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:44:47.252504+0000","last_peered":"2024-06-06T07:44:47.252504+0000","last_clean":"2024-06-06T07:44:47.252504+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:44:47.252504+0000","last_undegraded":"2024-06-06T07:44:47.252504+0000","last_fullsized":"2024-06-06T07:44:47.252504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.226453+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:45:01.226453+0000","last_peered":"2024-06-06T07:45:01.226453+0000","last_clean":"2024-06-06T07:45:01.226453+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:45:01.226453+0000","last_undegraded":"2024-06-06T07:45:01.226453+0000","last_fullsized":"2024-06-06T07:45:01.226453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249654+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:44:47.249654+0000","last_peered":"2024-06-06T07:44:47.249654+0000","last_clean":"2024-06-06T07:44:47.249654+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:44:47.249654+0000","last_undegraded":"2024-06-06T07:44:47.249654+0000","last_fullsized":"2024-06-06T07:44:47.249654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247162+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:44:47.247162+0000","last_peered":"2024-06-06T07:44:47.247162+0000","last_clean":"2024-06-06T07:44:47.247162+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:44:47.247162+0000","last_undegraded":"2024-06-06T07:44:47.247162+0000","last_fullsized":"2024-06-06T07:44:47.247162+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'4","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:28.283211+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:45:28.283211+0000","last_peered":"2024-06-06T07:45:28.283211+0000","last_clean":"2024-06-06T07:45:28.283211+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:45:28.283211+0000","last_undegraded":"2024-06-06T07:45:28.283211+0000","last_fullsized":"2024-06-06T07:45:28.283211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253158+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:44:47.253158+0000","last_peered":"2024-06-06T07:44:47.253158+0000","last_clean":"2024-06-06T07:44:47.253158+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:44:47.253158+0000","last_undegraded":"2024-06-06T07:44:47.253158+0000","last_fullsized":"2024-06-06T07:44:47.253158+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259966+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:44:47.259966+0000","last_peered":"2024-06-06T07:44:47.259966+0000","last_clean":"2024-06-06T07:44:47.259966+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:44:47.259966+0000","last_undegraded":"2024-06-06T07:44:47.259966+0000","last_fullsized":"2024-06-06T07:44:47.259966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":55,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.233624+0000","last_change":"2024-06-06T07:43:57.227439+0000","last_active":"2024-06-06T07:45:02.233624+0000","last_peered":"2024-06-06T07:45:02.233624+0000","last_clean":"2024-06-06T07:45:02.233624+0000","last_became_active":"2024-06-06T07:43:57.227041+0000","last_became_peered":"2024-06-06T07:43:57.227041+0000","last_unstale":"2024-06-06T07:45:02.233624+0000","last_undegraded":"2024-06-06T07:45:02.233624+0000","last_fullsized":"2024-06-06T07:45:02.233624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:31:38.713352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254216+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:44:47.254216+0000","last_peered":"2024-06-06T07:44:47.254216+0000","last_clean":"2024-06-06T07:44:47.254216+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:44:47.254216+0000","last_undegraded":"2024-06-06T07:44:47.254216+0000","last_fullsized":"2024-06-06T07:44:47.254216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254234+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:44:47.254234+0000","last_peered":"2024-06-06T07:44:47.254234+0000","last_clean":"2024-06-06T07:44:47.254234+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:44:47.254234+0000","last_undegraded":"2024-06-06T07:44:47.254234+0000","last_fullsized":"2024-06-06T07:44:47.254234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:04.228321+0000","last_change":"2024-06-06T07:43:57.219073+0000","last_active":"2024-06-06T07:45:04.228321+0000","last_peered":"2024-06-06T07:45:04.228321+0000","last_clean":"2024-06-06T07:45:04.228321+0000","last_became_active":"2024-06-06T07:43:57.218561+0000","last_became_peered":"2024-06-06T07:43:57.218561+0000","last_unstale":"2024-06-06T07:45:04.228321+0000","last_undegraded":"2024-06-06T07:45:04.228321+0000","last_fullsized":"2024-06-06T07:45:04.228321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:02.919529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252871+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:44:47.252871+0000","last_peered":"2024-06-06T07:44:47.252871+0000","last_clean":"2024-06-06T07:44:47.252871+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:44:47.252871+0000","last_undegraded":"2024-06-06T07:44:47.252871+0000","last_fullsized":"2024-06-06T07:44:47.252871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248875+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:44:47.248875+0000","last_peered":"2024-06-06T07:44:47.248875+0000","last_clean":"2024-06-06T07:44:47.248875+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:44:47.248875+0000","last_undegraded":"2024-06-06T07:44:47.248875+0000","last_fullsized":"2024-06-06T07:44:47.248875+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246783+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:44:47.246783+0000","last_peered":"2024-06-06T07:44:47.246783+0000","last_clean":"2024-06-06T07:44:47.246783+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:44:47.246783+0000","last_undegraded":"2024-06-06T07:44:47.246783+0000","last_fullsized":"2024-06-06T07:44:47.246783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251835+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:44:47.251835+0000","last_peered":"2024-06-06T07:44:47.251835+0000","last_clean":"2024-06-06T07:44:47.251835+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:44:47.251835+0000","last_undegraded":"2024-06-06T07:44:47.251835+0000","last_fullsized":"2024-06-06T07:44:47.251835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.260038+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:44:47.260038+0000","last_peered":"2024-06-06T07:44:47.260038+0000","last_clean":"2024-06-06T07:44:47.260038+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:44:47.260038+0000","last_undegraded":"2024-06-06T07:44:47.260038+0000","last_fullsized":"2024-06-06T07:44:47.260038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249566+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:44:47.249566+0000","last_peered":"2024-06-06T07:44:47.249566+0000","last_clean":"2024-06-06T07:44:47.249566+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:44:47.249566+0000","last_undegraded":"2024-06-06T07:44:47.249566+0000","last_fullsized":"2024-06-06T07:44:47.249566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251276+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:44:47.251276+0000","last_peered":"2024-06-06T07:44:47.251276+0000","last_clean":"2024-06-06T07:44:47.251276+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:44:47.251276+0000","last_undegraded":"2024-06-06T07:44:47.251276+0000","last_fullsized":"2024-06-06T07:44:47.251276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259777+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:44:47.259777+0000","last_peered":"2024-06-06T07:44:47.259777+0000","last_clean":"2024-06-06T07:44:47.259777+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:44:47.259777+0000","last_undegraded":"2024-06-06T07:44:47.259777+0000","last_fullsized":"2024-06-06T07:44:47.259777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253129+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:44:47.253129+0000","last_peered":"2024-06-06T07:44:47.253129+0000","last_clean":"2024-06-06T07:44:47.253129+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:44:47.253129+0000","last_undegraded":"2024-06-06T07:44:47.253129+0000","last_fullsized":"2024-06-06T07:44:47.253129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":65,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:23.275174+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:45:23.275174+0000","last_peered":"2024-06-06T07:45:23.275174+0000","last_clean":"2024-06-06T07:45:23.275174+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:45:23.275174+0000","last_undegraded":"2024-06-06T07:45:23.275174+0000","last_fullsized":"2024-06-06T07:45:23.275174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259458+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:44:47.259458+0000","last_peered":"2024-06-06T07:44:47.259458+0000","last_clean":"2024-06-06T07:44:47.259458+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:44:47.259458+0000","last_undegraded":"2024-06-06T07:44:47.259458+0000","last_fullsized":"2024-06-06T07:44:47.259458+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305217+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:44:47.305217+0000","last_peered":"2024-06-06T07:44:47.305217+0000","last_clean":"2024-06-06T07:44:47.305217+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:44:47.305217+0000","last_undegraded":"2024-06-06T07:44:47.305217+0000","last_fullsized":"2024-06-06T07:44:47.305217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:04.244281+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:45:04.244281+0000","last_peered":"2024-06-06T07:45:04.244281+0000","last_clean":"2024-06-06T07:45:04.244281+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:45:04.244281+0000","last_undegraded":"2024-06-06T07:45:04.244281+0000","last_fullsized":"2024-06-06T07:45:04.244281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249150+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:44:47.249150+0000","last_peered":"2024-06-06T07:44:47.249150+0000","last_clean":"2024-06-06T07:44:47.249150+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:44:47.249150+0000","last_undegraded":"2024-06-06T07:44:47.249150+0000","last_fullsized":"2024-06-06T07:44:47.249150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.260890+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:44:47.260890+0000","last_peered":"2024-06-06T07:44:47.260890+0000","last_clean":"2024-06-06T07:44:47.260890+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:44:47.260890+0000","last_undegraded":"2024-06-06T07:44:47.260890+0000","last_fullsized":"2024-06-06T07:44:47.260890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259474+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:44:47.259474+0000","last_peered":"2024-06-06T07:44:47.259474+0000","last_clean":"2024-06-06T07:44:47.259474+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:44:47.259474+0000","last_undegraded":"2024-06-06T07:44:47.259474+0000","last_fullsized":"2024-06-06T07:44:47.259474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232597+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:45:02.232597+0000","last_peered":"2024-06-06T07:45:02.232597+0000","last_clean":"2024-06-06T07:45:02.232597+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:45:02.232597+0000","last_undegraded":"2024-06-06T07:45:02.232597+0000","last_fullsized":"2024-06-06T07:45:02.232597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252369+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:44:47.252369+0000","last_peered":"2024-06-06T07:44:47.252369+0000","last_clean":"2024-06-06T07:44:47.252369+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:44:47.252369+0000","last_undegraded":"2024-06-06T07:44:47.252369+0000","last_fullsized":"2024-06-06T07:44:47.252369+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.222753+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:45:01.222753+0000","last_peered":"2024-06-06T07:45:01.222753+0000","last_clean":"2024-06-06T07:45:01.222753+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:45:01.222753+0000","last_undegraded":"2024-06-06T07:45:01.222753+0000","last_fullsized":"2024-06-06T07:45:01.222753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259736+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:44:47.259736+0000","last_peered":"2024-06-06T07:44:47.259736+0000","last_clean":"2024-06-06T07:44:47.259736+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:44:47.259736+0000","last_undegraded":"2024-06-06T07:44:47.259736+0000","last_fullsized":"2024-06-06T07:44:47.259736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305126+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:44:47.305126+0000","last_peered":"2024-06-06T07:44:47.305126+0000","last_clean":"2024-06-06T07:44:47.305126+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:44:47.305126+0000","last_undegraded":"2024-06-06T07:44:47.305126+0000","last_fullsized":"2024-06-06T07:44:47.305126+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'10","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:22.348144+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:45:22.348144+0000","last_peered":"2024-06-06T07:45:22.348144+0000","last_clean":"2024-06-06T07:45:22.348144+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:45:22.348144+0000","last_undegraded":"2024-06-06T07:45:22.348144+0000","last_fullsized":"2024-06-06T07:45:22.348144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305574+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:44:47.305574+0000","last_peered":"2024-06-06T07:44:47.305574+0000","last_clean":"2024-06-06T07:44:47.305574+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:44:47.305574+0000","last_undegraded":"2024-06-06T07:44:47.305574+0000","last_fullsized":"2024-06-06T07:44:47.305574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248016+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:44:47.248016+0000","last_peered":"2024-06-06T07:44:47.248016+0000","last_clean":"2024-06-06T07:44:47.248016+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:44:47.248016+0000","last_undegraded":"2024-06-06T07:44:47.248016+0000","last_fullsized":"2024-06-06T07:44:47.248016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.222304+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:45:01.222304+0000","last_peered":"2024-06-06T07:45:01.222304+0000","last_clean":"2024-06-06T07:45:01.222304+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:45:01.222304+0000","last_undegraded":"2024-06-06T07:45:01.222304+0000","last_fullsized":"2024-06-06T07:45:01.222304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.201542+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:44:48.201542+0000","last_peered":"2024-06-06T07:44:48.201542+0000","last_clean":"2024-06-06T07:44:48.201542+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:44:48.201542+0000","last_undegraded":"2024-06-06T07:44:48.201542+0000","last_fullsized":"2024-06-06T07:44:48.201542+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200592+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:44:48.200592+0000","last_peered":"2024-06-06T07:44:48.200592+0000","last_clean":"2024-06-06T07:44:48.200592+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:44:48.200592+0000","last_undegraded":"2024-06-06T07:44:48.200592+0000","last_fullsized":"2024-06-06T07:44:48.200592+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225857+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:45:01.225857+0000","last_peered":"2024-06-06T07:45:01.225857+0000","last_clean":"2024-06-06T07:45:01.225857+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:45:01.225857+0000","last_undegraded":"2024-06-06T07:45:01.225857+0000","last_fullsized":"2024-06-06T07:45:01.225857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248725+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:44:47.248725+0000","last_peered":"2024-06-06T07:44:47.248725+0000","last_clean":"2024-06-06T07:44:47.248725+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:44:47.248725+0000","last_undegraded":"2024-06-06T07:44:47.248725+0000","last_fullsized":"2024-06-06T07:44:47.248725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247102+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:44:47.247102+0000","last_peered":"2024-06-06T07:44:47.247102+0000","last_clean":"2024-06-06T07:44:47.247102+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:44:47.247102+0000","last_undegraded":"2024-06-06T07:44:47.247102+0000","last_fullsized":"2024-06-06T07:44:47.247102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225085+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:45:01.225085+0000","last_peered":"2024-06-06T07:45:01.225085+0000","last_clean":"2024-06-06T07:45:01.225085+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:45:01.225085+0000","last_undegraded":"2024-06-06T07:45:01.225085+0000","last_fullsized":"2024-06-06T07:45:01.225085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251868+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:44:47.251868+0000","last_peered":"2024-06-06T07:44:47.251868+0000","last_clean":"2024-06-06T07:44:47.251868+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:44:47.251868+0000","last_undegraded":"2024-06-06T07:44:47.251868+0000","last_fullsized":"2024-06-06T07:44:47.251868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200465+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:44:48.200465+0000","last_peered":"2024-06-06T07:44:48.200465+0000","last_clean":"2024-06-06T07:44:48.200465+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:44:48.200465+0000","last_undegraded":"2024-06-06T07:44:48.200465+0000","last_fullsized":"2024-06-06T07:44:48.200465+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247542+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:44:47.247542+0000","last_peered":"2024-06-06T07:44:47.247542+0000","last_clean":"2024-06-06T07:44:47.247542+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:44:47.247542+0000","last_undegraded":"2024-06-06T07:44:47.247542+0000","last_fullsized":"2024-06-06T07:44:47.247542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305383+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:44:47.305383+0000","last_peered":"2024-06-06T07:44:47.305383+0000","last_clean":"2024-06-06T07:44:47.305383+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:44:47.305383+0000","last_undegraded":"2024-06-06T07:44:47.305383+0000","last_fullsized":"2024-06-06T07:44:47.305383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200505+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:44:48.200505+0000","last_peered":"2024-06-06T07:44:48.200505+0000","last_clean":"2024-06-06T07:44:48.200505+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:44:48.200505+0000","last_undegraded":"2024-06-06T07:44:48.200505+0000","last_fullsized":"2024-06-06T07:44:48.200505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225614+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:45:01.225614+0000","last_peered":"2024-06-06T07:45:01.225614+0000","last_clean":"2024-06-06T07:45:01.225614+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:45:01.225614+0000","last_undegraded":"2024-06-06T07:45:01.225614+0000","last_fullsized":"2024-06-06T07:45:01.225614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249540+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:44:47.249540+0000","last_peered":"2024-06-06T07:44:47.249540+0000","last_clean":"2024-06-06T07:44:47.249540+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:44:47.249540+0000","last_undegraded":"2024-06-06T07:44:47.249540+0000","last_fullsized":"2024-06-06T07:44:47.249540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253577+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:44:47.253577+0000","last_peered":"2024-06-06T07:44:47.253577+0000","last_clean":"2024-06-06T07:44:47.253577+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:44:47.253577+0000","last_undegraded":"2024-06-06T07:44:47.253577+0000","last_fullsized":"2024-06-06T07:44:47.253577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232911+0000","last_change":"2024-06-06T07:43:57.531884+0000","last_active":"2024-06-06T07:45:02.232911+0000","last_peered":"2024-06-06T07:45:02.232911+0000","last_clean":"2024-06-06T07:45:02.232911+0000","last_became_active":"2024-06-06T07:43:57.531421+0000","last_became_peered":"2024-06-06T07:43:57.531421+0000","last_unstale":"2024-06-06T07:45:02.232911+0000","last_undegraded":"2024-06-06T07:45:02.232911+0000","last_fullsized":"2024-06-06T07:45:02.232911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:50:31.774675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250232+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:44:47.250232+0000","last_peered":"2024-06-06T07:44:47.250232+0000","last_clean":"2024-06-06T07:44:47.250232+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:44:47.250232+0000","last_undegraded":"2024-06-06T07:44:47.250232+0000","last_fullsized":"2024-06-06T07:44:47.250232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249205+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:44:47.249205+0000","last_peered":"2024-06-06T07:44:47.249205+0000","last_clean":"2024-06-06T07:44:47.249205+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:44:47.249205+0000","last_undegraded":"2024-06-06T07:44:47.249205+0000","last_fullsized":"2024-06-06T07:44:47.249205+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252776+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:44:47.252776+0000","last_peered":"2024-06-06T07:44:47.252776+0000","last_clean":"2024-06-06T07:44:47.252776+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:44:47.252776+0000","last_undegraded":"2024-06-06T07:44:47.252776+0000","last_fullsized":"2024-06-06T07:44:47.252776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249418+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:44:47.249418+0000","last_peered":"2024-06-06T07:44:47.249418+0000","last_clean":"2024-06-06T07:44:47.249418+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:44:47.249418+0000","last_undegraded":"2024-06-06T07:44:47.249418+0000","last_fullsized":"2024-06-06T07:44:47.249418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252563+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:44:47.252563+0000","last_peered":"2024-06-06T07:44:47.252563+0000","last_clean":"2024-06-06T07:44:47.252563+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:44:47.252563+0000","last_undegraded":"2024-06-06T07:44:47.252563+0000","last_fullsized":"2024-06-06T07:44:47.252563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259885+0000","last_change":"2024-06-06T07:43:57.213045+0000","last_active":"2024-06-06T07:44:47.259885+0000","last_peered":"2024-06-06T07:44:47.259885+0000","last_clean":"2024-06-06T07:44:47.259885+0000","last_became_active":"2024-06-06T07:43:57.212603+0000","last_became_peered":"2024-06-06T07:43:57.212603+0000","last_unstale":"2024-06-06T07:44:47.259885+0000","last_undegraded":"2024-06-06T07:44:47.259885+0000","last_fullsized":"2024-06-06T07:44:47.259885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:26.491105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250901+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:44:47.250901+0000","last_peered":"2024-06-06T07:44:47.250901+0000","last_clean":"2024-06-06T07:44:47.250901+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:44:47.250901+0000","last_undegraded":"2024-06-06T07:44:47.250901+0000","last_fullsized":"2024-06-06T07:44:47.250901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251990+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:44:47.251990+0000","last_peered":"2024-06-06T07:44:47.251990+0000","last_clean":"2024-06-06T07:44:47.251990+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:44:47.251990+0000","last_undegraded":"2024-06-06T07:44:47.251990+0000","last_fullsized":"2024-06-06T07:44:47.251990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248417+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:44:47.248417+0000","last_peered":"2024-06-06T07:44:47.248417+0000","last_clean":"2024-06-06T07:44:47.248417+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:44:47.248417+0000","last_undegraded":"2024-06-06T07:44:47.248417+0000","last_fullsized":"2024-06-06T07:44:47.248417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247464+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:44:47.247464+0000","last_peered":"2024-06-06T07:44:47.247464+0000","last_clean":"2024-06-06T07:44:47.247464+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:44:47.247464+0000","last_undegraded":"2024-06-06T07:44:47.247464+0000","last_fullsized":"2024-06-06T07:44:47.247464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249368+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:44:47.249368+0000","last_peered":"2024-06-06T07:44:47.249368+0000","last_clean":"2024-06-06T07:44:47.249368+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:44:47.249368+0000","last_undegraded":"2024-06-06T07:44:47.249368+0000","last_fullsized":"2024-06-06T07:44:47.249368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259728+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:44:47.259728+0000","last_peered":"2024-06-06T07:44:47.259728+0000","last_clean":"2024-06-06T07:44:47.259728+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:44:47.259728+0000","last_undegraded":"2024-06-06T07:44:47.259728+0000","last_fullsized":"2024-06-06T07:44:47.259728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.257583+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:44:47.257583+0000","last_peered":"2024-06-06T07:44:47.257583+0000","last_clean":"2024-06-06T07:44:47.257583+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:44:47.257583+0000","last_undegraded":"2024-06-06T07:44:47.257583+0000","last_fullsized":"2024-06-06T07:44:47.257583+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248138+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:44:47.248138+0000","last_peered":"2024-06-06T07:44:47.248138+0000","last_clean":"2024-06-06T07:44:47.248138+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:44:47.248138+0000","last_undegraded":"2024-06-06T07:44:47.248138+0000","last_fullsized":"2024-06-06T07:44:47.248138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305670+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:44:47.305670+0000","last_peered":"2024-06-06T07:44:47.305670+0000","last_clean":"2024-06-06T07:44:47.305670+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:44:47.305670+0000","last_undegraded":"2024-06-06T07:44:47.305670+0000","last_fullsized":"2024-06-06T07:44:47.305670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252932+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:44:47.252932+0000","last_peered":"2024-06-06T07:44:47.252932+0000","last_clean":"2024-06-06T07:44:47.252932+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:44:47.252932+0000","last_undegraded":"2024-06-06T07:44:47.252932+0000","last_fullsized":"2024-06-06T07:44:47.252932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305453+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:44:47.305453+0000","last_peered":"2024-06-06T07:44:47.305453+0000","last_clean":"2024-06-06T07:44:47.305453+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:44:47.305453+0000","last_undegraded":"2024-06-06T07:44:47.305453+0000","last_fullsized":"2024-06-06T07:44:47.305453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253855+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:44:47.253855+0000","last_peered":"2024-06-06T07:44:47.253855+0000","last_clean":"2024-06-06T07:44:47.253855+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:44:47.253855+0000","last_undegraded":"2024-06-06T07:44:47.253855+0000","last_fullsized":"2024-06-06T07:44:47.253855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259828+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:44:47.259828+0000","last_peered":"2024-06-06T07:44:47.259828+0000","last_clean":"2024-06-06T07:44:47.259828+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:44:47.259828+0000","last_undegraded":"2024-06-06T07:44:47.259828+0000","last_fullsized":"2024-06-06T07:44:47.259828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252682+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:44:47.252682+0000","last_peered":"2024-06-06T07:44:47.252682+0000","last_clean":"2024-06-06T07:44:47.252682+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:44:47.252682+0000","last_undegraded":"2024-06-06T07:44:47.252682+0000","last_fullsized":"2024-06-06T07:44:47.252682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61057,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":169,"num_read_kb":146,"num_write":164,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":178906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":550999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1794048,"data_stored":1199420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":551235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":551155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1716224,"data_stored":1147453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":546411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":602988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":547799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40652,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704500,"statfs":{"total":95995035648,"available":95953408000,"internally_reserved":0,"allocated":1183744,"data_stored":600463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":546605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":551451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1142651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:45:32.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:45:33.183 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:33 smithi195 ceph-mon[37040]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:33 smithi195 ceph-mon[37040]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-06T07:45:33.546 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:33 smithi087 ceph-mon[32276]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:33.546 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:33 smithi087 ceph-mon[32276]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-06T07:45:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:33 smithi181 ceph-mon[36790]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:33 smithi181 ceph-mon[36790]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-06T07:45:35.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 -- 172.21.15.87:0/282249157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102ef0 msgr2=0x7f842c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 --2- 172.21.15.87:0/282249157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102ef0 0x7f842c10f3a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f842400b0d0 tx=0x7f842400b3e0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 -- 172.21.15.87:0/282249157 shutdown_connections 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 --2- 172.21.15.87:0/282249157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 0x7f842c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 --2- 172.21.15.87:0/282249157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102ef0 0x7f842c10f3a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 --2- 172.21.15.87:0/282249157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102530 0x7f842c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 -- 172.21.15.87:0/282249157 >> 172.21.15.87:0/282249157 conn(0x7f842c079640 msgr2=0x7f842c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 -- 172.21.15.87:0/282249157 shutdown_connections 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.003+0000 7f84319ea700 1 -- 172.21.15.87:0/282249157 wait complete. 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 Processor -- start 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 -- start start 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:35.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102ef0 0x7f842c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 0x7f842c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f842c112840 con 0x7f842c102530 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1126c0 con 0x7f842c1085d0 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f84319ea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1129c0 con 0x7f842c102ef0 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f842affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f842affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33930/0 (socket says 172.21.15.87:33930) 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.004+0000 7f842affd700 1 -- 172.21.15.87:0/984956683 learned_addr learned my addr 172.21.15.87:0/984956683 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842a7fc700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102ef0 0x7f842c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842b7fe700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 0x7f842c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 -- 172.21.15.87:0/984956683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102ef0 msgr2=0x7f842c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102ef0 0x7f842c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 -- 172.21.15.87:0/984956683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 msgr2=0x7f842c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 0x7f842c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 -- 172.21.15.87:0/984956683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a70b0 con 0x7f842c102530 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f842affd700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c016070 con 0x7f842c102530 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a73a0 con 0x7f842c102530 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c009b40 con 0x7f842c102530 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a7930 con 0x7f842c102530 2024-06-06T07:45:35.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.005+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00f620 con 0x7f842c102530 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.007+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c103c40 con 0x7f842c102530 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.008+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f841c019070 con 0x7f842c102530 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.008+0000 7f84309e8700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 0x7f84140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.008+0000 7f842a7fc700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 0x7f84140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.008+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f841c0cf4a0 con 0x7f842c102530 2024-06-06T07:45:35.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.009+0000 7f842a7fc700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 0x7f84140a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8420005950 tx=0x7f84200058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:35.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.013+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f841c09bf40 con 0x7f842c102530 2024-06-06T07:45:35.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.207+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f842c107910 con 0x7f84140a2d30 2024-06-06T07:45:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.216+0000 7f84309e8700 1 -- 172.21.15.87:0/984956683 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f842c107910 con 0x7f84140a2d30 2024-06-06T07:45:35.221 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 msgr2=0x7f84140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 0x7f84140a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8420005950 tx=0x7f84200058e0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 msgr2=0x7f842c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dd10 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 shutdown_connections 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f84140a2d30 0x7f84140a51f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c1085d0 0x7f842c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c102530 0x7f842c074790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 --2- 172.21.15.87:0/984956683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f842c102ef0 0x7f842c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 >> 172.21.15.87:0/984956683 conn(0x7f842c079640 msgr2=0x7f842c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 shutdown_connections 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:35.219+0000 7f84319ea700 1 -- 172.21.15.87:0/984956683 wait complete. 2024-06-06T07:45:35.224 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:45:35.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:35 smithi087 ceph-mon[32276]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:45:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:35 smithi195 ceph-mon[37040]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:45:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:35 smithi181 ceph-mon[36790]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:45:36.113 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-06-06T07:45:34.161971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657062,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":331,"num_read_kb":286,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384992,"kb_used_data":15040,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556832,"statfs":{"total":1151940427776,"available":1151546195968,"internally_reserved":0,"allocated":15400960,"data_stored":8553701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9785,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":41,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003658"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253895+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:44:47.253895+0000","last_peered":"2024-06-06T07:44:47.253895+0000","last_clean":"2024-06-06T07:44:47.253895+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:44:47.253895+0000","last_undegraded":"2024-06-06T07:44:47.253895+0000","last_fullsized":"2024-06-06T07:44:47.253895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249504+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:44:47.249504+0000","last_peered":"2024-06-06T07:44:47.249504+0000","last_clean":"2024-06-06T07:44:47.249504+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:44:47.249504+0000","last_undegraded":"2024-06-06T07:44:47.249504+0000","last_fullsized":"2024-06-06T07:44:47.249504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249348+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:44:47.249348+0000","last_peered":"2024-06-06T07:44:47.249348+0000","last_clean":"2024-06-06T07:44:47.249348+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:44:47.249348+0000","last_undegraded":"2024-06-06T07:44:47.249348+0000","last_fullsized":"2024-06-06T07:44:47.249348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200667+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:44:48.200667+0000","last_peered":"2024-06-06T07:44:48.200667+0000","last_clean":"2024-06-06T07:44:48.200667+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:44:48.200667+0000","last_undegraded":"2024-06-06T07:44:48.200667+0000","last_fullsized":"2024-06-06T07:44:48.200667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249006+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:44:47.249006+0000","last_peered":"2024-06-06T07:44:47.249006+0000","last_clean":"2024-06-06T07:44:47.249006+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:44:47.249006+0000","last_undegraded":"2024-06-06T07:44:47.249006+0000","last_fullsized":"2024-06-06T07:44:47.249006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251412+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:44:47.251412+0000","last_peered":"2024-06-06T07:44:47.251412+0000","last_clean":"2024-06-06T07:44:47.251412+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:44:47.251412+0000","last_undegraded":"2024-06-06T07:44:47.251412+0000","last_fullsized":"2024-06-06T07:44:47.251412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249222+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:44:47.249222+0000","last_peered":"2024-06-06T07:44:47.249222+0000","last_clean":"2024-06-06T07:44:47.249222+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:44:47.249222+0000","last_undegraded":"2024-06-06T07:44:47.249222+0000","last_fullsized":"2024-06-06T07:44:47.249222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305125+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:44:47.305125+0000","last_peered":"2024-06-06T07:44:47.305125+0000","last_clean":"2024-06-06T07:44:47.305125+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:44:47.305125+0000","last_undegraded":"2024-06-06T07:44:47.305125+0000","last_fullsized":"2024-06-06T07:44:47.305125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.256770+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:44:47.256770+0000","last_peered":"2024-06-06T07:44:47.256770+0000","last_clean":"2024-06-06T07:44:47.256770+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:44:47.256770+0000","last_undegraded":"2024-06-06T07:44:47.256770+0000","last_fullsized":"2024-06-06T07:44:47.256770+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251703+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:44:47.251703+0000","last_peered":"2024-06-06T07:44:47.251703+0000","last_clean":"2024-06-06T07:44:47.251703+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:44:47.251703+0000","last_undegraded":"2024-06-06T07:44:47.251703+0000","last_fullsized":"2024-06-06T07:44:47.251703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249755+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:44:47.249755+0000","last_peered":"2024-06-06T07:44:47.249755+0000","last_clean":"2024-06-06T07:44:47.249755+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:44:47.249755+0000","last_undegraded":"2024-06-06T07:44:47.249755+0000","last_fullsized":"2024-06-06T07:44:47.249755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200653+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:44:48.200653+0000","last_peered":"2024-06-06T07:44:48.200653+0000","last_clean":"2024-06-06T07:44:48.200653+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:44:48.200653+0000","last_undegraded":"2024-06-06T07:44:48.200653+0000","last_fullsized":"2024-06-06T07:44:48.200653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259765+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:44:47.259765+0000","last_peered":"2024-06-06T07:44:47.259765+0000","last_clean":"2024-06-06T07:44:47.259765+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:44:47.259765+0000","last_undegraded":"2024-06-06T07:44:47.259765+0000","last_fullsized":"2024-06-06T07:44:47.259765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249177+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:44:47.249177+0000","last_peered":"2024-06-06T07:44:47.249177+0000","last_clean":"2024-06-06T07:44:47.249177+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:44:47.249177+0000","last_undegraded":"2024-06-06T07:44:47.249177+0000","last_fullsized":"2024-06-06T07:44:47.249177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251796+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:44:47.251796+0000","last_peered":"2024-06-06T07:44:47.251796+0000","last_clean":"2024-06-06T07:44:47.251796+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:44:47.251796+0000","last_undegraded":"2024-06-06T07:44:47.251796+0000","last_fullsized":"2024-06-06T07:44:47.251796+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253426+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:44:47.253426+0000","last_peered":"2024-06-06T07:44:47.253426+0000","last_clean":"2024-06-06T07:44:47.253426+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:44:47.253426+0000","last_undegraded":"2024-06-06T07:44:47.253426+0000","last_fullsized":"2024-06-06T07:44:47.253426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253668+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:44:47.253668+0000","last_peered":"2024-06-06T07:44:47.253668+0000","last_clean":"2024-06-06T07:44:47.253668+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:44:47.253668+0000","last_undegraded":"2024-06-06T07:44:47.253668+0000","last_fullsized":"2024-06-06T07:44:47.253668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246765+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:44:47.246765+0000","last_peered":"2024-06-06T07:44:47.246765+0000","last_clean":"2024-06-06T07:44:47.246765+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:44:47.246765+0000","last_undegraded":"2024-06-06T07:44:47.246765+0000","last_fullsized":"2024-06-06T07:44:47.246765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248017+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:44:47.248017+0000","last_peered":"2024-06-06T07:44:47.248017+0000","last_clean":"2024-06-06T07:44:47.248017+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:44:47.248017+0000","last_undegraded":"2024-06-06T07:44:47.248017+0000","last_fullsized":"2024-06-06T07:44:47.248017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248143+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:44:47.248143+0000","last_peered":"2024-06-06T07:44:47.248143+0000","last_clean":"2024-06-06T07:44:47.248143+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:44:47.248143+0000","last_undegraded":"2024-06-06T07:44:47.248143+0000","last_fullsized":"2024-06-06T07:44:47.248143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.233329+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:45:02.233329+0000","last_peered":"2024-06-06T07:45:02.233329+0000","last_clean":"2024-06-06T07:45:02.233329+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:45:02.233329+0000","last_undegraded":"2024-06-06T07:45:02.233329+0000","last_fullsized":"2024-06-06T07:45:02.233329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305425+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:44:47.305425+0000","last_peered":"2024-06-06T07:44:47.305425+0000","last_clean":"2024-06-06T07:44:47.305425+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:44:47.305425+0000","last_undegraded":"2024-06-06T07:44:47.305425+0000","last_fullsized":"2024-06-06T07:44:47.305425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200270+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:44:48.200270+0000","last_peered":"2024-06-06T07:44:48.200270+0000","last_clean":"2024-06-06T07:44:48.200270+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:44:48.200270+0000","last_undegraded":"2024-06-06T07:44:48.200270+0000","last_fullsized":"2024-06-06T07:44:48.200270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305743+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:44:47.305743+0000","last_peered":"2024-06-06T07:44:47.305743+0000","last_clean":"2024-06-06T07:44:47.305743+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:44:47.305743+0000","last_undegraded":"2024-06-06T07:44:47.305743+0000","last_fullsized":"2024-06-06T07:44:47.305743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249423+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:44:47.249423+0000","last_peered":"2024-06-06T07:44:47.249423+0000","last_clean":"2024-06-06T07:44:47.249423+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:44:47.249423+0000","last_undegraded":"2024-06-06T07:44:47.249423+0000","last_fullsized":"2024-06-06T07:44:47.249423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200244+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:44:48.200244+0000","last_peered":"2024-06-06T07:44:48.200244+0000","last_clean":"2024-06-06T07:44:48.200244+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:44:48.200244+0000","last_undegraded":"2024-06-06T07:44:48.200244+0000","last_fullsized":"2024-06-06T07:44:48.200244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232105+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:45:02.232105+0000","last_peered":"2024-06-06T07:45:02.232105+0000","last_clean":"2024-06-06T07:45:02.232105+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:45:02.232105+0000","last_undegraded":"2024-06-06T07:45:02.232105+0000","last_fullsized":"2024-06-06T07:45:02.232105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253227+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:44:47.253227+0000","last_peered":"2024-06-06T07:44:47.253227+0000","last_clean":"2024-06-06T07:44:47.253227+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:44:47.253227+0000","last_undegraded":"2024-06-06T07:44:47.253227+0000","last_fullsized":"2024-06-06T07:44:47.253227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.199895+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:44:48.199895+0000","last_peered":"2024-06-06T07:44:48.199895+0000","last_clean":"2024-06-06T07:44:48.199895+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:44:48.199895+0000","last_undegraded":"2024-06-06T07:44:48.199895+0000","last_fullsized":"2024-06-06T07:44:48.199895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249592+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:44:47.249592+0000","last_peered":"2024-06-06T07:44:47.249592+0000","last_clean":"2024-06-06T07:44:47.249592+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:44:47.249592+0000","last_undegraded":"2024-06-06T07:44:47.249592+0000","last_fullsized":"2024-06-06T07:44:47.249592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.257530+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:44:47.257530+0000","last_peered":"2024-06-06T07:44:47.257530+0000","last_clean":"2024-06-06T07:44:47.257530+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:44:47.257530+0000","last_undegraded":"2024-06-06T07:44:47.257530+0000","last_fullsized":"2024-06-06T07:44:47.257530+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247748+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:44:47.247748+0000","last_peered":"2024-06-06T07:44:47.247748+0000","last_clean":"2024-06-06T07:44:47.247748+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:44:47.247748+0000","last_undegraded":"2024-06-06T07:44:47.247748+0000","last_fullsized":"2024-06-06T07:44:47.247748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":79,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:32.288656+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:45:32.288656+0000","last_peered":"2024-06-06T07:45:32.288656+0000","last_clean":"2024-06-06T07:45:32.288656+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:45:32.288656+0000","last_undegraded":"2024-06-06T07:45:32.288656+0000","last_fullsized":"2024-06-06T07:45:32.288656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249256+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:44:47.249256+0000","last_peered":"2024-06-06T07:44:47.249256+0000","last_clean":"2024-06-06T07:44:47.249256+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:44:47.249256+0000","last_undegraded":"2024-06-06T07:44:47.249256+0000","last_fullsized":"2024-06-06T07:44:47.249256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248099+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:44:47.248099+0000","last_peered":"2024-06-06T07:44:47.248099+0000","last_clean":"2024-06-06T07:44:47.248099+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:44:47.248099+0000","last_undegraded":"2024-06-06T07:44:47.248099+0000","last_fullsized":"2024-06-06T07:44:47.248099+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231548+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:45:02.231548+0000","last_peered":"2024-06-06T07:45:02.231548+0000","last_clean":"2024-06-06T07:45:02.231548+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:45:02.231548+0000","last_undegraded":"2024-06-06T07:45:02.231548+0000","last_fullsized":"2024-06-06T07:45:02.231548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252280+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:44:47.252280+0000","last_peered":"2024-06-06T07:44:47.252280+0000","last_clean":"2024-06-06T07:44:47.252280+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:44:47.252280+0000","last_undegraded":"2024-06-06T07:44:47.252280+0000","last_fullsized":"2024-06-06T07:44:47.252280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253533+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:44:47.253533+0000","last_peered":"2024-06-06T07:44:47.253533+0000","last_clean":"2024-06-06T07:44:47.253533+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:44:47.253533+0000","last_undegraded":"2024-06-06T07:44:47.253533+0000","last_fullsized":"2024-06-06T07:44:47.253533+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231456+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:45:02.231456+0000","last_peered":"2024-06-06T07:45:02.231456+0000","last_clean":"2024-06-06T07:45:02.231456+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:45:02.231456+0000","last_undegraded":"2024-06-06T07:45:02.231456+0000","last_fullsized":"2024-06-06T07:45:02.231456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305230+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:44:47.305230+0000","last_peered":"2024-06-06T07:44:47.305230+0000","last_clean":"2024-06-06T07:44:47.305230+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:44:47.305230+0000","last_undegraded":"2024-06-06T07:44:47.305230+0000","last_fullsized":"2024-06-06T07:44:47.305230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249461+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:44:47.249461+0000","last_peered":"2024-06-06T07:44:47.249461+0000","last_clean":"2024-06-06T07:44:47.249461+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:44:47.249461+0000","last_undegraded":"2024-06-06T07:44:47.249461+0000","last_fullsized":"2024-06-06T07:44:47.249461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252930+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:44:47.252930+0000","last_peered":"2024-06-06T07:44:47.252930+0000","last_clean":"2024-06-06T07:44:47.252930+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:44:47.252930+0000","last_undegraded":"2024-06-06T07:44:47.252930+0000","last_fullsized":"2024-06-06T07:44:47.252930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246470+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:44:47.246470+0000","last_peered":"2024-06-06T07:44:47.246470+0000","last_clean":"2024-06-06T07:44:47.246470+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:44:47.246470+0000","last_undegraded":"2024-06-06T07:44:47.246470+0000","last_fullsized":"2024-06-06T07:44:47.246470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305217+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:44:47.305217+0000","last_peered":"2024-06-06T07:44:47.305217+0000","last_clean":"2024-06-06T07:44:47.305217+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:44:47.305217+0000","last_undegraded":"2024-06-06T07:44:47.305217+0000","last_fullsized":"2024-06-06T07:44:47.305217+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":70,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:30.284976+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:45:30.284976+0000","last_peered":"2024-06-06T07:45:30.284976+0000","last_clean":"2024-06-06T07:45:30.284976+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:45:30.284976+0000","last_undegraded":"2024-06-06T07:45:30.284976+0000","last_fullsized":"2024-06-06T07:45:30.284976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254337+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:44:47.254337+0000","last_peered":"2024-06-06T07:44:47.254337+0000","last_clean":"2024-06-06T07:44:47.254337+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:44:47.254337+0000","last_undegraded":"2024-06-06T07:44:47.254337+0000","last_fullsized":"2024-06-06T07:44:47.254337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.202849+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:44:48.202849+0000","last_peered":"2024-06-06T07:44:48.202849+0000","last_clean":"2024-06-06T07:44:48.202849+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:44:48.202849+0000","last_undegraded":"2024-06-06T07:44:48.202849+0000","last_fullsized":"2024-06-06T07:44:48.202849+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.224628+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:45:01.224628+0000","last_peered":"2024-06-06T07:45:01.224628+0000","last_clean":"2024-06-06T07:45:01.224628+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:45:01.224628+0000","last_undegraded":"2024-06-06T07:45:01.224628+0000","last_fullsized":"2024-06-06T07:45:01.224628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249401+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:44:47.249401+0000","last_peered":"2024-06-06T07:44:47.249401+0000","last_clean":"2024-06-06T07:44:47.249401+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:44:47.249401+0000","last_undegraded":"2024-06-06T07:44:47.249401+0000","last_fullsized":"2024-06-06T07:44:47.249401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200247+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:44:48.200247+0000","last_peered":"2024-06-06T07:44:48.200247+0000","last_clean":"2024-06-06T07:44:48.200247+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:44:48.200247+0000","last_undegraded":"2024-06-06T07:44:48.200247+0000","last_fullsized":"2024-06-06T07:44:48.200247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.226314+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:45:01.226314+0000","last_peered":"2024-06-06T07:45:01.226314+0000","last_clean":"2024-06-06T07:45:01.226314+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:45:01.226314+0000","last_undegraded":"2024-06-06T07:45:01.226314+0000","last_fullsized":"2024-06-06T07:45:01.226314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.261019+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:44:47.261019+0000","last_peered":"2024-06-06T07:44:47.261019+0000","last_clean":"2024-06-06T07:44:47.261019+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:44:47.261019+0000","last_undegraded":"2024-06-06T07:44:47.261019+0000","last_fullsized":"2024-06-06T07:44:47.261019+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246498+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:44:47.246498+0000","last_peered":"2024-06-06T07:44:47.246498+0000","last_clean":"2024-06-06T07:44:47.246498+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:44:47.246498+0000","last_undegraded":"2024-06-06T07:44:47.246498+0000","last_fullsized":"2024-06-06T07:44:47.246498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":137,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:31.287247+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:45:31.287247+0000","last_peered":"2024-06-06T07:45:31.287247+0000","last_clean":"2024-06-06T07:45:31.287247+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:45:31.287247+0000","last_undegraded":"2024-06-06T07:45:31.287247+0000","last_fullsized":"2024-06-06T07:45:31.287247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62070,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":65,"num_read_kb":93,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250462+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:44:47.250462+0000","last_peered":"2024-06-06T07:44:47.250462+0000","last_clean":"2024-06-06T07:44:47.250462+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:44:47.250462+0000","last_undegraded":"2024-06-06T07:44:47.250462+0000","last_fullsized":"2024-06-06T07:44:47.250462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253000+0000","last_change":"2024-06-06T07:44:11.398614+0000","last_active":"2024-06-06T07:44:47.253000+0000","last_peered":"2024-06-06T07:44:47.253000+0000","last_clean":"2024-06-06T07:44:47.253000+0000","last_became_active":"2024-06-06T07:44:11.396846+0000","last_became_peered":"2024-06-06T07:44:11.396846+0000","last_unstale":"2024-06-06T07:44:47.253000+0000","last_undegraded":"2024-06-06T07:44:47.253000+0000","last_fullsized":"2024-06-06T07:44:47.253000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:45:49.835897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.231884+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:45:02.231884+0000","last_peered":"2024-06-06T07:45:02.231884+0000","last_clean":"2024-06-06T07:45:02.231884+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:45:02.231884+0000","last_undegraded":"2024-06-06T07:45:02.231884+0000","last_fullsized":"2024-06-06T07:45:02.231884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259507+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:44:47.259507+0000","last_peered":"2024-06-06T07:44:47.259507+0000","last_clean":"2024-06-06T07:44:47.259507+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:44:47.259507+0000","last_undegraded":"2024-06-06T07:44:47.259507+0000","last_fullsized":"2024-06-06T07:44:47.259507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252504+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:44:47.252504+0000","last_peered":"2024-06-06T07:44:47.252504+0000","last_clean":"2024-06-06T07:44:47.252504+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:44:47.252504+0000","last_undegraded":"2024-06-06T07:44:47.252504+0000","last_fullsized":"2024-06-06T07:44:47.252504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.226453+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:45:01.226453+0000","last_peered":"2024-06-06T07:45:01.226453+0000","last_clean":"2024-06-06T07:45:01.226453+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:45:01.226453+0000","last_undegraded":"2024-06-06T07:45:01.226453+0000","last_fullsized":"2024-06-06T07:45:01.226453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249654+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:44:47.249654+0000","last_peered":"2024-06-06T07:44:47.249654+0000","last_clean":"2024-06-06T07:44:47.249654+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:44:47.249654+0000","last_undegraded":"2024-06-06T07:44:47.249654+0000","last_fullsized":"2024-06-06T07:44:47.249654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247162+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:44:47.247162+0000","last_peered":"2024-06-06T07:44:47.247162+0000","last_clean":"2024-06-06T07:44:47.247162+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:44:47.247162+0000","last_undegraded":"2024-06-06T07:44:47.247162+0000","last_fullsized":"2024-06-06T07:44:47.247162+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'4","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:28.283211+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:45:28.283211+0000","last_peered":"2024-06-06T07:45:28.283211+0000","last_clean":"2024-06-06T07:45:28.283211+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:45:28.283211+0000","last_undegraded":"2024-06-06T07:45:28.283211+0000","last_fullsized":"2024-06-06T07:45:28.283211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253158+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:44:47.253158+0000","last_peered":"2024-06-06T07:44:47.253158+0000","last_clean":"2024-06-06T07:44:47.253158+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:44:47.253158+0000","last_undegraded":"2024-06-06T07:44:47.253158+0000","last_fullsized":"2024-06-06T07:44:47.253158+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259966+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:44:47.259966+0000","last_peered":"2024-06-06T07:44:47.259966+0000","last_clean":"2024-06-06T07:44:47.259966+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:44:47.259966+0000","last_undegraded":"2024-06-06T07:44:47.259966+0000","last_fullsized":"2024-06-06T07:44:47.259966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":55,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.233624+0000","last_change":"2024-06-06T07:43:57.227439+0000","last_active":"2024-06-06T07:45:02.233624+0000","last_peered":"2024-06-06T07:45:02.233624+0000","last_clean":"2024-06-06T07:45:02.233624+0000","last_became_active":"2024-06-06T07:43:57.227041+0000","last_became_peered":"2024-06-06T07:43:57.227041+0000","last_unstale":"2024-06-06T07:45:02.233624+0000","last_undegraded":"2024-06-06T07:45:02.233624+0000","last_fullsized":"2024-06-06T07:45:02.233624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:31:38.713352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254216+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:44:47.254216+0000","last_peered":"2024-06-06T07:44:47.254216+0000","last_clean":"2024-06-06T07:44:47.254216+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:44:47.254216+0000","last_undegraded":"2024-06-06T07:44:47.254216+0000","last_fullsized":"2024-06-06T07:44:47.254216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.254234+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:44:47.254234+0000","last_peered":"2024-06-06T07:44:47.254234+0000","last_clean":"2024-06-06T07:44:47.254234+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:44:47.254234+0000","last_undegraded":"2024-06-06T07:44:47.254234+0000","last_fullsized":"2024-06-06T07:44:47.254234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:04.228321+0000","last_change":"2024-06-06T07:43:57.219073+0000","last_active":"2024-06-06T07:45:04.228321+0000","last_peered":"2024-06-06T07:45:04.228321+0000","last_clean":"2024-06-06T07:45:04.228321+0000","last_became_active":"2024-06-06T07:43:57.218561+0000","last_became_peered":"2024-06-06T07:43:57.218561+0000","last_unstale":"2024-06-06T07:45:04.228321+0000","last_undegraded":"2024-06-06T07:45:04.228321+0000","last_fullsized":"2024-06-06T07:45:04.228321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:02.919529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252871+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:44:47.252871+0000","last_peered":"2024-06-06T07:44:47.252871+0000","last_clean":"2024-06-06T07:44:47.252871+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:44:47.252871+0000","last_undegraded":"2024-06-06T07:44:47.252871+0000","last_fullsized":"2024-06-06T07:44:47.252871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248875+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:44:47.248875+0000","last_peered":"2024-06-06T07:44:47.248875+0000","last_clean":"2024-06-06T07:44:47.248875+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:44:47.248875+0000","last_undegraded":"2024-06-06T07:44:47.248875+0000","last_fullsized":"2024-06-06T07:44:47.248875+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.246783+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:44:47.246783+0000","last_peered":"2024-06-06T07:44:47.246783+0000","last_clean":"2024-06-06T07:44:47.246783+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:44:47.246783+0000","last_undegraded":"2024-06-06T07:44:47.246783+0000","last_fullsized":"2024-06-06T07:44:47.246783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251835+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:44:47.251835+0000","last_peered":"2024-06-06T07:44:47.251835+0000","last_clean":"2024-06-06T07:44:47.251835+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:44:47.251835+0000","last_undegraded":"2024-06-06T07:44:47.251835+0000","last_fullsized":"2024-06-06T07:44:47.251835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.260038+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:44:47.260038+0000","last_peered":"2024-06-06T07:44:47.260038+0000","last_clean":"2024-06-06T07:44:47.260038+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:44:47.260038+0000","last_undegraded":"2024-06-06T07:44:47.260038+0000","last_fullsized":"2024-06-06T07:44:47.260038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249566+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:44:47.249566+0000","last_peered":"2024-06-06T07:44:47.249566+0000","last_clean":"2024-06-06T07:44:47.249566+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:44:47.249566+0000","last_undegraded":"2024-06-06T07:44:47.249566+0000","last_fullsized":"2024-06-06T07:44:47.249566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251276+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:44:47.251276+0000","last_peered":"2024-06-06T07:44:47.251276+0000","last_clean":"2024-06-06T07:44:47.251276+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:44:47.251276+0000","last_undegraded":"2024-06-06T07:44:47.251276+0000","last_fullsized":"2024-06-06T07:44:47.251276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259777+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:44:47.259777+0000","last_peered":"2024-06-06T07:44:47.259777+0000","last_clean":"2024-06-06T07:44:47.259777+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:44:47.259777+0000","last_undegraded":"2024-06-06T07:44:47.259777+0000","last_fullsized":"2024-06-06T07:44:47.259777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253129+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:44:47.253129+0000","last_peered":"2024-06-06T07:44:47.253129+0000","last_clean":"2024-06-06T07:44:47.253129+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:44:47.253129+0000","last_undegraded":"2024-06-06T07:44:47.253129+0000","last_fullsized":"2024-06-06T07:44:47.253129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":70,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:28.282755+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:45:28.282755+0000","last_peered":"2024-06-06T07:45:28.282755+0000","last_clean":"2024-06-06T07:45:28.282755+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:45:28.282755+0000","last_undegraded":"2024-06-06T07:45:28.282755+0000","last_fullsized":"2024-06-06T07:45:28.282755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259458+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:44:47.259458+0000","last_peered":"2024-06-06T07:44:47.259458+0000","last_clean":"2024-06-06T07:44:47.259458+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:44:47.259458+0000","last_undegraded":"2024-06-06T07:44:47.259458+0000","last_fullsized":"2024-06-06T07:44:47.259458+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305217+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:44:47.305217+0000","last_peered":"2024-06-06T07:44:47.305217+0000","last_clean":"2024-06-06T07:44:47.305217+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:44:47.305217+0000","last_undegraded":"2024-06-06T07:44:47.305217+0000","last_fullsized":"2024-06-06T07:44:47.305217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:04.244281+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:45:04.244281+0000","last_peered":"2024-06-06T07:45:04.244281+0000","last_clean":"2024-06-06T07:45:04.244281+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:45:04.244281+0000","last_undegraded":"2024-06-06T07:45:04.244281+0000","last_fullsized":"2024-06-06T07:45:04.244281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249150+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:44:47.249150+0000","last_peered":"2024-06-06T07:44:47.249150+0000","last_clean":"2024-06-06T07:44:47.249150+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:44:47.249150+0000","last_undegraded":"2024-06-06T07:44:47.249150+0000","last_fullsized":"2024-06-06T07:44:47.249150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.260890+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:44:47.260890+0000","last_peered":"2024-06-06T07:44:47.260890+0000","last_clean":"2024-06-06T07:44:47.260890+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:44:47.260890+0000","last_undegraded":"2024-06-06T07:44:47.260890+0000","last_fullsized":"2024-06-06T07:44:47.260890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259474+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:44:47.259474+0000","last_peered":"2024-06-06T07:44:47.259474+0000","last_clean":"2024-06-06T07:44:47.259474+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:44:47.259474+0000","last_undegraded":"2024-06-06T07:44:47.259474+0000","last_fullsized":"2024-06-06T07:44:47.259474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232597+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:45:02.232597+0000","last_peered":"2024-06-06T07:45:02.232597+0000","last_clean":"2024-06-06T07:45:02.232597+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:45:02.232597+0000","last_undegraded":"2024-06-06T07:45:02.232597+0000","last_fullsized":"2024-06-06T07:45:02.232597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252369+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:44:47.252369+0000","last_peered":"2024-06-06T07:44:47.252369+0000","last_clean":"2024-06-06T07:44:47.252369+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:44:47.252369+0000","last_undegraded":"2024-06-06T07:44:47.252369+0000","last_fullsized":"2024-06-06T07:44:47.252369+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.222753+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:45:01.222753+0000","last_peered":"2024-06-06T07:45:01.222753+0000","last_clean":"2024-06-06T07:45:01.222753+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:45:01.222753+0000","last_undegraded":"2024-06-06T07:45:01.222753+0000","last_fullsized":"2024-06-06T07:45:01.222753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259736+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:44:47.259736+0000","last_peered":"2024-06-06T07:44:47.259736+0000","last_clean":"2024-06-06T07:44:47.259736+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:44:47.259736+0000","last_undegraded":"2024-06-06T07:44:47.259736+0000","last_fullsized":"2024-06-06T07:44:47.259736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305126+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:44:47.305126+0000","last_peered":"2024-06-06T07:44:47.305126+0000","last_clean":"2024-06-06T07:44:47.305126+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:44:47.305126+0000","last_undegraded":"2024-06-06T07:44:47.305126+0000","last_fullsized":"2024-06-06T07:44:47.305126+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'11","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:25.348479+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:45:25.348479+0000","last_peered":"2024-06-06T07:45:25.348479+0000","last_clean":"2024-06-06T07:45:25.348479+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:45:25.348479+0000","last_undegraded":"2024-06-06T07:45:25.348479+0000","last_fullsized":"2024-06-06T07:45:25.348479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305574+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:44:47.305574+0000","last_peered":"2024-06-06T07:44:47.305574+0000","last_clean":"2024-06-06T07:44:47.305574+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:44:47.305574+0000","last_undegraded":"2024-06-06T07:44:47.305574+0000","last_fullsized":"2024-06-06T07:44:47.305574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248016+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:44:47.248016+0000","last_peered":"2024-06-06T07:44:47.248016+0000","last_clean":"2024-06-06T07:44:47.248016+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:44:47.248016+0000","last_undegraded":"2024-06-06T07:44:47.248016+0000","last_fullsized":"2024-06-06T07:44:47.248016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.222304+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:45:01.222304+0000","last_peered":"2024-06-06T07:45:01.222304+0000","last_clean":"2024-06-06T07:45:01.222304+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:45:01.222304+0000","last_undegraded":"2024-06-06T07:45:01.222304+0000","last_fullsized":"2024-06-06T07:45:01.222304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.201542+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:44:48.201542+0000","last_peered":"2024-06-06T07:44:48.201542+0000","last_clean":"2024-06-06T07:44:48.201542+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:44:48.201542+0000","last_undegraded":"2024-06-06T07:44:48.201542+0000","last_fullsized":"2024-06-06T07:44:48.201542+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200592+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:44:48.200592+0000","last_peered":"2024-06-06T07:44:48.200592+0000","last_clean":"2024-06-06T07:44:48.200592+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:44:48.200592+0000","last_undegraded":"2024-06-06T07:44:48.200592+0000","last_fullsized":"2024-06-06T07:44:48.200592+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225857+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:45:01.225857+0000","last_peered":"2024-06-06T07:45:01.225857+0000","last_clean":"2024-06-06T07:45:01.225857+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:45:01.225857+0000","last_undegraded":"2024-06-06T07:45:01.225857+0000","last_fullsized":"2024-06-06T07:45:01.225857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248725+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:44:47.248725+0000","last_peered":"2024-06-06T07:44:47.248725+0000","last_clean":"2024-06-06T07:44:47.248725+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:44:47.248725+0000","last_undegraded":"2024-06-06T07:44:47.248725+0000","last_fullsized":"2024-06-06T07:44:47.248725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247102+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:44:47.247102+0000","last_peered":"2024-06-06T07:44:47.247102+0000","last_clean":"2024-06-06T07:44:47.247102+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:44:47.247102+0000","last_undegraded":"2024-06-06T07:44:47.247102+0000","last_fullsized":"2024-06-06T07:44:47.247102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225085+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:45:01.225085+0000","last_peered":"2024-06-06T07:45:01.225085+0000","last_clean":"2024-06-06T07:45:01.225085+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:45:01.225085+0000","last_undegraded":"2024-06-06T07:45:01.225085+0000","last_fullsized":"2024-06-06T07:45:01.225085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251868+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:44:47.251868+0000","last_peered":"2024-06-06T07:44:47.251868+0000","last_clean":"2024-06-06T07:44:47.251868+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:44:47.251868+0000","last_undegraded":"2024-06-06T07:44:47.251868+0000","last_fullsized":"2024-06-06T07:44:47.251868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200465+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:44:48.200465+0000","last_peered":"2024-06-06T07:44:48.200465+0000","last_clean":"2024-06-06T07:44:48.200465+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:44:48.200465+0000","last_undegraded":"2024-06-06T07:44:48.200465+0000","last_fullsized":"2024-06-06T07:44:48.200465+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247542+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:44:47.247542+0000","last_peered":"2024-06-06T07:44:47.247542+0000","last_clean":"2024-06-06T07:44:47.247542+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:44:47.247542+0000","last_undegraded":"2024-06-06T07:44:47.247542+0000","last_fullsized":"2024-06-06T07:44:47.247542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305383+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:44:47.305383+0000","last_peered":"2024-06-06T07:44:47.305383+0000","last_clean":"2024-06-06T07:44:47.305383+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:44:47.305383+0000","last_undegraded":"2024-06-06T07:44:47.305383+0000","last_fullsized":"2024-06-06T07:44:47.305383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:48.200505+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:44:48.200505+0000","last_peered":"2024-06-06T07:44:48.200505+0000","last_clean":"2024-06-06T07:44:48.200505+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:44:48.200505+0000","last_undegraded":"2024-06-06T07:44:48.200505+0000","last_fullsized":"2024-06-06T07:44:48.200505+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:01.225614+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:45:01.225614+0000","last_peered":"2024-06-06T07:45:01.225614+0000","last_clean":"2024-06-06T07:45:01.225614+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:45:01.225614+0000","last_undegraded":"2024-06-06T07:45:01.225614+0000","last_fullsized":"2024-06-06T07:45:01.225614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249540+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:44:47.249540+0000","last_peered":"2024-06-06T07:44:47.249540+0000","last_clean":"2024-06-06T07:44:47.249540+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:44:47.249540+0000","last_undegraded":"2024-06-06T07:44:47.249540+0000","last_fullsized":"2024-06-06T07:44:47.249540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253577+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:44:47.253577+0000","last_peered":"2024-06-06T07:44:47.253577+0000","last_clean":"2024-06-06T07:44:47.253577+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:44:47.253577+0000","last_undegraded":"2024-06-06T07:44:47.253577+0000","last_fullsized":"2024-06-06T07:44:47.253577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:45:02.232911+0000","last_change":"2024-06-06T07:43:57.531884+0000","last_active":"2024-06-06T07:45:02.232911+0000","last_peered":"2024-06-06T07:45:02.232911+0000","last_clean":"2024-06-06T07:45:02.232911+0000","last_became_active":"2024-06-06T07:43:57.531421+0000","last_became_peered":"2024-06-06T07:43:57.531421+0000","last_unstale":"2024-06-06T07:45:02.232911+0000","last_undegraded":"2024-06-06T07:45:02.232911+0000","last_fullsized":"2024-06-06T07:45:02.232911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:50:31.774675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250232+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:44:47.250232+0000","last_peered":"2024-06-06T07:44:47.250232+0000","last_clean":"2024-06-06T07:44:47.250232+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:44:47.250232+0000","last_undegraded":"2024-06-06T07:44:47.250232+0000","last_fullsized":"2024-06-06T07:44:47.250232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249205+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:44:47.249205+0000","last_peered":"2024-06-06T07:44:47.249205+0000","last_clean":"2024-06-06T07:44:47.249205+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:44:47.249205+0000","last_undegraded":"2024-06-06T07:44:47.249205+0000","last_fullsized":"2024-06-06T07:44:47.249205+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252776+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:44:47.252776+0000","last_peered":"2024-06-06T07:44:47.252776+0000","last_clean":"2024-06-06T07:44:47.252776+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:44:47.252776+0000","last_undegraded":"2024-06-06T07:44:47.252776+0000","last_fullsized":"2024-06-06T07:44:47.252776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249418+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:44:47.249418+0000","last_peered":"2024-06-06T07:44:47.249418+0000","last_clean":"2024-06-06T07:44:47.249418+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:44:47.249418+0000","last_undegraded":"2024-06-06T07:44:47.249418+0000","last_fullsized":"2024-06-06T07:44:47.249418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252563+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:44:47.252563+0000","last_peered":"2024-06-06T07:44:47.252563+0000","last_clean":"2024-06-06T07:44:47.252563+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:44:47.252563+0000","last_undegraded":"2024-06-06T07:44:47.252563+0000","last_fullsized":"2024-06-06T07:44:47.252563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259885+0000","last_change":"2024-06-06T07:43:57.213045+0000","last_active":"2024-06-06T07:44:47.259885+0000","last_peered":"2024-06-06T07:44:47.259885+0000","last_clean":"2024-06-06T07:44:47.259885+0000","last_became_active":"2024-06-06T07:43:57.212603+0000","last_became_peered":"2024-06-06T07:43:57.212603+0000","last_unstale":"2024-06-06T07:44:47.259885+0000","last_undegraded":"2024-06-06T07:44:47.259885+0000","last_fullsized":"2024-06-06T07:44:47.259885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:26.491105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.250901+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:44:47.250901+0000","last_peered":"2024-06-06T07:44:47.250901+0000","last_clean":"2024-06-06T07:44:47.250901+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:44:47.250901+0000","last_undegraded":"2024-06-06T07:44:47.250901+0000","last_fullsized":"2024-06-06T07:44:47.250901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.251990+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:44:47.251990+0000","last_peered":"2024-06-06T07:44:47.251990+0000","last_clean":"2024-06-06T07:44:47.251990+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:44:47.251990+0000","last_undegraded":"2024-06-06T07:44:47.251990+0000","last_fullsized":"2024-06-06T07:44:47.251990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248417+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:44:47.248417+0000","last_peered":"2024-06-06T07:44:47.248417+0000","last_clean":"2024-06-06T07:44:47.248417+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:44:47.248417+0000","last_undegraded":"2024-06-06T07:44:47.248417+0000","last_fullsized":"2024-06-06T07:44:47.248417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.247464+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:44:47.247464+0000","last_peered":"2024-06-06T07:44:47.247464+0000","last_clean":"2024-06-06T07:44:47.247464+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:44:47.247464+0000","last_undegraded":"2024-06-06T07:44:47.247464+0000","last_fullsized":"2024-06-06T07:44:47.247464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.249368+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:44:47.249368+0000","last_peered":"2024-06-06T07:44:47.249368+0000","last_clean":"2024-06-06T07:44:47.249368+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:44:47.249368+0000","last_undegraded":"2024-06-06T07:44:47.249368+0000","last_fullsized":"2024-06-06T07:44:47.249368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259728+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:44:47.259728+0000","last_peered":"2024-06-06T07:44:47.259728+0000","last_clean":"2024-06-06T07:44:47.259728+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:44:47.259728+0000","last_undegraded":"2024-06-06T07:44:47.259728+0000","last_fullsized":"2024-06-06T07:44:47.259728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.257583+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:44:47.257583+0000","last_peered":"2024-06-06T07:44:47.257583+0000","last_clean":"2024-06-06T07:44:47.257583+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:44:47.257583+0000","last_undegraded":"2024-06-06T07:44:47.257583+0000","last_fullsized":"2024-06-06T07:44:47.257583+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.248138+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:44:47.248138+0000","last_peered":"2024-06-06T07:44:47.248138+0000","last_clean":"2024-06-06T07:44:47.248138+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:44:47.248138+0000","last_undegraded":"2024-06-06T07:44:47.248138+0000","last_fullsized":"2024-06-06T07:44:47.248138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305670+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:44:47.305670+0000","last_peered":"2024-06-06T07:44:47.305670+0000","last_clean":"2024-06-06T07:44:47.305670+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:44:47.305670+0000","last_undegraded":"2024-06-06T07:44:47.305670+0000","last_fullsized":"2024-06-06T07:44:47.305670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252932+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:44:47.252932+0000","last_peered":"2024-06-06T07:44:47.252932+0000","last_clean":"2024-06-06T07:44:47.252932+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:44:47.252932+0000","last_undegraded":"2024-06-06T07:44:47.252932+0000","last_fullsized":"2024-06-06T07:44:47.252932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.305453+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:44:47.305453+0000","last_peered":"2024-06-06T07:44:47.305453+0000","last_clean":"2024-06-06T07:44:47.305453+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:44:47.305453+0000","last_undegraded":"2024-06-06T07:44:47.305453+0000","last_fullsized":"2024-06-06T07:44:47.305453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.253855+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:44:47.253855+0000","last_peered":"2024-06-06T07:44:47.253855+0000","last_clean":"2024-06-06T07:44:47.253855+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:44:47.253855+0000","last_undegraded":"2024-06-06T07:44:47.253855+0000","last_fullsized":"2024-06-06T07:44:47.253855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.259828+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:44:47.259828+0000","last_peered":"2024-06-06T07:44:47.259828+0000","last_clean":"2024-06-06T07:44:47.259828+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:44:47.259828+0000","last_undegraded":"2024-06-06T07:44:47.259828+0000","last_fullsized":"2024-06-06T07:44:47.259828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T07:44:47.252682+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:44:47.252682+0000","last_peered":"2024-06-06T07:44:47.252682+0000","last_clean":"2024-06-06T07:44:47.252682+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:44:47.252682+0000","last_undegraded":"2024-06-06T07:44:47.252682+0000","last_fullsized":"2024-06-06T07:44:47.252682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66460,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":193977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":550999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28772,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716380,"statfs":{"total":95995035648,"available":95965573120,"internally_reserved":0,"allocated":1798144,"data_stored":1204823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":551235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":551155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1716224,"data_stored":1147453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":546411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":608391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":547799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40656,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704496,"statfs":{"total":95995035648,"available":95953403904,"internally_reserved":0,"allocated":1187840,"data_stored":604728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":546605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":551451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1142651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:45:36.118 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T07:45:36.119 INFO:tasks.cephadm:Setting up client nodes... 2024-06-06T07:45:36.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-06T07:45:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:36 smithi087 ceph-mon[32276]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:36.372 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:36 smithi195 ceph-mon[37040]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:36 smithi181 ceph-mon[36790]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:45:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:37 smithi195 ceph-mon[37040]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:37 smithi181 ceph-mon[36790]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:37.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:37 smithi087 ceph-mon[32276]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:38.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:38 smithi087 ceph-mon[32276]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 -- 172.21.15.87:0/2765739297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30102fb0 msgr2=0x7f2c30109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2765739297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30102fb0 0x7f2c30109860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b0d0 tx=0x7f2c2800b3e0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 -- 172.21.15.87:0/2765739297 shutdown_connections 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2765739297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30102fb0 0x7f2c30109860 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2765739297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30101410 0x7f2c30101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2765739297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 0x7f2c30102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 -- 172.21.15.87:0/2765739297 >> 172.21.15.87:0/2765739297 conn(0x7f2c300fc9b0 msgr2=0x7f2c300fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 -- 172.21.15.87:0/2765739297 shutdown_connections 2024-06-06T07:45:38.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.334+0000 7f2c368ff700 1 -- 172.21.15.87:0/2765739297 wait complete. 2024-06-06T07:45:38.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 Processor -- start 2024-06-06T07:45:38.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 -- start start 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 0x7f2c30198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 0x7f2c3019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010cff0 con 0x7f2c30101410 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010ce70 con 0x7f2c30102610 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c368ff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010d170 con 0x7f2c30102fb0 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45690/0 (socket says 172.21.15.87:45690) 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.335+0000 7f2c2ffff700 1 -- 172.21.15.87:0/2851670259 learned_addr learned my addr 172.21.15.87:0/2851670259 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 0x7f2c30198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 -- 172.21.15.87:0/2851670259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 msgr2=0x7f2c3019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:38.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c34e9c700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 0x7f2c3019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 0x7f2c3019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 -- 172.21.15.87:0/2851670259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 msgr2=0x7f2c30198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 0x7f2c30198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 -- 172.21.15.87:0/2851670259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c30103830 con 0x7f2c30101410 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c34e9c700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 0x7f2c3019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2ffff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c920 tx=0x7f2c2000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20004220 con 0x7f2c30101410 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c30103b20 con 0x7f2c30101410 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c20007da0 con 0x7f2c30101410 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20003e10 con 0x7f2c30101410 2024-06-06T07:45:38.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.336+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c30104010 con 0x7f2c30101410 2024-06-06T07:45:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.337+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c30050a00 con 0x7f2c30101410 2024-06-06T07:45:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.338+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2c200078c0 con 0x7f2c30101410 2024-06-06T07:45:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.339+0000 7f2c2d7fa700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 0x7f2c180a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.339+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 0x7f2c180a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:38.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.339+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2c200cf610 con 0x7f2c30101410 2024-06-06T07:45:38.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.339+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 0x7f2c180a5220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c24005950 tx=0x7f2c240058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:38.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.343+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c20014030 con 0x7f2c30101410 2024-06-06T07:45:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:38 smithi181 ceph-mon[36790]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:38.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.622+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2c30104520 con 0x7f2c30101410 2024-06-06T07:45:38.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.624+0000 7f2c2d7fa700 1 -- 172.21.15.87:0/2851670259 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2c2009c050 con 0x7f2c30101410 2024-06-06T07:45:38.624 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-06-06T07:45:38.624 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAiaWFmc/wtJRAAtVVyZLmBoEyuRN0sGIvv4g== 2024-06-06T07:45:38.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.626+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 msgr2=0x7f2c180a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.626+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 0x7f2c180a5220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c24005950 tx=0x7f2c240058e0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.626+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 msgr2=0x7f2c301985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.626+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c920 tx=0x7f2c2000cce0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 shutdown_connections 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2c180a2d60 0x7f2c180a5220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c30102fb0 0x7f2c3019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101410 0x7f2c301985b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 --2- 172.21.15.87:0/2851670259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30102610 0x7f2c30198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 >> 172.21.15.87:0/2851670259 conn(0x7f2c300fc9b0 msgr2=0x7f2c300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 shutdown_connections 2024-06-06T07:45:38.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:38.627+0000 7f2c368ff700 1 -- 172.21.15.87:0/2851670259 wait complete. 2024-06-06T07:45:38.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:38 smithi195 ceph-mon[37040]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:45:39.440 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:45:39.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-06T07:45:39.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-06T07:45:39.481 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-06T07:45:39.481 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-06T07:45:39.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-06T07:45:39.544 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:39 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T07:45:39.544 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:39 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T07:45:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:39 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T07:45:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:39 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T07:45:39.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:39 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T07:45:39.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:39 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2851670259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T07:45:39.763 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:40 smithi181 ceph-mon[36790]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-06T07:45:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:40 smithi087 ceph-mon[32276]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-06T07:45:40.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:40 smithi195 ceph-mon[37040]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 -- 172.21.15.87:0/3437709781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6800108580 msgr2=0x7f6800108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 --2- 172.21.15.87:0/3437709781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6800108580 0x7f6800108960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0099c0 tx=0x7f67fc009cd0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 -- 172.21.15.87:0/3437709781 shutdown_connections 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 --2- 172.21.15.87:0/3437709781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68001013c0 0x7f680010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 --2- 172.21.15.87:0/3437709781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6800108580 0x7f6800108960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 --2- 172.21.15.87:0/3437709781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 0x7f6800100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 -- 172.21.15.87:0/3437709781 >> 172.21.15.87:0/3437709781 conn(0x7f6800077990 msgr2=0x7f68000fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 -- 172.21.15.87:0/3437709781 shutdown_connections 2024-06-06T07:45:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.504+0000 7f68083bb700 1 -- 172.21.15.87:0/3437709781 wait complete. 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 Processor -- start 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 -- start start 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 0x7f680019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 0x7f6800198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f68001127d0 con 0x7f68001013c0 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6800112650 con 0x7f6800108580 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f68083bb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6800112950 con 0x7f6800100a00 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45698/0 (socket says 172.21.15.87:45698) 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 -- 172.21.15.87:0/1327699480 learned_addr learned my addr 172.21.15.87:0/1327699480 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6806157700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 0x7f680019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6806958700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 0x7f6800198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 -- 172.21.15.87:0/1327699480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 msgr2=0x7f680019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 0x7f680019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 -- 172.21.15.87:0/1327699480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 msgr2=0x7f6800198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 0x7f6800198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6805956700 1 -- 172.21.15.87:0/1327699480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68001990f0 con 0x7f68001013c0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.505+0000 7f6806958700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 0x7f6800198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f6805956700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f67f000d980 tx=0x7f67f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0017070 con 0x7f68001013c0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68001993e0 con 0x7f68001013c0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f0005020 con 0x7f68001013c0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0009bc0 con 0x7f68001013c0 2024-06-06T07:45:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.506+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68001a5650 con 0x7f68001013c0 2024-06-06T07:45:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.507+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680010aa10 con 0x7f68001013c0 2024-06-06T07:45:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.508+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67f00051d0 con 0x7f68001013c0 2024-06-06T07:45:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.509+0000 7f67f77fe700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 0x7f67ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.509+0000 7f6806157700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 0x7f67ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:41.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.509+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f67f00cf460 con 0x7f68001013c0 2024-06-06T07:45:41.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.509+0000 7f6806157700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 0x7f67ec0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000c00 tx=0x7f67fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:41.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.513+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67f0020020 con 0x7f68001013c0 2024-06-06T07:45:41.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.765+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f680010a3f0 con 0x7f68001013c0 2024-06-06T07:45:41.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.771+0000 7f67f77fe700 1 -- 172.21.15.87:0/1327699480 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f67f009bf00 con 0x7f68001013c0 2024-06-06T07:45:41.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:41.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 msgr2=0x7f67ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 0x7f67ec0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000c00 tx=0x7f67fc005fb0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 msgr2=0x7f680019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f67f000d980 tx=0x7f67f000dc90 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 shutdown_connections 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f67ec0a2cb0 0x7f67ec0a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800108580 0x7f6800198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6800100a00 0x7f680019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 --2- 172.21.15.87:0/1327699480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68001013c0 0x7f680019ecc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 >> 172.21.15.87:0/1327699480 conn(0x7f6800077990 msgr2=0x7f68000feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 shutdown_connections 2024-06-06T07:45:41.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:41.775+0000 7f68083bb700 1 -- 172.21.15.87:0/1327699480 wait complete. 2024-06-06T07:45:42.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1327699480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T07:45:42.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:41 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1327699480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T07:45:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:41 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1327699480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T07:45:42.623 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":2460003046},{"type":"v1","addr":"172.21.15.87:6801","nonce":2460003046}]},"active_addr":"172.21.15.87:6801/2460003046","active_change":"2024-06-06T07:36:46.051976+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.87:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":207935511}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":2153987725}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":4065374776}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3344462507}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3532213242}]}]} 2024-06-06T07:45:42.629 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-06T07:45:42.629 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-06T07:45:42.629 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:45:42.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:42 smithi087 ceph-mon[32276]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-06T07:45:42.872 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:43.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:42 smithi195 ceph-mon[37040]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-06T07:45:43.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:42 smithi181 ceph-mon[36790]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.754+0000 7f5fba098700 1 -- 172.21.15.87:0/4219198042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4107420 msgr2=0x7f5fb4107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.754+0000 7f5fba098700 1 --2- 172.21.15.87:0/4219198042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4107420 0x7f5fb4107800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009990 tx=0x7f5f9c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 -- 172.21.15.87:0/4219198042 shutdown_connections 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 --2- 172.21.15.87:0/4219198042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101d30 0x7f5fb410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 --2- 172.21.15.87:0/4219198042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4107420 0x7f5fb4107800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 --2- 172.21.15.87:0/4219198042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4101370 0x7f5fb41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 -- 172.21.15.87:0/4219198042 >> 172.21.15.87:0/4219198042 conn(0x7f5fb4078360 msgr2=0x7f5fb4078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:44.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 -- 172.21.15.87:0/4219198042 shutdown_connections 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 -- 172.21.15.87:0/4219198042 wait complete. 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.755+0000 7f5fba098700 1 Processor -- start 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 -- start start 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 0x7f5fb419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 0x7f5fb4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4111730 con 0x7f5fb4101d30 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41115b0 con 0x7f5fb4107420 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fba098700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41118b0 con 0x7f5fb4101370 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45706/0 (socket says 172.21.15.87:45706) 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 -- 172.21.15.87:0/151899931 learned_addr learned my addr 172.21.15.87:0/151899931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:44.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb37fe700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 0x7f5fb419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:44.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb3fff700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 0x7f5fb4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:44.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 -- 172.21.15.87:0/151899931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 msgr2=0x7f5fb419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:44.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 0x7f5fb419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 -- 172.21.15.87:0/151899931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 msgr2=0x7f5fb4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 0x7f5fb4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 -- 172.21.15.87:0/151899931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb4198210 con 0x7f5fb4101d30 2024-06-06T07:45:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb3fff700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 0x7f5fb4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb37fe700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 0x7f5fb419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb2ffd700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00ba50 tx=0x7f5fac00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.756+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac019070 con 0x7f5fb4101d30 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.757+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fac014440 con 0x7f5fb4101d30 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.757+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0075d0 con 0x7f5fb4101d30 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.757+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb4198500 con 0x7f5fb4101d30 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.757+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41a47c0 con 0x7f5fb4101d30 2024-06-06T07:45:44.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.758+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4069180 con 0x7f5fb4101d30 2024-06-06T07:45:44.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.759+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5fac01c070 con 0x7f5fb4101d30 2024-06-06T07:45:44.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.759+0000 7f5fb0ff9700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 0x7f5fa00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:44.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.759+0000 7f5fb37fe700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 0x7f5fa00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:44.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.759+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f5fac0ce840 con 0x7f5fb4101d30 2024-06-06T07:45:44.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.760+0000 7f5fb37fe700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 0x7f5fa00a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.764+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5fac09b3f0 con 0x7f5fb4101d30 2024-06-06T07:45:44.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.961+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fb402dca0 con 0x7f5fb4101d30 2024-06-06T07:45:44.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.962+0000 7f5fb0ff9700 1 -- 172.21.15.87:0/151899931 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24625 (secure 0 0 0) 0x7f5fac09acf0 con 0x7f5fb4101d30 2024-06-06T07:45:44.963 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:44.963 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":97,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:44:47.243465+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 msgr2=0x7f5fa00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 0x7f5fa00a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 msgr2=0x7f5fb419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00ba50 tx=0x7f5fac00be10 comp rx=0 tx=0).stop 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 shutdown_connections 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f5fa00a2d30 0x7f5fa00a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fb4107420 0x7f5fb4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fb4101370 0x7f5fb419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 --2- 172.21.15.87:0/151899931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fb4101d30 0x7f5fb419de30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:44.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 >> 172.21.15.87:0/151899931 conn(0x7f5fb4078360 msgr2=0x7f5fb4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:44.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 shutdown_connections 2024-06-06T07:45:44.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:44.965+0000 7f5fba098700 1 -- 172.21.15.87:0/151899931 wait complete. 2024-06-06T07:45:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:45 smithi195 ceph-mon[37040]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:45:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:45 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/151899931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:45 smithi181 ceph-mon[36790]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:45:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:45 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/151899931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:45.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:45 smithi087 ceph-mon[32276]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:45:45.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:45 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/151899931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:45.766 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-06T07:45:45.766 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:45:46.009 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:45:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-06-06T07:45:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T07:45:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-06-06T07:45:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:45:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-06-06T07:45:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T07:45:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-06-06T07:45:47.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:45:47.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-06-06T07:45:47.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T07:45:47.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T07:45:47.581 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-06-06T07:45:47.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 -- 172.21.15.87:0/3194320604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 msgr2=0x7f300c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:47.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 --2- 172.21.15.87:0/3194320604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c10f320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f300400b0a0 tx=0x7f300400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 -- 172.21.15.87:0/3194320604 shutdown_connections 2024-06-06T07:45:47.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 --2- 172.21.15.87:0/3194320604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c10f320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 --2- 172.21.15.87:0/3194320604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c108500 0x7f300c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 --2- 172.21.15.87:0/3194320604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300c06abc0 0x7f300c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 -- 172.21.15.87:0/3194320604 >> 172.21.15.87:0/3194320604 conn(0x7f300c078340 msgr2=0x7f300c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 -- 172.21.15.87:0/3194320604 shutdown_connections 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 -- 172.21.15.87:0/3194320604 wait complete. 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.828+0000 7f301152a700 1 Processor -- start 2024-06-06T07:45:47.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 -- start start 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c06abc0 0x7f300c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300c108500 0x7f300c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f300c112760 con 0x7f300c06b580 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1125e0 con 0x7f300c108500 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f301152a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1128e0 con 0x7f300c06abc0 2024-06-06T07:45:47.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:47.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52682/0 (socket says 172.21.15.87:52682) 2024-06-06T07:45:47.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 -- 172.21.15.87:0/2595545242 learned_addr learned my addr 172.21.15.87:0/2595545242 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:45:47.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300affd700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c06abc0 0x7f300c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:47.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 -- 172.21.15.87:0/2595545242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c06abc0 msgr2=0x7f300c101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:47.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c06abc0 0x7f300c101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 -- 172.21.15.87:0/2595545242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300c108500 msgr2=0x7f300c101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300c108500 0x7f300c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 -- 172.21.15.87:0/2595545242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1a70d0 con 0x7f300c06b580 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.829+0000 7f300a7fc700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f300000c920 tx=0x7f300000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.830+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000007b00 con 0x7f300c06b580 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.830+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000004d10 con 0x7f300c06b580 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.830+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a73c0 con 0x7f300c06b580 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.830+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000005630 con 0x7f300c06b580 2024-06-06T07:45:47.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.830+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a7900 con 0x7f300c06b580 2024-06-06T07:45:47.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.831+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c069180 con 0x7f300c06b580 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.833+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3000004750 con 0x7f300c06b580 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.833+0000 7f2ff3fff700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 0x7f2ff40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.834+0000 7f300affd700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 0x7f2ff40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.834+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9237+0+0 (secure 0 0 0) 0x7f30000cf590 con 0x7f300c06b580 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.834+0000 7f300affd700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 0x7f2ff40a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005950 tx=0x7f2ffc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:45:47.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:47.836+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3000014030 con 0x7f300c06b580 2024-06-06T07:45:48.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.032+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f300c02dca0 con 0x7f300c06b580 2024-06-06T07:45:48.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.034+0000 7f2ff3fff700 1 -- 172.21.15.87:0/2595545242 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24916 (secure 0 0 0) 0x7f30000cf090 con 0x7f300c06b580 2024-06-06T07:45:48.035 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:45:48.035 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":98,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:45:47.220282+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 msgr2=0x7f2ff40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 0x7f2ff40a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005950 tx=0x7f2ffc00b410 comp rx=0 tx=0).stop 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 msgr2=0x7f300c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f300000c920 tx=0x7f300000cc30 comp rx=0 tx=0).stop 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 shutdown_connections 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ff40a2cb0 0x7f2ff40a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:48.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300c108500 0x7f300c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f300c06abc0 0x7f300c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 --2- 172.21.15.87:0/2595545242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f300c06b580 0x7f300c1019d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:45:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 >> 172.21.15.87:0/2595545242 conn(0x7f300c078340 msgr2=0x7f300c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:45:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 shutdown_connections 2024-06-06T07:45:48.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:45:48.036+0000 7f301152a700 1 -- 172.21.15.87:0/2595545242 wait complete. 2024-06-06T07:45:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-06-06T07:45:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T07:45:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-06T07:45:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T07:45:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-06-06T07:45:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T07:45:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2595545242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:48 smithi087 ceph-mon[32276]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2595545242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:48 smithi195 ceph-mon[37040]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2595545242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:45:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:48 smithi181 ceph-mon[36790]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T07:45:48.826 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-06T07:45:48.826 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-06T07:45:48.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-06T07:45:48.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-06T07:45:48.828 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-06T07:45:48.828 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-06T07:45:48.829 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-06T07:45:48.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-06T07:45:48.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-06T07:45:48.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-06T07:45:48.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-06T07:45:48.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-06T07:45:49.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:49 smithi195 ceph-mon[37040]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T07:45:49.550 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.555 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.555 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.556 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:49 smithi181 ceph-mon[36790]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T07:45:49.563 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.568 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.579 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.588 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:49.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:49 smithi087 ceph-mon[32276]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T07:45:49.706 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:51 smithi195 ceph-mon[37040]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T07:45:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:51 smithi181 ceph-mon[36790]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T07:45:51.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:51 smithi087 ceph-mon[32276]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T07:45:52.425 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:52.426 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:45:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:53 smithi195 ceph-mon[37040]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-06-06T07:45:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:53 smithi181 ceph-mon[36790]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-06-06T07:45:53.571 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:53 smithi087 ceph-mon[32276]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-06-06T07:45:55.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:55 smithi087 ceph-mon[32276]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 89 B/s, 1 objects/s recovering 2024-06-06T07:45:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:55 smithi195 ceph-mon[37040]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 89 B/s, 1 objects/s recovering 2024-06-06T07:45:55.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:55 smithi181 ceph-mon[36790]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 89 B/s, 1 objects/s recovering 2024-06-06T07:45:57.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:57 smithi087 ceph-mon[32276]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 80 B/s, 1 objects/s recovering 2024-06-06T07:45:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:57 smithi195 ceph-mon[37040]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 80 B/s, 1 objects/s recovering 2024-06-06T07:45:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:57 smithi181 ceph-mon[36790]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 80 B/s, 1 objects/s recovering 2024-06-06T07:45:59.337 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:59 smithi087 ceph-mon[32276]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 71 B/s, 1 objects/s recovering 2024-06-06T07:45:59.337 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:45:59 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:45:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:59 smithi195 ceph-mon[37040]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 71 B/s, 1 objects/s recovering 2024-06-06T07:45:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:45:59 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:45:59.512 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:59 smithi181 ceph-mon[36790]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 71 B/s, 1 objects/s recovering 2024-06-06T07:45:59.512 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:45:59 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:46:01.466 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:01 smithi087 ceph-mon[32276]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:01.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:01 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:01 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:01 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:01 smithi195 ceph-mon[37040]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:01 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:01 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:01 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:01 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:01 smithi181 ceph-mon[36790]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:01.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:01 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:01.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:01 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 -- 172.21.15.87:0/1584651810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101620 msgr2=0x7f6424101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 --2- 172.21.15.87:0/1584651810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101620 0x7f6424101a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f64180099c0 tx=0x7f6418009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 -- 172.21.15.87:0/1584651810 shutdown_connections 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 --2- 172.21.15.87:0/1584651810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64241066b0 0x7f642410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 --2- 172.21.15.87:0/1584651810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101620 0x7f6424101a00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 --2- 172.21.15.87:0/1584651810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101fd0 0x7f6424105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 -- 172.21.15.87:0/1584651810 >> 172.21.15.87:0/1584651810 conn(0x7f6424078340 msgr2=0x7f6424078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:02.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.006+0000 7f642b005700 1 -- 172.21.15.87:0/1584651810 shutdown_connections 2024-06-06T07:46:02.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 -- 172.21.15.87:0/1584651810 wait complete. 2024-06-06T07:46:02.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 Processor -- start 2024-06-06T07:46:02.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 -- start start 2024-06-06T07:46:02.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101620 0x7f642419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 0x7f642419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64241066b0 0x7f6424198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64241116c0 con 0x7f6424101fd0 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6424111540 con 0x7f6424101620 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f642b005700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6424111840 con 0x7f64241066b0 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.007+0000 7f6423fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 0x7f642419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 0x7f642419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39438/0 (socket says 172.21.15.87:39438) 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 -- 172.21.15.87:0/1420656384 learned_addr learned my addr 172.21.15.87:0/1420656384 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6428da1700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101620 0x7f642419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f64295a2700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64241066b0 0x7f6424198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 -- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64241066b0 msgr2=0x7f6424198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64241066b0 0x7f6424198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 -- 172.21.15.87:0/1420656384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101620 msgr2=0x7f642419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101620 0x7f642419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6424199200 con 0x7f6424101fd0 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6428da1700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6424101620 0x7f642419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6423fff700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 0x7f642419edd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f641000c9e0 tx=0x7f641000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410007aa0 con 0x7f6424101fd0 2024-06-06T07:46:02.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410007c40 con 0x7f6424101fd0 2024-06-06T07:46:02.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410018780 con 0x7f6424101fd0 2024-06-06T07:46:02.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64241994f0 con 0x7f6424101fd0 2024-06-06T07:46:02.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.008+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64241a5760 con 0x7f6424101fd0 2024-06-06T07:46:02.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.009+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f642406ae90 con 0x7f6424101fd0 2024-06-06T07:46:02.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.010+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f641000f450 con 0x7f6424101fd0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6421ffb700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f64140a2d30 0x7f64140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6428da1700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f64140a2d30 0x7f64140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f64100cf8b0 con 0x7f6424101fd0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6421ffb700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] conn(0x7f64140ae110 0x7f64140b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6410015e70 con 0x7f64140ae110 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f64100cfd00 con 0x7f6424101fd0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.011+0000 7f6428da1700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f64140a2d30 0x7f64140a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6418000c00 tx=0x7f6418005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.012+0000 7f64295a2700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] conn(0x7f64140ae110 0x7f64140b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.012+0000 7f64295a2700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] conn(0x7f64140ae110 0x7f64140b05b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.013+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== osd.8 v2:172.21.15.195:6816/2643069924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6410015e70 con 0x7f64140ae110 2024-06-06T07:46:02.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.029+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 --> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6424102ab0 con 0x7f64140ae110 2024-06-06T07:46:02.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.029+0000 7f6421ffb700 1 -- 172.21.15.87:0/1420656384 <== osd.8 v2:172.21.15.195:6816/2643069924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6424102ab0 con 0x7f64140ae110 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] conn(0x7f64140ae110 msgr2=0x7f64140b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] conn(0x7f64140ae110 0x7f64140b05b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f64140a2d30 msgr2=0x7f64140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f64140a2d30 0x7f64140a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6418000c00 tx=0x7f6418005940 comp rx=0 tx=0).stop 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 msgr2=0x7f642419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 --2- 172.21.15.87:0/1420656384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424101fd0 0x7f642419edd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f641000c9e0 tx=0x7f641000cda0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f64295a2700 1 -- 172.21.15.87:0/1420656384 reap_dead start 2024-06-06T07:46:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 shutdown_connections 2024-06-06T07:46:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 >> 172.21.15.87:0/1420656384 conn(0x7f6424078340 msgr2=0x7f64241052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 shutdown_connections 2024-06-06T07:46:02.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.030+0000 7f642b005700 1 -- 172.21.15.87:0/1420656384 wait complete. 2024-06-06T07:46:02.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 -- 172.21.15.87:0/3645791437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501073f0 msgr2=0x7f6d501077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 --2- 172.21.15.87:0/3645791437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501073f0 0x7f6d501077d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009990 tx=0x7f6d40009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 -- 172.21.15.87:0/3645791437 shutdown_connections 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 --2- 172.21.15.87:0/3645791437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d50101d40 0x7f6d5010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 --2- 172.21.15.87:0/3645791437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501073f0 0x7f6d501077d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 --2- 172.21.15.87:0/3645791437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d50101380 0x7f6d50101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 -- 172.21.15.87:0/3645791437 >> 172.21.15.87:0/3645791437 conn(0x7f6d50078300 msgr2=0x7f6d50078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 -- 172.21.15.87:0/3645791437 shutdown_connections 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.231+0000 7f6d55067700 1 -- 172.21.15.87:0/3645791437 wait complete. 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.232+0000 7f6d55067700 1 Processor -- start 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.232+0000 7f6d55067700 1 -- start start 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 0x7f6d50199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d50101d40 0x7f6d5019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d501073f0 0x7f6d501a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d50111760 con 0x7f6d50101380 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501115e0 con 0x7f6d501073f0 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501118e0 con 0x7f6d50101d40 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 0x7f6d50199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 0x7f6d50199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39456/0 (socket says 172.21.15.87:39456) 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 -- 172.21.15.87:0/2932250638 learned_addr learned my addr 172.21.15.87:0/2932250638 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4e59c700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d50101d40 0x7f6d5019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4f59e700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d501073f0 0x7f6d501a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 -- 172.21.15.87:0/2932250638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d50101d40 msgr2=0x7f6d5019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d50101d40 0x7f6d5019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 -- 172.21.15.87:0/2932250638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d501073f0 msgr2=0x7f6d501a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d501073f0 0x7f6d501a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d501a51a0 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4f59e700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d501073f0 0x7f6d501a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d4ed9d700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 0x7f6d50199be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40027070 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d501a5370 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d40004190 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000f460 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.233+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501a5820 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.234+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d50062ba0 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.236+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6d4002a070 con 0x7f6d50101380 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.236+0000 7f6d37fff700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d380a2cb0 0x7f6d380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.236+0000 7f6d4e59c700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d380a2cb0 0x7f6d380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6d400d0c90 con 0x7f6d50101380 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d37fff700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] conn(0x7f6d380ae090 0x7f6d380b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d40001d30 con 0x7f6d380ae090 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f6d400d10e0 con 0x7f6d50101380 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d4f59e700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] conn(0x7f6d380ae090 0x7f6d380b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d4e59c700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d380a2cb0 0x7f6d380a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6d44006fd0 tx=0x7f6d44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.237+0000 7f6d4f59e700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] conn(0x7f6d380ae090 0x7f6d380b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:02.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.238+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== osd.6 v2:172.21.15.87:6818/2277253418 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d40001d30 con 0x7f6d380ae090 2024-06-06T07:46:02.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.254+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 --> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d50063050 con 0x7f6d380ae090 2024-06-06T07:46:02.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.254+0000 7f6d37fff700 1 -- 172.21.15.87:0/2932250638 <== osd.6 v2:172.21.15.87:6818/2277253418 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d50063050 con 0x7f6d380ae090 2024-06-06T07:46:02.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] conn(0x7f6d380ae090 msgr2=0x7f6d380b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] conn(0x7f6d380ae090 0x7f6d380b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d380a2cb0 msgr2=0x7f6d380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6d380a2cb0 0x7f6d380a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6d44006fd0 tx=0x7f6d44008040 comp rx=0 tx=0).stop 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 msgr2=0x7f6d50199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 --2- 172.21.15.87:0/2932250638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50101380 0x7f6d50199be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40003730 comp rx=0 tx=0).stop 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d4f59e700 1 -- 172.21.15.87:0/2932250638 reap_dead start 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 shutdown_connections 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 >> 172.21.15.87:0/2932250638 conn(0x7f6d50078300 msgr2=0x7f6d50100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 shutdown_connections 2024-06-06T07:46:02.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:02.255+0000 7f6d55067700 1 -- 172.21.15.87:0/2932250638 wait complete. 2024-06-06T07:46:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:03 smithi195 ceph-mon[37040]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:03 smithi181 ceph-mon[36790]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:03.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:03 smithi087 ceph-mon[32276]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 59 B/s, 1 objects/s recovering 2024-06-06T07:46:03.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- 172.21.15.87:0/3308947215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006ab30 msgr2=0x7fba2010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 --2- 172.21.15.87:0/3308947215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006ab30 0x7fba2010f320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fba1800b0a0 tx=0x7fba1800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- 172.21.15.87:0/3308947215 shutdown_connections 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 --2- 172.21.15.87:0/3308947215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba20108500 0x7fba201088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 --2- 172.21.15.87:0/3308947215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006ab30 0x7fba2010f320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 --2- 172.21.15.87:0/3308947215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba2006a170 0x7fba2006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- 172.21.15.87:0/3308947215 >> 172.21.15.87:0/3308947215 conn(0x7fba20078340 msgr2=0x7fba20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- 172.21.15.87:0/3308947215 shutdown_connections 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- 172.21.15.87:0/3308947215 wait complete. 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 Processor -- start 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.909+0000 7fba268e5700 1 -- start start 2024-06-06T07:46:03.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 0x7fba20101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba2006ab30 0x7fba201019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba20108500 0x7fba20101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fba20112760 con 0x7fba2006a170 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba201125e0 con 0x7fba2006ab30 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fba201128e0 con 0x7fba20108500 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 0x7fba20101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 0x7fba20101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39418/0 (socket says 172.21.15.87:39418) 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 -- 172.21.15.87:0/3112066777 learned_addr learned my addr 172.21.15.87:0/3112066777 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba24e82700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba20108500 0x7fba20101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba17fff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba2006ab30 0x7fba201019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 -- 172.21.15.87:0/3112066777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba20108500 msgr2=0x7fba20101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba20108500 0x7fba20101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 -- 172.21.15.87:0/3112066777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba2006ab30 msgr2=0x7fba201019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba2006ab30 0x7fba201019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba201a7140 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1ffff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 0x7fba20101460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fba1000d980 tx=0x7fba1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10016070 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba10004d10 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10005670 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.910+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba201a7430 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.911+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a7920 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.911+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba20062ba0 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fba10004750 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fba0c0a2d30 0x7fba0c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba17fff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fba0c0a2d30 0x7fba0c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fba100cf900 con 0x7fba2006a170 2024-06-06T07:46:03.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] conn(0x7fba0c0ae110 0x7fba0c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba10015ec0 con 0x7fba0c0ae110 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.913+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fba100cfd50 con 0x7fba2006a170 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.914+0000 7fba17fff700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fba0c0a2d30 0x7fba0c0a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba200788d0 tx=0x7fba080073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.914+0000 7fba24e82700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] conn(0x7fba0c0ae110 0x7fba0c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.914+0000 7fba24e82700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] conn(0x7fba0c0ae110 0x7fba0c0b05b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.915+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== osd.0 v2:172.21.15.87:6802/3682387176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba10015ec0 con 0x7fba0c0ae110 2024-06-06T07:46:03.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 --> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba20062ff0 con 0x7fba0c0ae110 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba1dffb700 1 -- 172.21.15.87:0/3112066777 <== osd.0 v2:172.21.15.87:6802/3682387176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fba20062ff0 con 0x7fba0c0ae110 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] conn(0x7fba0c0ae110 msgr2=0x7fba0c0b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] conn(0x7fba0c0ae110 0x7fba0c0b05b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fba0c0a2d30 msgr2=0x7fba0c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fba0c0a2d30 0x7fba0c0a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba200788d0 tx=0x7fba080073d0 comp rx=0 tx=0).stop 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 msgr2=0x7fba20101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba268e5700 1 --2- 172.21.15.87:0/3112066777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba2006a170 0x7fba20101460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fba1000d980 tx=0x7fba1000dc90 comp rx=0 tx=0).stop 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.931+0000 7fba24e82700 1 -- 172.21.15.87:0/3112066777 reap_dead start 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.932+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 shutdown_connections 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.932+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 >> 172.21.15.87:0/3112066777 conn(0x7fba20078340 msgr2=0x7fba200ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.932+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 shutdown_connections 2024-06-06T07:46:03.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:00.932+0000 7fba268e5700 1 -- 172.21.15.87:0/3112066777 wait complete. 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 -- 172.21.15.87:0/2477317468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec20106720 msgr2=0x7fec2010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 --2- 172.21.15.87:0/2477317468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec20106720 0x7fec2010e2c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec100099c0 tx=0x7fec10009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 -- 172.21.15.87:0/2477317468 shutdown_connections 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 --2- 172.21.15.87:0/2477317468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec201019d0 0x7fec20101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 --2- 172.21.15.87:0/2477317468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec20106720 0x7fec2010e2c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 --2- 172.21.15.87:0/2477317468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec20102380 0x7fec20105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 -- 172.21.15.87:0/2477317468 >> 172.21.15.87:0/2477317468 conn(0x7fec20078340 msgr2=0x7fec20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:04.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 -- 172.21.15.87:0/2477317468 shutdown_connections 2024-06-06T07:46:04.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 -- 172.21.15.87:0/2477317468 wait complete. 2024-06-06T07:46:04.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.740+0000 7fec25766700 1 Processor -- start 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 -- start start 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 0x7fec2019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec20102380 0x7fec2019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec20106720 0x7fec201978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec20111700 con 0x7fec201019d0 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fec20111580 con 0x7fec20102380 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec25766700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec20111880 con 0x7fec20106720 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 0x7fec2019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 0x7fec2019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39496/0 (socket says 172.21.15.87:39496) 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 -- 172.21.15.87:0/2836198739 learned_addr learned my addr 172.21.15.87:0/2836198739 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1e7fc700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec20102380 0x7fec2019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1f7fe700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec20106720 0x7fec201978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 -- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec20106720 msgr2=0x7fec201978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec20106720 0x7fec201978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 -- 172.21.15.87:0/2836198739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec20102380 msgr2=0x7fec2019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec20102380 0x7fec2019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20198040 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.741+0000 7fec1effd700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 0x7fec2019d6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c5a0 tx=0x7fec1800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.742+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18011880 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.742+0000 7fec25766700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec20198330 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.742+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec1801a400 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.742+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18010630 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.742+0000 7fec25766700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2006ab60 con 0x7fec201019d0 2024-06-06T07:46:04.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.743+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec2006b0f0 con 0x7fec201019d0 2024-06-06T07:46:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.744+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec18004ad0 con 0x7fec201019d0 2024-06-06T07:46:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.744+0000 7fec17fff700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fec0c0a2cb0 0x7fec0c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec1e7fc700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fec0c0a2cb0 0x7fec0c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fec180d22b0 con 0x7fec201019d0 2024-06-06T07:46:04.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec17fff700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] conn(0x7fec0c0ae010 0x7fec0c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec0c0b0b80 con 0x7fec0c0ae010 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fec180d2700 con 0x7fec201019d0 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec1e7fc700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fec0c0a2cb0 0x7fec0c0a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec20107a90 tx=0x7fec0800b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.745+0000 7fec1f7fe700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] conn(0x7fec0c0ae010 0x7fec0c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.746+0000 7fec1f7fe700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] conn(0x7fec0c0ae010 0x7fec0c0b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:04.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.746+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== osd.5 v2:172.21.15.195:6808/1086493667 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec0c0b0b80 con 0x7fec0c0ae010 2024-06-06T07:46:04.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.762+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 --> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec2006b4f0 con 0x7fec0c0ae010 2024-06-06T07:46:04.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec17fff700 1 -- 172.21.15.87:0/2836198739 <== osd.5 v2:172.21.15.195:6808/1086493667 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec2006b4f0 con 0x7fec0c0ae010 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] conn(0x7fec0c0ae010 msgr2=0x7fec0c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] conn(0x7fec0c0ae010 0x7fec0c0b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fec0c0a2cb0 msgr2=0x7fec0c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fec0c0a2cb0 0x7fec0c0a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec20107a90 tx=0x7fec0800b590 comp rx=0 tx=0).stop 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 msgr2=0x7fec2019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 --2- 172.21.15.87:0/2836198739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec201019d0 0x7fec2019d6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c5a0 tx=0x7fec1800c960 comp rx=0 tx=0).stop 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec1f7fe700 1 -- 172.21.15.87:0/2836198739 reap_dead start 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 shutdown_connections 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.763+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 >> 172.21.15.87:0/2836198739 conn(0x7fec20078340 msgr2=0x7fec200ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.764+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 shutdown_connections 2024-06-06T07:46:04.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.764+0000 7fec15ffb700 1 -- 172.21.15.87:0/2836198739 wait complete. 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1667622714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc107840 msgr2=0x7f9ebc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1667622714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc107840 0x7f9ebc10f3c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0099c0 tx=0x7f9eac009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1667622714 shutdown_connections 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1667622714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ebc102740 0x7f9ebc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1667622714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc107840 0x7f9ebc10f3c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1667622714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc1030f0 0x7f9ebc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1667622714 >> 172.21.15.87:0/1667622714 conn(0x7f9ebc0795f0 msgr2=0x7f9ebc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1667622714 shutdown_connections 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.615+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1667622714 wait complete. 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 Processor -- start 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 -- start start 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 0x7f9ebc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ebc1030f0 0x7f9ebc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc107840 0x7f9ebc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc112760 con 0x7f9ebc102740 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1125e0 con 0x7f9ebc1030f0 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ec0efe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1128e0 con 0x7f9ebc107840 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 0x7f9ebc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:05.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 0x7f9ebc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39470/0 (socket says 172.21.15.87:39470) 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 -- 172.21.15.87:0/1145148314 learned_addr learned my addr 172.21.15.87:0/1145148314 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9ebad9d700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc107840 0x7f9ebc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eb9d9b700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ebc1030f0 0x7f9ebc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 -- 172.21.15.87:0/1145148314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc107840 msgr2=0x7f9ebc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc107840 0x7f9ebc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 -- 172.21.15.87:0/1145148314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ebc1030f0 msgr2=0x7f9ebc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ebc1030f0 0x7f9ebc19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.616+0000 7f9eba59c700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc199100 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9eba59c700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 0x7f9ebc19e790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400ada0 tx=0x7f9eb400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4017020 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1993f0 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004bb0 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40054c0 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a5660 con 0x7f9ebc102740 2024-06-06T07:46:05.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.617+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ebc062c20 con 0x7f9ebc102740 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.619+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9eb400c980 con 0x7f9ebc102740 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.619+0000 7f9eb37fe700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9ea80a2cb0 0x7f9ea80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.619+0000 7f9eb9d9b700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9ea80a2cb0 0x7f9ea80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9eb40cf750 con 0x7f9ebc102740 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9eb37fe700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] conn(0x7f9ea80ae090 0x7f9ea80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ea80b0c00 con 0x7f9ea80ae090 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f9eb40cfba0 con 0x7f9ebc102740 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9ebad9d700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] conn(0x7f9ea80ae090 0x7f9ea80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9eb9d9b700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9ea80a2cb0 0x7f9ea80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.620+0000 7f9ebad9d700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] conn(0x7f9ea80ae090 0x7f9ea80b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.621+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== osd.3 v2:172.21.15.87:6810/4203203594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ea80b0c00 con 0x7f9ea80ae090 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.637+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 --> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ebc063020 con 0x7f9ea80ae090 2024-06-06T07:46:05.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.637+0000 7f9eb37fe700 1 -- 172.21.15.87:0/1145148314 <== osd.3 v2:172.21.15.87:6810/4203203594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ebc063020 con 0x7f9ea80ae090 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] conn(0x7f9ea80ae090 msgr2=0x7f9ea80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] conn(0x7f9ea80ae090 0x7f9ea80b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9ea80a2cb0 msgr2=0x7f9ea80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9ea80a2cb0 0x7f9ea80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea40058e0 comp rx=0 tx=0).stop 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 msgr2=0x7f9ebc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 --2- 172.21.15.87:0/1145148314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ebc102740 0x7f9ebc19e790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400ada0 tx=0x7f9eb400c330 comp rx=0 tx=0).stop 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ebad9d700 1 -- 172.21.15.87:0/1145148314 reap_dead start 2024-06-06T07:46:05.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 shutdown_connections 2024-06-06T07:46:05.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 >> 172.21.15.87:0/1145148314 conn(0x7f9ebc0795f0 msgr2=0x7f9ebc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:05.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 shutdown_connections 2024-06-06T07:46:05.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:04.638+0000 7f9ec0efe700 1 -- 172.21.15.87:0/1145148314 wait complete. 2024-06-06T07:46:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:05 smithi195 ceph-mon[37040]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-06-06T07:46:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:05 smithi181 ceph-mon[36790]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-06-06T07:46:05.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:05 smithi087 ceph-mon[32276]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-06-06T07:46:06.140 INFO:teuthology.orchestra.run.smithi087.stdout:197568495682 2024-06-06T07:46:06.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-06T07:46:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.322+0000 7f416eccb700 1 -- 172.21.15.87:0/1454695363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 msgr2=0x7f4168106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.322+0000 7f416eccb700 1 --2- 172.21.15.87:0/1454695363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168106020 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41540099c0 tx=0x7f4154009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 -- 172.21.15.87:0/1454695363 shutdown_connections 2024-06-06T07:46:06.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 --2- 172.21.15.87:0/1454695363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4168106750 0x7f416810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 --2- 172.21.15.87:0/1454695363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4168101df0 0x7f41681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 --2- 172.21.15.87:0/1454695363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168106020 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 -- 172.21.15.87:0/1454695363 >> 172.21.15.87:0/1454695363 conn(0x7f4168077a00 msgr2=0x7f41680fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:06.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 -- 172.21.15.87:0/1454695363 shutdown_connections 2024-06-06T07:46:06.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 -- 172.21.15.87:0/1454695363 wait complete. 2024-06-06T07:46:06.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 Processor -- start 2024-06-06T07:46:06.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.323+0000 7f416eccb700 1 -- start start 2024-06-06T07:46:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4168101df0 0x7f4168199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4168106750 0x7f41681a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f4167fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f4167fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39512/0 (socket says 172.21.15.87:39512) 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f4167fff700 1 -- 172.21.15.87:0/2503962847 learned_addr learned my addr 172.21.15.87:0/2503962847 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4168111660 con 0x7f41681027a0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41681114e0 con 0x7f4168101df0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41681117e0 con 0x7f4168106750 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416ca67700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4168101df0 0x7f4168199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.324+0000 7f416d268700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4168106750 0x7f41681a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 -- 172.21.15.87:0/2503962847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4168106750 msgr2=0x7f41681a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4168106750 0x7f41681a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 -- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4168101df0 msgr2=0x7f4168199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4168101df0 0x7f4168199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681a4fe0 con 0x7f41681027a0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4167fff700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168199f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4154006010 tx=0x7f4154009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154026070 con 0x7f41681027a0 2024-06-06T07:46:06.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4154004cd0 con 0x7f41681027a0 2024-06-06T07:46:06.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681a51b0 con 0x7f41681027a0 2024-06-06T07:46:06.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.325+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154003ba0 con 0x7f41681027a0 2024-06-06T07:46:06.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.326+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681a5550 con 0x7f41681027a0 2024-06-06T07:46:06.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.326+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4168062be0 con 0x7f41681027a0 2024-06-06T07:46:06.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.328+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4154029070 con 0x7f41681027a0 2024-06-06T07:46:06.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f4165ffb700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f41580a2cb0 0x7f41580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:06.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f416ca67700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f41580a2cb0 0x7f41580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:06.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f41540cfec0 con 0x7f41681027a0 2024-06-06T07:46:06.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f4165ffb700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] conn(0x7f41580ae090 0x7f41580b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f415400b0b0 con 0x7f41580ae090 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f41540d2580 con 0x7f41681027a0 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.329+0000 7f416ca67700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f41580a2cb0 0x7f41580a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f415c005950 tx=0x7f415c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.330+0000 7f416d268700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] conn(0x7f41580ae090 0x7f41580b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.330+0000 7f416d268700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] conn(0x7f41580ae090 0x7f41580b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:06.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.331+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== osd.7 v2:172.21.15.181:6816/1036398045 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f415400b0b0 con 0x7f41580ae090 2024-06-06T07:46:06.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.362+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 --> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4168063090 con 0x7f41580ae090 2024-06-06T07:46:06.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.362+0000 7f4165ffb700 1 -- 172.21.15.87:0/2503962847 <== osd.7 v2:172.21.15.181:6816/1036398045 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4168063090 con 0x7f41580ae090 2024-06-06T07:46:06.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] conn(0x7f41580ae090 msgr2=0x7f41580b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] conn(0x7f41580ae090 0x7f41580b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f41580a2cb0 msgr2=0x7f41580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f41580a2cb0 0x7f41580a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f415c005950 tx=0x7f415c00b410 comp rx=0 tx=0).stop 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 msgr2=0x7f4168199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 --2- 172.21.15.87:0/2503962847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41681027a0 0x7f4168199f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4154006010 tx=0x7f4154009f80 comp rx=0 tx=0).stop 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416d268700 1 -- 172.21.15.87:0/2503962847 reap_dead start 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 shutdown_connections 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 >> 172.21.15.87:0/2503962847 conn(0x7f4168077a00 msgr2=0x7f4168104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 shutdown_connections 2024-06-06T07:46:06.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:06.363+0000 7f416eccb700 1 -- 172.21.15.87:0/2503962847 wait complete. 2024-06-06T07:46:07.326 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:07 smithi087 ceph-mon[32276]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:07 smithi195 ceph-mon[37040]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:07 smithi181 ceph-mon[36790]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:07.575 INFO:teuthology.orchestra.run.smithi087.stdout:163208757319 2024-06-06T07:46:07.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-06T07:46:08.981 INFO:teuthology.orchestra.run.smithi087.stdout:34359738463 2024-06-06T07:46:08.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 -- 172.21.15.87:0/1246476639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c106be0 msgr2=0x7f928c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 --2- 172.21.15.87:0/1246476639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c106be0 0x7f928c106fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f92800099c0 tx=0x7f9280009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 -- 172.21.15.87:0/1246476639 shutdown_connections 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 --2- 172.21.15.87:0/1246476639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c1014f0 0x7f928c10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 --2- 172.21.15.87:0/1246476639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c106be0 0x7f928c106fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 --2- 172.21.15.87:0/1246476639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c100b30 0x7f928c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 -- 172.21.15.87:0/1246476639 >> 172.21.15.87:0/1246476639 conn(0x7f928c077990 msgr2=0x7f928c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:09.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 -- 172.21.15.87:0/1246476639 shutdown_connections 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 -- 172.21.15.87:0/1246476639 wait complete. 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.170+0000 7f92930de700 1 Processor -- start 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 -- start start 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c100b30 0x7f928c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 0x7f928c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c106be0 0x7f928c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f928c111790 con 0x7f928c1014f0 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f928c111610 con 0x7f928c106be0 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f92930de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f928c111910 con 0x7f928c100b30 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 0x7f928c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 0x7f928c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45102/0 (socket says 172.21.15.87:45102) 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 -- 172.21.15.87:0/2428674867 learned_addr learned my addr 172.21.15.87:0/2428674867 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:09.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f929167b700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c106be0 0x7f928c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f9290e7a700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c100b30 0x7f928c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 -- 172.21.15.87:0/2428674867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c100b30 msgr2=0x7f928c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c100b30 0x7f928c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 -- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c106be0 msgr2=0x7f928c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c106be0 0x7f928c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f928bfff700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c19ce50 con 0x7f928c1014f0 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f9290e7a700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928c100b30 0x7f928c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.171+0000 7f929167b700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928c106be0 0x7f928c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f928bfff700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 0x7f928c197e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f927800b530 tx=0x7f927800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278015020 con 0x7f928c1014f0 2024-06-06T07:46:09.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c19d140 con 0x7f928c1014f0 2024-06-06T07:46:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9278011d10 con 0x7f928c1014f0 2024-06-06T07:46:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278010e50 con 0x7f928c1014f0 2024-06-06T07:46:09.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.172+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c19d680 con 0x7f928c1014f0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.173+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f928c19dc90 con 0x7f928c1014f0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.174+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9278018020 con 0x7f928c1014f0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9289ffb700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f927c0a2e50 0x7f927c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f92780cf3c0 con 0x7f928c1014f0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9290e7a700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f927c0a2e50 0x7f927c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9289ffb700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] conn(0x7f927c0ae230 0x7f927c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92780044e0 con 0x7f927c0ae230 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f92780cf810 con 0x7f928c1014f0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f9290e7a700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f927c0a2e50 0x7f927c0a5310 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f928000b580 tx=0x7f9280009f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.175+0000 7f929167b700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] conn(0x7f927c0ae230 0x7f927c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:09.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.176+0000 7f929167b700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] conn(0x7f927c0ae230 0x7f927c0b06d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:09.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.177+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== osd.4 v2:172.21.15.181:6808/3265557015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92780044e0 con 0x7f927c0ae230 2024-06-06T07:46:09.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.192+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 --> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f928c062b20 con 0x7f927c0ae230 2024-06-06T07:46:09.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.193+0000 7f9289ffb700 1 -- 172.21.15.87:0/2428674867 <== osd.4 v2:172.21.15.181:6808/3265557015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f928c062b20 con 0x7f927c0ae230 2024-06-06T07:46:09.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] conn(0x7f927c0ae230 msgr2=0x7f927c0b06d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] conn(0x7f927c0ae230 0x7f927c0b06d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f927c0a2e50 msgr2=0x7f927c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f927c0a2e50 0x7f927c0a5310 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f928000b580 tx=0x7f9280009f30 comp rx=0 tx=0).stop 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 msgr2=0x7f928c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 --2- 172.21.15.87:0/2428674867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f928c1014f0 0x7f928c197e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f927800b530 tx=0x7f927800b8f0 comp rx=0 tx=0).stop 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f929167b700 1 -- 172.21.15.87:0/2428674867 reap_dead start 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 shutdown_connections 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 >> 172.21.15.87:0/2428674867 conn(0x7f928c077990 msgr2=0x7f928c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 shutdown_connections 2024-06-06T07:46:09.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:09.194+0000 7f92930de700 1 -- 172.21.15.87:0/2428674867 wait complete. 2024-06-06T07:46:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:09 smithi195 ceph-mon[37040]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:09 smithi181 ceph-mon[36790]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:09.590 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:09 smithi087 ceph-mon[32276]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:10.288 INFO:teuthology.orchestra.run.smithi087.stdout:219043332157 2024-06-06T07:46:10.289 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-06T07:46:10.969 INFO:teuthology.orchestra.run.smithi087.stdout:253403070519 2024-06-06T07:46:10.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-06T07:46:11.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 -- 172.21.15.87:0/3603324281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae81030f0 msgr2=0x7faae8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 --2- 172.21.15.87:0/3603324281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae81030f0 0x7faae8107110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faad40099c0 tx=0x7faad4009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.418 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:11 smithi087 ceph-mon[32276]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:11.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 -- 172.21.15.87:0/3603324281 shutdown_connections 2024-06-06T07:46:11.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 --2- 172.21.15.87:0/3603324281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 0x7faae810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 --2- 172.21.15.87:0/3603324281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae8102740 0x7faae8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 --2- 172.21.15.87:0/3603324281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae81030f0 0x7faae8107110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 -- 172.21.15.87:0/3603324281 >> 172.21.15.87:0/3603324281 conn(0x7faae80795f0 msgr2=0x7faae8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 -- 172.21.15.87:0/3603324281 shutdown_connections 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.417+0000 7faaeeeca700 1 -- 172.21.15.87:0/3603324281 wait complete. 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 Processor -- start 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 -- start start 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 0x7faae819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae81030f0 0x7faae819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 0x7faae819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faae8112750 con 0x7faae8102740 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faae81125d0 con 0x7faae8107840 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaeeeca700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faae81128d0 con 0x7faae81030f0 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 0x7faae819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 0x7faae819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45110/0 (socket says 172.21.15.87:45110) 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 -- 172.21.15.87:0/2686346522 learned_addr learned my addr 172.21.15.87:0/2686346522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:11.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faae7fff700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae81030f0 0x7faae819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaed467700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 0x7faae819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 -- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae81030f0 msgr2=0x7faae819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae81030f0 0x7faae819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 -- 172.21.15.87:0/2686346522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 msgr2=0x7faae819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 0x7faae819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaecc66700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae819fd90 con 0x7faae8102740 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faae7fff700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faae81030f0 0x7faae819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:11.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.418+0000 7faaed467700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae8107840 0x7faae819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faaecc66700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 0x7faae819add0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d980 tx=0x7faadc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc016070 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae81a0080 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc004500 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc004d10 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.419+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae81a7900 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.420+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faae8101ff0 con 0x7faae8102740 2024-06-06T07:46:11.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.422+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faadc00bbf0 con 0x7faae8102740 2024-06-06T07:46:11.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.422+0000 7faae5ffb700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7faad80a2cb0 0x7faad80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.422+0000 7faae7fff700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7faad80a2cb0 0x7faad80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.422+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7faadc0cfb20 con 0x7faae8102740 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faae5ffb700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] conn(0x7faad80ae090 0x7faad80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faadc015ec0 con 0x7faad80ae090 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7faadc0cff70 con 0x7faae8102740 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faae7fff700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7faad80a2cb0 0x7faad80a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faae810f770 tx=0x7faad4005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faaed467700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] conn(0x7faad80ae090 0x7faad80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.423+0000 7faaed467700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] conn(0x7faad80ae090 0x7faad80b0530 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.424+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== osd.11 v2:172.21.15.195:6824/1273097292 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faadc015ec0 con 0x7faad80ae090 2024-06-06T07:46:11.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.440+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 --> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faae8103bd0 con 0x7faad80ae090 2024-06-06T07:46:11.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faae5ffb700 1 -- 172.21.15.87:0/2686346522 <== osd.11 v2:172.21.15.195:6824/1273097292 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faae8103bd0 con 0x7faad80ae090 2024-06-06T07:46:11.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] conn(0x7faad80ae090 msgr2=0x7faad80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] conn(0x7faad80ae090 0x7faad80b0530 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7faad80a2cb0 msgr2=0x7faad80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7faad80a2cb0 0x7faad80a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faae810f770 tx=0x7faad4005ab0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 msgr2=0x7faae819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 --2- 172.21.15.87:0/2686346522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faae8102740 0x7faae819add0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d980 tx=0x7faadc00dd40 comp rx=0 tx=0).stop 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaed467700 1 -- 172.21.15.87:0/2686346522 reap_dead start 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 shutdown_connections 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 >> 172.21.15.87:0/2686346522 conn(0x7faae80795f0 msgr2=0x7faae8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 shutdown_connections 2024-06-06T07:46:11.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.441+0000 7faaeeeca700 1 -- 172.21.15.87:0/2686346522 wait complete. 2024-06-06T07:46:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:11 smithi195 ceph-mon[37040]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:11 smithi181 ceph-mon[36790]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:11.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.641+0000 7f316b1b8700 1 -- 172.21.15.87:0/2169560728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 msgr2=0x7f3164101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.641+0000 7f316b1b8700 1 --2- 172.21.15.87:0/2169560728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f3164101800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3150009990 tx=0x7f3150009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 -- 172.21.15.87:0/2169560728 shutdown_connections 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 --2- 172.21.15.87:0/2169560728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 0x7f31641077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 --2- 172.21.15.87:0/2169560728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 0x7f316410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 --2- 172.21.15.87:0/2169560728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f3164101800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 -- 172.21.15.87:0/2169560728 >> 172.21.15.87:0/2169560728 conn(0x7f3164078300 msgr2=0x7f3164078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 -- 172.21.15.87:0/2169560728 shutdown_connections 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 -- 172.21.15.87:0/2169560728 wait complete. 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 Processor -- start 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.642+0000 7f316b1b8700 1 -- start start 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f316419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 0x7f316419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 0x7f3164198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31641116e0 con 0x7f3164101380 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3164111560 con 0x7f31641073f0 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f316b1b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3164111860 con 0x7f3164101d40 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f316419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f316419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45132/0 (socket says 172.21.15.87:45132) 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 -- 172.21.15.87:0/3463662627 learned_addr learned my addr 172.21.15.87:0/3463662627 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3163fff700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 0x7f316419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3169755700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 0x7f3164198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 -- 172.21.15.87:0/3463662627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 msgr2=0x7f316419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 0x7f316419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 -- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 msgr2=0x7f3164198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 0x7f3164198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641991b0 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3169755700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31641073f0 0x7f3164198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3163fff700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3164101d40 0x7f316419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3168f54700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f316419e840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f315800b530 tx=0x7f315800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.643+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158015020 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.644+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3158011d10 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.644+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158010e50 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.644+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31641994a0 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.644+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641a5710 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.645+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f316406ad00 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.647+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3158018020 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3161ffb700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31540a2ed0 0x7f31540a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3163fff700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31540a2ed0 0x7f31540a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f31580cf330 con 0x7f3164101380 2024-06-06T07:46:11.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3161ffb700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] conn(0x7f31540ae2b0 0x7f31540b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3158012e80 con 0x7f31540ae2b0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.648+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f31580cf780 con 0x7f3164101380 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.649+0000 7f3169755700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] conn(0x7f31540ae2b0 0x7f31540b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.649+0000 7f3163fff700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31540a2ed0 0x7f31540a5390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3150006010 tx=0x7f315000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.649+0000 7f3169755700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] conn(0x7f31540ae2b0 0x7f31540b0750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.650+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== osd.10 v2:172.21.15.181:6824/649796316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3158012e80 con 0x7f31540ae2b0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.667+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 --> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3164102aa0 con 0x7f31540ae2b0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f3161ffb700 1 -- 172.21.15.87:0/3463662627 <== osd.10 v2:172.21.15.181:6824/649796316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3164102aa0 con 0x7f31540ae2b0 2024-06-06T07:46:11.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] conn(0x7f31540ae2b0 msgr2=0x7f31540b0750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] conn(0x7f31540ae2b0 0x7f31540b0750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31540a2ed0 msgr2=0x7f31540a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f31540a2ed0 0x7f31540a5390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3150006010 tx=0x7f315000b540 comp rx=0 tx=0).stop 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 msgr2=0x7f316419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.668+0000 7f316b1b8700 1 --2- 172.21.15.87:0/3463662627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3164101380 0x7f316419e840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f315800b530 tx=0x7f315800b8f0 comp rx=0 tx=0).stop 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.669+0000 7f3169755700 1 -- 172.21.15.87:0/3463662627 reap_dead start 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.669+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 shutdown_connections 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.669+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 >> 172.21.15.87:0/3463662627 conn(0x7f3164078300 msgr2=0x7f3164105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.669+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 shutdown_connections 2024-06-06T07:46:11.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:11.669+0000 7f316b1b8700 1 -- 172.21.15.87:0/3463662627 wait complete. 2024-06-06T07:46:12.850 INFO:teuthology.orchestra.run.smithi087.stdout:141733920844 2024-06-06T07:46:12.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 -- 172.21.15.87:0/1482219706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 msgr2=0x7f9f2c10e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 --2- 172.21.15.87:0/1482219706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c10e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009990 tx=0x7f9f1c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 -- 172.21.15.87:0/1482219706 shutdown_connections 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 --2- 172.21.15.87:0/1482219706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2c06aa30 0x7f9f2c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 --2- 172.21.15.87:0/1482219706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c10ef30 0x7f9f2c111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 --2- 172.21.15.87:0/1482219706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c10e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 -- 172.21.15.87:0/1482219706 >> 172.21.15.87:0/1482219706 conn(0x7f9f2c078300 msgr2=0x7f9f2c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:13.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 -- 172.21.15.87:0/1482219706 shutdown_connections 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.238+0000 7f9f344b8700 1 -- 172.21.15.87:0/1482219706 wait complete. 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 Processor -- start 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 -- start start 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c06aa30 0x7f9f2c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2c10ef30 0x7f9f2c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1149c0 con 0x7f9f2c06b350 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c114840 con 0x7f9f2c10ef30 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f344b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c114b40 con 0x7f9f2c06aa30 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45156/0 (socket says 172.21.15.87:45156) 2024-06-06T07:46:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 -- 172.21.15.87:0/4161744262 learned_addr learned my addr 172.21.15.87:0/4161744262 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:13.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f32a55700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2c10ef30 0x7f9f2c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:13.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f32254700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c06aa30 0x7f9f2c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 -- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c06aa30 msgr2=0x7f9f2c19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c06aa30 0x7f9f2c19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 -- 172.21.15.87:0/4161744262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2c10ef30 msgr2=0x7f9f2c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2c10ef30 0x7f9f2c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f31a53700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c19fef0 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.239+0000 7f9f32254700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f2c06aa30 0x7f9f2c19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f31a53700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c19b470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0038b0 tx=0x7f9f1c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c027070 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c004050 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c1a0180 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c022cd0 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.240+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a7a60 con 0x7f9f2c06b350 2024-06-06T07:46:13.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.241+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f2c062ba0 con 0x7f9f2c06b350 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.242+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9f1c02a070 con 0x7f9f2c06b350 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.242+0000 7f9f237fe700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9f180a2cb0 0x7f9f180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f32254700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9f180a2cb0 0x7f9f180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9f1c0d0f10 con 0x7f9f2c06b350 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f237fe700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] conn(0x7f9f180ae090 0x7f9f180b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f1c024ec0 con 0x7f9f180ae090 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c0d1360 con 0x7f9f2c06b350 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f32254700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9f180a2cb0 0x7f9f180a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f28005950 tx=0x7f9f280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f32a55700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] conn(0x7f9f180ae090 0x7f9f180b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.243+0000 7f9f32a55700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] conn(0x7f9f180ae090 0x7f9f180b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.244+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== osd.2 v2:172.21.15.195:6800/2910401977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f1c024ec0 con 0x7f9f180ae090 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.261+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 --> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f2c063020 con 0x7f9f180ae090 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.261+0000 7f9f237fe700 1 -- 172.21.15.87:0/4161744262 <== osd.2 v2:172.21.15.195:6800/2910401977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f2c063020 con 0x7f9f180ae090 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.261+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] conn(0x7f9f180ae090 msgr2=0x7f9f180b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.261+0000 7f9f344b8700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] conn(0x7f9f180ae090 0x7f9f180b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9f180a2cb0 msgr2=0x7f9f180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9f180a2cb0 0x7f9f180a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f28005950 tx=0x7f9f280058e0 comp rx=0 tx=0).stop 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 msgr2=0x7f9f2c19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 --2- 172.21.15.87:0/4161744262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2c06b350 0x7f9f2c19b470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0038b0 tx=0x7f9f1c009f50 comp rx=0 tx=0).stop 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f32a55700 1 -- 172.21.15.87:0/4161744262 reap_dead start 2024-06-06T07:46:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 shutdown_connections 2024-06-06T07:46:13.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 >> 172.21.15.87:0/4161744262 conn(0x7f9f2c078300 msgr2=0x7f9f2c110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:13.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 shutdown_connections 2024-06-06T07:46:13.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:12.262+0000 7f9f344b8700 1 -- 172.21.15.87:0/4161744262 wait complete. 2024-06-06T07:46:13.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:13 smithi087 ceph-mon[32276]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:13 smithi195 ceph-mon[37040]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:13 smithi181 ceph-mon[36790]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:13.919 INFO:teuthology.orchestra.run.smithi087.stdout:107374182481 2024-06-06T07:46:13.920 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 -- 172.21.15.87:0/237083752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 msgr2=0x7f19ec1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 --2- 172.21.15.87:0/237083752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec1029d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009990 tx=0x7f19dc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 -- 172.21.15.87:0/237083752 shutdown_connections 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 --2- 172.21.15.87:0/237083752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec1084f0 0x7f19ec1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 --2- 172.21.15.87:0/237083752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec102f10 0x7f19ec10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 --2- 172.21.15.87:0/237083752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec1029d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.089+0000 7f19f45a1700 1 -- 172.21.15.87:0/237083752 >> 172.21.15.87:0/237083752 conn(0x7f19ec079650 msgr2=0x7f19ec079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- 172.21.15.87:0/237083752 shutdown_connections 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- 172.21.15.87:0/237083752 wait complete. 2024-06-06T07:46:14.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 Processor -- start 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- start start 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec102f10 0x7f19ec074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec1084f0 0x7f19ec075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec112760 con 0x7f19ec102550 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec1125e0 con 0x7f19ec102f10 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.090+0000 7f19f45a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec1128e0 con 0x7f19ec1084f0 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45162/0 (socket says 172.21.15.87:45162) 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 -- 172.21.15.87:0/657037038 learned_addr learned my addr 172.21.15.87:0/657037038 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f2b3e700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec1084f0 0x7f19ec075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:14.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f1b3c700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec102f10 0x7f19ec074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:14.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 -- 172.21.15.87:0/657037038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec1084f0 msgr2=0x7f19ec075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec1084f0 0x7f19ec075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 -- 172.21.15.87:0/657037038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec102f10 msgr2=0x7f19ec074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec102f10 0x7f19ec074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec1a7150 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f2b3e700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec1084f0 0x7f19ec075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f233d700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec074790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c920 tx=0x7f19e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8004a30 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec1a7440 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8005500 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.091+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007d50 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.092+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec1a79a0 con 0x7f19ec102550 2024-06-06T07:46:14.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.092+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19d0001030 con 0x7f19ec102550 2024-06-06T07:46:14.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f19e8004bd0 con 0x7f19ec102550 2024-06-06T07:46:14.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f19d80a2d30 0x7f19d80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:14.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19f1b3c700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f19d80a2d30 0x7f19d80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:14.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f19e80cf4b0 con 0x7f19ec102550 2024-06-06T07:46:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] conn(0x7f19d80ae110 0x7f19d80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19d80b0c80 con 0x7f19d80ae110 2024-06-06T07:46:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.095+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f19e80cf900 con 0x7f19ec102550 2024-06-06T07:46:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.096+0000 7f19f2b3e700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] conn(0x7f19d80ae110 0x7f19d80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.096+0000 7f19f1b3c700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f19d80a2d30 0x7f19d80a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0098e0 tx=0x7f19dc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.096+0000 7f19f2b3e700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] conn(0x7f19d80ae110 0x7f19d80b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.097+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== osd.9 v2:172.21.15.87:6826/2016619295 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19d80b0c80 con 0x7f19d80ae110 2024-06-06T07:46:14.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.114+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 --> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19d0002e60 con 0x7f19d80ae110 2024-06-06T07:46:14.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19e37fe700 1 -- 172.21.15.87:0/657037038 <== osd.9 v2:172.21.15.87:6826/2016619295 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f19d0002e60 con 0x7f19d80ae110 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] conn(0x7f19d80ae110 msgr2=0x7f19d80b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] conn(0x7f19d80ae110 0x7f19d80b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f19d80a2d30 msgr2=0x7f19d80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f19d80a2d30 0x7f19d80a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0098e0 tx=0x7f19dc005960 comp rx=0 tx=0).stop 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 msgr2=0x7f19ec074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 --2- 172.21.15.87:0/657037038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19ec102550 0x7f19ec074790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c920 tx=0x7f19e800cc30 comp rx=0 tx=0).stop 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f2b3e700 1 -- 172.21.15.87:0/657037038 reap_dead start 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 shutdown_connections 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.115+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 >> 172.21.15.87:0/657037038 conn(0x7f19ec079650 msgr2=0x7f19ec100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.116+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 shutdown_connections 2024-06-06T07:46:14.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:14.116+0000 7f19f45a1700 1 -- 172.21.15.87:0/657037038 wait complete. 2024-06-06T07:46:14.514 INFO:teuthology.orchestra.run.smithi087.stdout:309237645359 2024-06-06T07:46:14.514 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-06T07:46:15.287 INFO:teuthology.orchestra.run.smithi087.stdout:343597383722 2024-06-06T07:46:15.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-06T07:46:15.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:15 smithi195 ceph-mon[37040]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:15.535 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:15 smithi087 ceph-mon[32276]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:15.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:15 smithi181 ceph-mon[36790]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 -- 172.21.15.87:0/1772546067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 msgr2=0x7f36d4111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 --2- 172.21.15.87:0/1772546067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d4111570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b0a0 tx=0x7f36cc00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 -- 172.21.15.87:0/1772546067 shutdown_connections 2024-06-06T07:46:15.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 --2- 172.21.15.87:0/1772546067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d4111570 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 --2- 172.21.15.87:0/1772546067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d410a6a0 0x7f36d410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 --2- 172.21.15.87:0/1772546067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d41046a0 0x7f36d4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 -- 172.21.15.87:0/1772546067 >> 172.21.15.87:0/1772546067 conn(0x7f36d4100190 msgr2=0x7f36d41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 -- 172.21.15.87:0/1772546067 shutdown_connections 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.561+0000 7f36d9870700 1 -- 172.21.15.87:0/1772546067 wait complete. 2024-06-06T07:46:15.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.563+0000 7f36d9870700 1 Processor -- start 2024-06-06T07:46:15.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.563+0000 7f36d9870700 1 -- start start 2024-06-06T07:46:15.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.563+0000 7f36d9870700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 0x7f36d41a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d9870700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 0x7f36d41a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 0x7f36d41a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:32924/0 (socket says 172.21.15.87:32924) 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d41a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45170/0 (socket says 172.21.15.87:45170) 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d27fc700 1 -- 172.21.15.87:0/3232007414 learned_addr learned my addr 172.21.15.87:0/3232007414 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:15.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d9870700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d410a6a0 0x7f36d419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114a80 con 0x7f36d4105060 2024-06-06T07:46:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114900 con 0x7f36d410a6a0 2024-06-06T07:46:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114c00 con 0x7f36d41046a0 2024-06-06T07:46:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.564+0000 7f36d37fe700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d410a6a0 0x7f36d419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:15.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 -- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d410a6a0 msgr2=0x7f36d419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d410a6a0 0x7f36d419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 -- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 msgr2=0x7f36d41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d419b170 con 0x7f36d41046a0 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d27fc700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36d4105060 0x7f36d41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d2ffd700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 0x7f36d41a0850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9f0 tx=0x7f36c400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4004220 con 0x7f36d41046a0 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.565+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4004d10 con 0x7f36d41046a0 2024-06-06T07:46:15.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.566+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d419b460 con 0x7f36d41046a0 2024-06-06T07:46:15.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.566+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a7720 con 0x7f36d41046a0 2024-06-06T07:46:15.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.566+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4004220 con 0x7f36d41046a0 2024-06-06T07:46:15.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.567+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36d4100f90 con 0x7f36d41046a0 2024-06-06T07:46:15.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.570+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f36c40043c0 con 0x7f36d41046a0 2024-06-06T07:46:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d886e700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f36bc0a3130 0x7f36bc0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f36c4004020 con 0x7f36d41046a0 2024-06-06T07:46:15.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d27fc700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f36bc0a3130 0x7f36bc0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d886e700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] conn(0x7f36bc0ae510 0x7f36bc0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36bc0b1080 con 0x7f36bc0ae510 2024-06-06T07:46:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.571+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f36c40cf270 con 0x7f36d41046a0 2024-06-06T07:46:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.572+0000 7f36d37fe700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] conn(0x7f36bc0ae510 0x7f36bc0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:15.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.572+0000 7f36d27fc700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f36bc0a3130 0x7f36bc0a55f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f36d4102b30 tx=0x7f36c800a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:15.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.572+0000 7f36d37fe700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] conn(0x7f36bc0ae510 0x7f36bc0b09b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:15.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.573+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== osd.1 v2:172.21.15.181:6800/1976649118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36bc0b1080 con 0x7f36bc0ae510 2024-06-06T07:46:15.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.592+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 --> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36d4105de0 con 0x7f36bc0ae510 2024-06-06T07:46:15.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d886e700 1 -- 172.21.15.87:0/3232007414 <== osd.1 v2:172.21.15.181:6800/1976649118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f36d4105de0 con 0x7f36bc0ae510 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] conn(0x7f36bc0ae510 msgr2=0x7f36bc0b09b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] conn(0x7f36bc0ae510 0x7f36bc0b09b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f36bc0a3130 msgr2=0x7f36bc0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f36bc0a3130 0x7f36bc0a55f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f36d4102b30 tx=0x7f36c800a400 comp rx=0 tx=0).stop 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 msgr2=0x7f36d41a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d9870700 1 --2- 172.21.15.87:0/3232007414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d41046a0 0x7f36d41a0850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9f0 tx=0x7f36c400ed00 comp rx=0 tx=0).stop 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.593+0000 7f36d37fe700 1 -- 172.21.15.87:0/3232007414 reap_dead start 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.594+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 shutdown_connections 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.594+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 >> 172.21.15.87:0/3232007414 conn(0x7f36d4100190 msgr2=0x7f36d4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.594+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 shutdown_connections 2024-06-06T07:46:15.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:15.594+0000 7f36d9870700 1 -- 172.21.15.87:0/3232007414 wait complete. 2024-06-06T07:46:16.516 INFO:teuthology.orchestra.run.smithi087.stdout:55834574940 2024-06-06T07:46:16.516 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-06T07:46:16.978 INFO:teuthology.orchestra.run.smithi087.stdout:77309411416 2024-06-06T07:46:16.979 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-06T07:46:17.440 INFO:teuthology.orchestra.run.smithi087.stdout:287762808884 2024-06-06T07:46:17.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-06T07:46:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:17 smithi195 ceph-mon[37040]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T07:46:17.521 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.528 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.546 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.546 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.546 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.552 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:17 smithi181 ceph-mon[36790]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T07:46:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:17 smithi087 ceph-mon[32276]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T07:46:19.165 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:19.167 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:19.273 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:19.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:19 smithi087 ceph-mon[32276]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:19.391 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:19 smithi195 ceph-mon[37040]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:19.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:19 smithi181 ceph-mon[36790]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:20.853 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:21 smithi195 ceph-mon[37040]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:21.513 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:21 smithi087 ceph-mon[32276]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:21 smithi181 ceph-mon[36790]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:23.311 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:23 smithi087 ceph-mon[32276]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:23 smithi195 ceph-mon[37040]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:23 smithi181 ceph-mon[36790]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.431+0000 7f628837d700 1 -- 172.21.15.87:0/1333083441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f628006abc0 msgr2=0x7f628006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.431+0000 7f628837d700 1 --2- 172.21.15.87:0/1333083441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f628006abc0 0x7f628006b040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62700099c0 tx=0x7f6270009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 -- 172.21.15.87:0/1333083441 shutdown_connections 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 --2- 172.21.15.87:0/1333083441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 0x7f628010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 --2- 172.21.15.87:0/1333083441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62801085a0 0x7f6280108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 --2- 172.21.15.87:0/1333083441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f628006abc0 0x7f628006b040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 -- 172.21.15.87:0/1333083441 >> 172.21.15.87:0/1333083441 conn(0x7f6280077200 msgr2=0x7f6280077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 -- 172.21.15.87:0/1333083441 shutdown_connections 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 -- 172.21.15.87:0/1333083441 wait complete. 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 Processor -- start 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.432+0000 7f628837d700 1 -- start start 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f628006abc0 0x7f6280198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 0x7f62801992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62801128d0 con 0x7f62801085a0 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112750 con 0x7f628006b580 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112a50 con 0x7f628006abc0 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:24.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51860/0 (socket says 172.21.15.87:51860) 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 -- 172.21.15.87:0/2992766172 learned_addr learned my addr 172.21.15.87:0/2992766172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f6286119700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f628006abc0 0x7f6280198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f6285918700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 0x7f62801992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 -- 172.21.15.87:0/2992766172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f628006abc0 msgr2=0x7f6280198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f628006abc0 0x7f6280198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 -- 172.21.15.87:0/2992766172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 msgr2=0x7f62801992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 0x7f62801992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 -- 172.21.15.87:0/2992766172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019dd40 con 0x7f62801085a0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628691a700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f627800b970 tx=0x7f627800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627800d860 con 0x7f62801085a0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019e030 con 0x7f62801085a0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.433+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627800e430 con 0x7f62801085a0 2024-06-06T07:46:24.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.434+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801016e0 con 0x7f62801085a0 2024-06-06T07:46:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.434+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278017660 con 0x7f62801085a0 2024-06-06T07:46:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.435+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6280103bf0 con 0x7f62801085a0 2024-06-06T07:46:24.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.436+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f627800e5e0 con 0x7f62801085a0 2024-06-06T07:46:24.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.437+0000 7f62777fe700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 0x7f626c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:24.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.437+0000 7f6286119700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 0x7f626c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:24.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.437+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f62780cf0e0 con 0x7f62801085a0 2024-06-06T07:46:24.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.437+0000 7f6286119700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 0x7f626c0a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f627c009a10 tx=0x7f627c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:24.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.441+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6278013030 con 0x7f62801085a0 2024-06-06T07:46:24.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.644+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f628002dca0 con 0x7f62801085a0 2024-06-06T07:46:24.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.644+0000 7f62777fe700 1 -- 172.21.15.87:0/2992766172 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f627809baf0 con 0x7f62801085a0 2024-06-06T07:46:24.644 INFO:teuthology.orchestra.run.smithi087.stdout:197568495686 2024-06-06T07:46:24.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 msgr2=0x7f626c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:24.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 0x7f626c0a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f627c009a10 tx=0x7f627c009190 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 msgr2=0x7f628019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f627800b970 tx=0x7f627800bc80 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 shutdown_connections 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f626c0a2d30 0x7f626c0a51f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62801085a0 0x7f628019d600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f628006abc0 0x7f6280198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 --2- 172.21.15.87:0/2992766172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628006b580 0x7f62801992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 >> 172.21.15.87:0/2992766172 conn(0x7f6280077200 msgr2=0x7f62801003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 shutdown_connections 2024-06-06T07:46:24.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:24.646+0000 7f628837d700 1 -- 172.21.15.87:0/2992766172 wait complete. 2024-06-06T07:46:25.484 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:25 smithi087 ceph-mon[32276]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:25.484 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:25 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2992766172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T07:46:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:25 smithi195 ceph-mon[37040]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:25.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:25 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2992766172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T07:46:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:25 smithi181 ceph-mon[36790]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:25 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2992766172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T07:46:26.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495682 got 197568495686 for osd.6 2024-06-06T07:46:26.117 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:27.458 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:27 smithi087 ceph-mon[32276]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:27.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:27 smithi195 ceph-mon[37040]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:27 smithi181 ceph-mon[36790]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.953+0000 7f38f0f69700 1 -- 172.21.15.87:0/4120602771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec102030 msgr2=0x7f38ec105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.953+0000 7f38f0f69700 1 --2- 172.21.15.87:0/4120602771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec102030 0x7f38ec105ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38e00099c0 tx=0x7f38e0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 -- 172.21.15.87:0/4120602771 shutdown_connections 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 --2- 172.21.15.87:0/4120602771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec101680 0x7f38ec101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 --2- 172.21.15.87:0/4120602771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec1066d0 0x7f38ec10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 --2- 172.21.15.87:0/4120602771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec102030 0x7f38ec105ff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 -- 172.21.15.87:0/4120602771 >> 172.21.15.87:0/4120602771 conn(0x7f38ec0773e0 msgr2=0x7f38ec0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 -- 172.21.15.87:0/4120602771 shutdown_connections 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 -- 172.21.15.87:0/4120602771 wait complete. 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.954+0000 7f38f0f69700 1 Processor -- start 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 -- start start 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec102030 0x7f38ec199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:27.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec1066d0 0x7f38ec1a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111690 con 0x7f38ec101680 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111510 con 0x7f38ec1066d0 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38f0f69700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111810 con 0x7f38ec102030 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46102/0 (socket says 172.21.15.87:46102) 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 -- 172.21.15.87:0/707631432 learned_addr learned my addr 172.21.15.87:0/707631432 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38e9d9b700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec102030 0x7f38ec199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ead9d700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec1066d0 0x7f38ec1a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 -- 172.21.15.87:0/707631432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec102030 msgr2=0x7f38ec199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec102030 0x7f38ec199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 -- 172.21.15.87:0/707631432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec1066d0 msgr2=0x7f38ec1a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec1066d0 0x7f38ec1a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:27.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 -- 172.21.15.87:0/707631432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec1a5010 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38ea59c700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c9b0 tx=0x7f38dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.955+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc007b00 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.956+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc007ca0 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.956+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec1a51e0 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.956+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc018730 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.956+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a56f0 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.956+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec069180 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.958+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f38dc0188d0 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.958+0000 7f38db7fe700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 0x7f38d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.958+0000 7f38e9d9b700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 0x7f38d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.959+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38dc0cf820 con 0x7f38ec101680 2024-06-06T07:46:27.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.959+0000 7f38e9d9b700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 0x7f38d40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:27.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:27.962+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38dc014030 con 0x7f38ec101680 2024-06-06T07:46:28.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.171+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f38ec02dca0 con 0x7f38ec101680 2024-06-06T07:46:28.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.171+0000 7f38db7fe700 1 -- 172.21.15.87:0/707631432 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f38dc09c2c0 con 0x7f38ec101680 2024-06-06T07:46:28.171 INFO:teuthology.orchestra.run.smithi087.stdout:163208757323 2024-06-06T07:46:28.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 msgr2=0x7f38d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 0x7f38d40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 msgr2=0x7f38ec199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c9b0 tx=0x7f38dc00ccc0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 shutdown_connections 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38d40a2cb0 0x7f38d40a5170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec1066d0 0x7f38ec1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ec101680 0x7f38ec199a50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 --2- 172.21.15.87:0/707631432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38ec102030 0x7f38ec199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.173+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 >> 172.21.15.87:0/707631432 conn(0x7f38ec0773e0 msgr2=0x7f38ec0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.174+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 shutdown_connections 2024-06-06T07:46:28.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.174+0000 7f38f0f69700 1 -- 172.21.15.87:0/707631432 wait complete. 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.241+0000 7eff6563f700 1 -- 172.21.15.87:0/3651954757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff60108650 msgr2=0x7eff60108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.241+0000 7eff6563f700 1 --2- 172.21.15.87:0/3651954757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff60108650 0x7eff60108a30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7eff50009990 tx=0x7eff50009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 -- 172.21.15.87:0/3651954757 shutdown_connections 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 --2- 172.21.15.87:0/3651954757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60103170 0x7eff6010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 --2- 172.21.15.87:0/3651954757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff60108650 0x7eff60108a30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 --2- 172.21.15.87:0/3651954757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff601027b0 0x7eff60102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 -- 172.21.15.87:0/3651954757 >> 172.21.15.87:0/3651954757 conn(0x7eff6007a420 msgr2=0x7eff6007a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 -- 172.21.15.87:0/3651954757 shutdown_connections 2024-06-06T07:46:28.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 -- 172.21.15.87:0/3651954757 wait complete. 2024-06-06T07:46:28.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.242+0000 7eff6563f700 1 Processor -- start 2024-06-06T07:46:28.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 -- start start 2024-06-06T07:46:28.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff60103170 0x7eff60074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60108650 0x7eff60075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7eff601128b0 con 0x7eff601027b0 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff60112730 con 0x7eff60103170 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff6563f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff60112a30 con 0x7eff60108650 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46106/0 (socket says 172.21.15.87:46106) 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 -- 172.21.15.87:0/485749564 learned_addr learned my addr 172.21.15.87:0/485749564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5f7fe700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60108650 0x7eff60075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5e7fc700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff60103170 0x7eff60074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 -- 172.21.15.87:0/485749564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60108650 msgr2=0x7eff60075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60108650 0x7eff60075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 -- 172.21.15.87:0/485749564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff60103170 msgr2=0x7eff60074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff60103170 0x7eff60074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.243+0000 7eff5effd700 1 -- 172.21.15.87:0/485749564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a70e0 con 0x7eff601027b0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff5effd700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50027070 con 0x7eff601027b0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50004050 con 0x7eff601027b0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a7370 con 0x7eff601027b0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000f460 con 0x7eff601027b0 2024-06-06T07:46:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.244+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a77c0 con 0x7eff601027b0 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.245+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60069180 con 0x7eff601027b0 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.246+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff5002a070 con 0x7eff601027b0 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.246+0000 7eff47fff700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 0x7eff480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.247+0000 7eff5e7fc700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 0x7eff480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.247+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7eff500d0580 con 0x7eff601027b0 2024-06-06T07:46:28.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.247+0000 7eff5e7fc700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 0x7eff480a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff54005ea0 tx=0x7eff54005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:28.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.251+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff5005e850 con 0x7eff601027b0 2024-06-06T07:46:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:28 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/707631432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T07:46:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.456+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7eff601a7b80 con 0x7eff601027b0 2024-06-06T07:46:28.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.457+0000 7eff47fff700 1 -- 172.21.15.87:0/485749564 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff50013070 con 0x7eff601027b0 2024-06-06T07:46:28.457 INFO:teuthology.orchestra.run.smithi087.stdout:141733920848 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 msgr2=0x7eff480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 0x7eff480a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff54005ea0 tx=0x7eff54005e30 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 msgr2=0x7eff60074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500038b0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 shutdown_connections 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7eff480a2cb0 0x7eff480a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60108650 0x7eff60075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff601027b0 0x7eff60074790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 --2- 172.21.15.87:0/485749564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff60103170 0x7eff60074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.459+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 >> 172.21.15.87:0/485749564 conn(0x7eff6007a420 msgr2=0x7eff60101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.460+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 shutdown_connections 2024-06-06T07:46:28.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:28.460+0000 7eff6563f700 1 -- 172.21.15.87:0/485749564 wait complete. 2024-06-06T07:46:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:28 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/707631432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T07:46:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:28 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/707631432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T07:46:29.461 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:29 smithi087 ceph-mon[32276]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:29.461 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/485749564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T07:46:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:29 smithi181 ceph-mon[36790]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/485749564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T07:46:29.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:29 smithi195 ceph-mon[37040]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:29.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/485749564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T07:46:29.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.983+0000 7f37a2ef8700 1 -- 172.21.15.87:0/50346337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c1074a0 msgr2=0x7f379c107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:29.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.983+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/50346337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c1074a0 0x7f379c107880 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3790009990 tx=0x7f3790009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.983+0000 7f37a2ef8700 1 -- 172.21.15.87:0/50346337 shutdown_connections 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.983+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/50346337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c102050 0x7f379c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.983+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/50346337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c1074a0 0x7f379c107880 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/50346337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101690 0x7f379c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- 172.21.15.87:0/50346337 >> 172.21.15.87:0/50346337 conn(0x7f379c078190 msgr2=0x7f379c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- 172.21.15.87:0/50346337 shutdown_connections 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- 172.21.15.87:0/50346337 wait complete. 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 Processor -- start 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- start start 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c102050 0x7f379c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 0x7f379c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:29.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f379c111800 con 0x7f379c101690 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f379c111680 con 0x7f379c102050 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.984+0000 7f37a2ef8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f379c111980 con 0x7f379c1074a0 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46122/0 (socket says 172.21.15.87:46122) 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 -- 172.21.15.87:0/1361872940 learned_addr learned my addr 172.21.15.87:0/1361872940 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f379bfff700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c102050 0x7f379c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a1495700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 0x7f379c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 -- 172.21.15.87:0/1361872940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 msgr2=0x7f379c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 0x7f379c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 -- 172.21.15.87:0/1361872940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c102050 msgr2=0x7f379c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c102050 0x7f379c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 -- 172.21.15.87:0/1361872940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a5180 con 0x7f379c101690 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a1495700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 0x7f379c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a0c94700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3790000c00 tx=0x7f3790003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:29.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790027070 con 0x7f379c101690 2024-06-06T07:46:29.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3790004110 con 0x7f379c101690 2024-06-06T07:46:29.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a5350 con 0x7f379c101690 2024-06-06T07:46:29.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379000f460 con 0x7f379c101690 2024-06-06T07:46:29.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.985+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a57a0 con 0x7f379c101690 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.986+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c069180 con 0x7f379c101690 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.988+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f379002a070 con 0x7f379c101690 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.988+0000 7f3799ffb700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 0x7f378c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.988+0000 7f379bfff700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 0x7f378c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.988+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f37900d0940 con 0x7f379c101690 2024-06-06T07:46:29.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.989+0000 7f379bfff700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 0x7f378c0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f379c078800 tx=0x7f3788009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:29.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:29.992+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3790031070 con 0x7f379c101690 2024-06-06T07:46:30.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.195+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f379c02dca0 con 0x7f379c101690 2024-06-06T07:46:30.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.195+0000 7f3799ffb700 1 -- 172.21.15.87:0/1361872940 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3790013070 con 0x7f379c101690 2024-06-06T07:46:30.196 INFO:teuthology.orchestra.run.smithi087.stdout:77309411419 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 msgr2=0x7f378c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 0x7f378c0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f379c078800 tx=0x7f3788009190 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 msgr2=0x7f379c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3790000c00 tx=0x7f3790003680 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 shutdown_connections 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f378c0a2cb0 0x7f378c0a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f379c1074a0 0x7f379c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f379c101690 0x7f379c199bc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 --2- 172.21.15.87:0/1361872940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c102050 0x7f379c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 >> 172.21.15.87:0/1361872940 conn(0x7f379c078190 msgr2=0x7f379c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:30.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.197+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 shutdown_connections 2024-06-06T07:46:30.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.198+0000 7f37a2ef8700 1 -- 172.21.15.87:0/1361872940 wait complete. 2024-06-06T07:46:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:30 smithi181 ceph-mon[36790]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:30 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1361872940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T07:46:30.599 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:30 smithi087 ceph-mon[32276]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:30.599 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:30 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1361872940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T07:46:30.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:30 smithi195 ceph-mon[37040]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:30.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:30 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1361872940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T07:46:30.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 -- 172.21.15.87:0/3622022059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 msgr2=0x7f8bcc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 --2- 172.21.15.87:0/3622022059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1077e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b0a0 tx=0x7f8bbc00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 -- 172.21.15.87:0/3622022059 shutdown_connections 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 --2- 172.21.15.87:0/3622022059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 0x7f8bcc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 --2- 172.21.15.87:0/3622022059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1077e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 --2- 172.21.15.87:0/3622022059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 0x7f8bcc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 -- 172.21.15.87:0/3622022059 >> 172.21.15.87:0/3622022059 conn(0x7f8bcc078340 msgr2=0x7f8bcc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:30.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 -- 172.21.15.87:0/3622022059 shutdown_connections 2024-06-06T07:46:30.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.997+0000 7f8bd3013700 1 -- 172.21.15.87:0/3622022059 wait complete. 2024-06-06T07:46:31.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 Processor -- start 2024-06-06T07:46:31.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 -- start start 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 0x7f8bcc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 0x7f8bcc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1116a0 con 0x7f8bcc107400 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc111520 con 0x7f8bcc101390 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd3013700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc111820 con 0x7f8bcc101d50 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd15b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd15b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46148/0 (socket says 172.21.15.87:46148) 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd15b0700 1 -- 172.21.15.87:0/2812641241 learned_addr learned my addr 172.21.15.87:0/2812641241 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:31.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.998+0000 7f8bd0daf700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 0x7f8bcc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bcbfff700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 0x7f8bcc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 -- 172.21.15.87:0/2812641241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 msgr2=0x7f8bcc199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 0x7f8bcc199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 -- 172.21.15.87:0/2812641241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 msgr2=0x7f8bcc199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 0x7f8bcc199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 -- 172.21.15.87:0/2812641241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc1a5020 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd15b0700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400d980 tx=0x7f8bc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4017070 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc4005020 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400fbf0 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1a5250 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:30.999+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc1a5790 con 0x7f8bcc107400 2024-06-06T07:46:31.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.000+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc069180 con 0x7f8bcc107400 2024-06-06T07:46:31.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.001+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8bc40051d0 con 0x7f8bcc107400 2024-06-06T07:46:31.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.002+0000 7f8bc9ffb700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 0x7f8bb80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:31.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.002+0000 7f8bd0daf700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 0x7f8bb80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:31.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.002+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8bc40cf2b0 con 0x7f8bcc107400 2024-06-06T07:46:31.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.003+0000 7f8bd0daf700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 0x7f8bb80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e60 tx=0x7f8bbc00b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:31.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.006+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bc4020020 con 0x7f8bcc107400 2024-06-06T07:46:31.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.211+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8bcc02de60 con 0x7f8bcc107400 2024-06-06T07:46:31.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.211+0000 7f8bc9ffb700 1 -- 172.21.15.87:0/2812641241 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8bc409bcc0 con 0x7f8bcc107400 2024-06-06T07:46:31.212 INFO:teuthology.orchestra.run.smithi087.stdout:55834574943 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 msgr2=0x7f8bb80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 0x7f8bb80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e60 tx=0x7f8bbc00b7b0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 msgr2=0x7f8bcc1a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400d980 tx=0x7f8bc400dc90 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 shutdown_connections 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8bb80a2cb0 0x7f8bb80a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc101390 0x7f8bcc199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bcc107400 0x7f8bcc1a4ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 --2- 172.21.15.87:0/2812641241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc101d50 0x7f8bcc199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 >> 172.21.15.87:0/2812641241 conn(0x7f8bcc078340 msgr2=0x7f8bcc0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 shutdown_connections 2024-06-06T07:46:31.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:31.214+0000 7f8bd3013700 1 -- 172.21.15.87:0/2812641241 wait complete. 2024-06-06T07:46:31.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757319 got 163208757323 for osd.5 2024-06-06T07:46:31.427 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:31.510 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:31 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2812641241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T07:46:31.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:31 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2812641241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T07:46:31.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:31 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2812641241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T07:46:32.527 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:32 smithi087 ceph-mon[32276]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:32 smithi181 ceph-mon[36790]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:32 smithi195 ceph-mon[37040]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:32.867 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:33.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920844 got 141733920848 for osd.4 2024-06-06T07:46:33.762 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/791856724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28103150 msgr2=0x7f1d28107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/791856724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28103150 0x7f1d28107140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d100099c0 tx=0x7f1d10009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/791856724 shutdown_connections 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/791856724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d281027a0 0x7f1d28102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/791856724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d28107870 0x7f1d2810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/791856724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28103150 0x7f1d28107140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/791856724 >> 172.21.15.87:0/791856724 conn(0x7f1d28079520 msgr2=0x7f1d28079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/791856724 shutdown_connections 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.276+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/791856724 wait complete. 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 Processor -- start 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 -- start start 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 0x7f1d2819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d28103150 0x7f1d2819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28112780 con 0x7f1d28107870 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28112600 con 0x7f1d28103150 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2ca0a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28112900 con 0x7f1d281027a0 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d26d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:34.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d26d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46174/0 (socket says 172.21.15.87:46174) 2024-06-06T07:46:34.904 INFO:teuthology.orchestra.run.smithi087.stdout:219043332162 2024-06-06T07:46:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d26d9d700 1 -- 172.21.15.87:0/2678887745 learned_addr learned my addr 172.21.15.87:0/2678887745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d2659c700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 0x7f1d2819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.277+0000 7f1d1dd9b700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d28103150 0x7f1d2819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 -- 172.21.15.87:0/2678887745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 msgr2=0x7f1d2819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 0x7f1d2819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 -- 172.21.15.87:0/2678887745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d28103150 msgr2=0x7f1d2819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d28103150 0x7f1d2819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 -- 172.21.15.87:0/2678887745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2819fd40 con 0x7f1d28107870 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d2659c700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 0x7f1d2819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d26d9d700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d20007d70 tx=0x7f1d2000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20016020 con 0x7f1d28107870 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d281a0030 con 0x7f1d28107870 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2000ac90 con 0x7f1d28107870 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200115c0 con 0x7f1d28107870 2024-06-06T07:46:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.278+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d281a78b0 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.279+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d28069180 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.280+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d20019020 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.281+0000 7f1d1ffff700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.281+0000 7f1d2659c700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.281+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f1d200cf820 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.281+0000 7f1d2659c700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 0x7f1d140a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005950 tx=0x7f1d1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.284+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d20020020 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.486+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1d2819c090 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.486+0000 7f1d1ffff700 1 -- 172.21.15.87:0/2678887745 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d2819c090 con 0x7f1d28107870 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.488+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 msgr2=0x7f1d140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.488+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 0x7f1d140a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005950 tx=0x7f1d1800b410 comp rx=0 tx=0).stop 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.488+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 msgr2=0x7f1d2819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.488+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d20007d70 tx=0x7f1d2000a340 comp rx=0 tx=0).stop 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 shutdown_connections 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d28107870 0x7f1d2819f600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d281027a0 0x7f1d2819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 --2- 172.21.15.87:0/2678887745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d28103150 0x7f1d2819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 >> 172.21.15.87:0/2678887745 conn(0x7f1d28079520 msgr2=0x7f1d28100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 shutdown_connections 2024-06-06T07:46:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:34.489+0000 7f1d2ca0a700 1 -- 172.21.15.87:0/2678887745 wait complete. 2024-06-06T07:46:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:35 smithi195 ceph-mon[37040]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:35 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2678887745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T07:46:35.541 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:35 smithi087 ceph-mon[32276]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:35.542 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:35 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2678887745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T07:46:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:35 smithi181 ceph-mon[36790]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:35 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2678887745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T07:46:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.644+0000 7f62b78cc700 1 -- 172.21.15.87:0/2943612023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 msgr2=0x7f62b01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.644+0000 7f62b78cc700 1 --2- 172.21.15.87:0/2943612023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b01089d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009990 tx=0x7f62ac009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.644+0000 7f62b78cc700 1 -- 172.21.15.87:0/2943612023 shutdown_connections 2024-06-06T07:46:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.644+0000 7f62b78cc700 1 --2- 172.21.15.87:0/2943612023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b0102ea0 0x7f62b010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 --2- 172.21.15.87:0/2943612023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b01089d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 --2- 172.21.15.87:0/2943612023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b01024e0 0x7f62b0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- 172.21.15.87:0/2943612023 >> 172.21.15.87:0/2943612023 conn(0x7f62b0079640 msgr2=0x7f62b0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- 172.21.15.87:0/2943612023 shutdown_connections 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- 172.21.15.87:0/2943612023 wait complete. 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 Processor -- start 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- start start 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b01024e0 0x7f62b019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b0102ea0 0x7f62b019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:35.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01127b0 con 0x7f62b01085f0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0112630 con 0x7f62b01024e0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.645+0000 7f62b78cc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0112930 con 0x7f62b0102ea0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5668700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b01024e0 0x7f62b019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46180/0 (socket says 172.21.15.87:46180) 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 -- 172.21.15.87:0/1170116420 learned_addr learned my addr 172.21.15.87:0/1170116420 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b4e67700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b0102ea0 0x7f62b019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 -- 172.21.15.87:0/1170116420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b0102ea0 msgr2=0x7f62b019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b0102ea0 0x7f62b019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 -- 172.21.15.87:0/1170116420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b01024e0 msgr2=0x7f62b019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b01024e0 0x7f62b019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 -- 172.21.15.87:0/1170116420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b0199250 con 0x7f62b01085f0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b5e69700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007f10 tx=0x7f62a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:35.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8009e20 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004bb0 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b0199540 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8005960 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.646+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a5800 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.647+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0069180 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.648+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f62a801a020 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.649+0000 7f62a67fc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 0x7f629c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.650+0000 7f62b5668700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 0x7f629c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.650+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f62a80cf2a0 con 0x7f62b01085f0 2024-06-06T07:46:35.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.650+0000 7f62b5668700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 0x7f629c0a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b5c0 tx=0x7f62ac005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:35.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.653+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f62a8010070 con 0x7f62b01085f0 2024-06-06T07:46:35.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.860+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f62b002dca0 con 0x7f62b01085f0 2024-06-06T07:46:35.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.860+0000 7f62a67fc700 1 -- 172.21.15.87:0/1170116420 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f62a809bc50 con 0x7f62b01085f0 2024-06-06T07:46:35.861 INFO:teuthology.orchestra.run.smithi087.stdout:107374182486 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.862+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 msgr2=0x7f629c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.862+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 0x7f629c0a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b5c0 tx=0x7f62ac005960 comp rx=0 tx=0).stop 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.862+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 msgr2=0x7f62b0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.862+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007f10 tx=0x7f62a800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 shutdown_connections 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f629c0a2cb0 0x7f629c0a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b01024e0 0x7f62b019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b01085f0 0x7f62b0198af0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 --2- 172.21.15.87:0/1170116420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b0102ea0 0x7f62b019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:35.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 >> 172.21.15.87:0/1170116420 conn(0x7f62b0079640 msgr2=0x7f62b0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:35.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 shutdown_connections 2024-06-06T07:46:35.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:35.863+0000 7f62b78cc700 1 -- 172.21.15.87:0/1170116420 wait complete. 2024-06-06T07:46:36.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 -- 172.21.15.87:0/3741401661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 msgr2=0x7fa68010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:36.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 --2- 172.21.15.87:0/3741401661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68010e240 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa67800b0a0 tx=0x7fa67800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 -- 172.21.15.87:0/3741401661 shutdown_connections 2024-06-06T07:46:36.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 --2- 172.21.15.87:0/3741401661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680107400 0x7fa6801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 --2- 172.21.15.87:0/3741401661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68010e240 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 --2- 172.21.15.87:0/3741401661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680101390 0x7fa680101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 -- 172.21.15.87:0/3741401661 >> 172.21.15.87:0/3741401661 conn(0x7fa680078340 msgr2=0x7fa680078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 -- 172.21.15.87:0/3741401661 shutdown_connections 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.308+0000 7fa68637d700 1 -- 172.21.15.87:0/3741401661 wait complete. 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 Processor -- start 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 -- start start 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680107400 0x7fa680198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801116f0 con 0x7fa680101d50 2024-06-06T07:46:36.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa680111570 con 0x7fa680101390 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68637d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa680111870 con 0x7fa680107400 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa67ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa67ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:42870/0 (socket says 172.21.15.87:42870) 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa677fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa67ffff700 1 -- 172.21.15.87:0/2967757024 learned_addr learned my addr 172.21.15.87:0/2967757024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.309+0000 7fa68491a700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680107400 0x7fa680198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 -- 172.21.15.87:0/2967757024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680107400 msgr2=0x7fa680198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680107400 0x7fa680198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 -- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 msgr2=0x7fa68019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 -- 172.21.15.87:0/2967757024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6801991c0 con 0x7fa680101390 2024-06-06T07:46:36.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa677fff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67ffff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d950 tx=0x7fa67000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670016070 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801994b0 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa670009e20 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670020060 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.310+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a5770 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.311+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680069180 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.312+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa670019070 con 0x7fa680101390 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.313+0000 7fa67dffb700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.313+0000 7fa677fff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:36.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.313+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa6700cf540 con 0x7fa680101390 2024-06-06T07:46:36.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.313+0000 7fa677fff700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6800788d0 tx=0x7fa668009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:36.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.317+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa67009bf50 con 0x7fa680101390 2024-06-06T07:46:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:36 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1170116420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T07:46:36.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.523+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa6801a59a0 con 0x7fa680101390 2024-06-06T07:46:36.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:36 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1170116420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T07:46:36.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.523+0000 7fa67dffb700 1 -- 172.21.15.87:0/2967757024 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa67009b850 con 0x7fa680101390 2024-06-06T07:46:36.523 INFO:teuthology.orchestra.run.smithi087.stdout:309237645364 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 msgr2=0x7fa66c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6800788d0 tx=0x7fa668009190 comp rx=0 tx=0).stop 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 msgr2=0x7fa68019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d950 tx=0x7fa67000dc60 comp rx=0 tx=0).stop 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 shutdown_connections 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa66c0a2cb0 0x7fa66c0a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa680101390 0x7fa68019e8a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680107400 0x7fa680198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 --2- 172.21.15.87:0/2967757024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa680101d50 0x7fa68019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:36.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 >> 172.21.15.87:0/2967757024 conn(0x7fa680078340 msgr2=0x7fa6801005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:36.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 shutdown_connections 2024-06-06T07:46:36.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:36.526+0000 7fa68637d700 1 -- 172.21.15.87:0/2967757024 wait complete. 2024-06-06T07:46:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:36 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1170116420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T07:46:37.207 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182486 for osd.3 2024-06-06T07:46:37.207 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:37.476 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:37 smithi087 ceph-mon[32276]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:37.476 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:37 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2967757024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T07:46:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:37 smithi195 ceph-mon[37040]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:37 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2967757024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T07:46:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:37 smithi181 ceph-mon[36790]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:37 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2967757024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T07:46:38.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 -- 172.21.15.87:0/2721090260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1141086f0 msgr2=0x7fc114108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 --2- 172.21.15.87:0/2721090260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1141086f0 0x7fc114108ad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc104009990 tx=0x7fc104009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 -- 172.21.15.87:0/2721090260 shutdown_connections 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 --2- 172.21.15.87:0/2721090260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114101440 0x7fc11410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 --2- 172.21.15.87:0/2721090260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1141086f0 0x7fc114108ad0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 --2- 172.21.15.87:0/2721090260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc114100a80 0x7fc114100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 -- 172.21.15.87:0/2721090260 >> 172.21.15.87:0/2721090260 conn(0x7fc114077a00 msgr2=0x7fc1140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 -- 172.21.15.87:0/2721090260 shutdown_connections 2024-06-06T07:46:38.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.216+0000 7fc11be26700 1 -- 172.21.15.87:0/2721090260 wait complete. 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 Processor -- start 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 -- start start 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114100a80 0x7fc114198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1141086f0 0x7fc11419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141129f0 con 0x7fc114101440 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc114112870 con 0x7fc1141086f0 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc11be26700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc114112b70 con 0x7fc114100a80 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc1193c1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc1193c1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40434/0 (socket says 172.21.15.87:40434) 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.217+0000 7fc1193c1700 1 -- 172.21.15.87:0/2947975030 learned_addr learned my addr 172.21.15.87:0/2947975030 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:38.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc119bc2700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114100a80 0x7fc114198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc11a3c3700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1141086f0 0x7fc11419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 -- 172.21.15.87:0/2947975030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114100a80 msgr2=0x7fc114198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114100a80 0x7fc114198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 -- 172.21.15.87:0/2947975030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1141086f0 msgr2=0x7fc11419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1141086f0 0x7fc11419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 -- 172.21.15.87:0/2947975030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419de20 con 0x7fc114101440 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc1193c1700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b530 tx=0x7fc11000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110015020 con 0x7fc114101440 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc110011d10 con 0x7fc114101440 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419e110 con 0x7fc114101440 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11000f2e0 con 0x7fc114101440 2024-06-06T07:46:38.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.218+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a5a10 con 0x7fc114101440 2024-06-06T07:46:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.220+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc114069180 con 0x7fc114101440 2024-06-06T07:46:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.221+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc110018020 con 0x7fc114101440 2024-06-06T07:46:38.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.221+0000 7fc10affd700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 0x7fc1000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.222+0000 7fc119bc2700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 0x7fc1000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.222+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc1100cecd0 con 0x7fc114101440 2024-06-06T07:46:38.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.222+0000 7fc119bc2700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 0x7fc1000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc104006010 tx=0x7fc104005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:38.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.226+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc11001f020 con 0x7fc114101440 2024-06-06T07:46:38.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.433+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc11419a170 con 0x7fc114101440 2024-06-06T07:46:38.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.434+0000 7fc10affd700 1 -- 172.21.15.87:0/2947975030 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc11009b7f0 con 0x7fc114101440 2024-06-06T07:46:38.434 INFO:teuthology.orchestra.run.smithi087.stdout:34359738471 2024-06-06T07:46:38.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 msgr2=0x7fc1000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:38.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 0x7fc1000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc104006010 tx=0x7fc104005960 comp rx=0 tx=0).stop 2024-06-06T07:46:38.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 msgr2=0x7fc1141993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b530 tx=0x7fc11000b840 comp rx=0 tx=0).stop 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 shutdown_connections 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc1000a2d30 0x7fc1000a51f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1141086f0 0x7fc11419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc114100a80 0x7fc114198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 --2- 172.21.15.87:0/2947975030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc114101440 0x7fc1141993a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 >> 172.21.15.87:0/2947975030 conn(0x7fc114077a00 msgr2=0x7fc1140ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 shutdown_connections 2024-06-06T07:46:38.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:38.436+0000 7fc11be26700 1 -- 172.21.15.87:0/2947975030 wait complete. 2024-06-06T07:46:38.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332157 got 219043332162 for osd.7 2024-06-06T07:46:38.807 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:39 smithi195 ceph-mon[37040]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:39.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:39 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2947975030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T07:46:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:39 smithi181 ceph-mon[36790]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:39 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2947975030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T07:46:39.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:39 smithi087 ceph-mon[32276]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:46:39.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:39 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2947975030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T07:46:39.589 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738463 got 34359738471 for osd.0 2024-06-06T07:46:39.589 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:40.010 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574943 for osd.1 2024-06-06T07:46:40.011 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:40.536 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645359 got 309237645364 for osd.10 2024-06-06T07:46:40.536 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:41.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- 172.21.15.87:0/3595323621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a4104ec0 msgr2=0x7fc5a4105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:41.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 --2- 172.21.15.87:0/3595323621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a4104ec0 0x7fc5a4105370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b0a0 tx=0x7fc59c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- 172.21.15.87:0/3595323621 shutdown_connections 2024-06-06T07:46:41.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 --2- 172.21.15.87:0/3595323621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a41073b0 0x7fc5a406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 --2- 172.21.15.87:0/3595323621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a4104ec0 0x7fc5a4105370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 --2- 172.21.15.87:0/3595323621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- 172.21.15.87:0/3595323621 >> 172.21.15.87:0/3595323621 conn(0x7fc5a4078360 msgr2=0x7fc5a4078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- 172.21.15.87:0/3595323621 shutdown_connections 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- 172.21.15.87:0/3595323621 wait complete. 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 Processor -- start 2024-06-06T07:46:41.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.169+0000 7fc5aa675700 1 -- start start 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 0x7fc5a419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4118e60 con 0x7fc5a41073b0 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4118ce0 con 0x7fc5a4104ec0 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5aa675700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4118fe0 con 0x7fc5a406b120 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40462/0 (socket says 172.21.15.87:40462) 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 -- 172.21.15.87:0/2139486328 learned_addr learned my addr 172.21.15.87:0/2139486328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:41.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a3fff700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a37fe700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 0x7fc5a419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 -- 172.21.15.87:0/2139486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 msgr2=0x7fc5a419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 -- 172.21.15.87:0/2139486328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 msgr2=0x7fc5a419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 0x7fc5a419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 -- 172.21.15.87:0/2139486328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41a4300 con 0x7fc5a41073b0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a3fff700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a37fe700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 0x7fc5a419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a8c12700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0062a0 tx=0x7fc59c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.170+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c021070 con 0x7fc5a41073b0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.171+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a4590 con 0x7fc5a41073b0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.171+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c007db0 con 0x7fc5a41073b0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.171+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41abe70 con 0x7fc5a41073b0 2024-06-06T07:46:41.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.171+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005220 con 0x7fc5a41073b0 2024-06-06T07:46:41.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.172+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a4069180 con 0x7fc5a41073b0 2024-06-06T07:46:41.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.173+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc59c024070 con 0x7fc5a41073b0 2024-06-06T07:46:41.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.173+0000 7fc5a17fa700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 0x7fc58c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:41.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.173+0000 7fc5a3fff700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 0x7fc58c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:41.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.174+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fc59c0d0880 con 0x7fc5a41073b0 2024-06-06T07:46:41.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.174+0000 7fc5a3fff700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 0x7fc58c0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5940099d0 tx=0x7fc594008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:41.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.178+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc59c02b030 con 0x7fc5a41073b0 2024-06-06T07:46:41.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:41 smithi087 ceph-mon[32276]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.382+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc5a41ac040 con 0x7fc5a41073b0 2024-06-06T07:46:41.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.382+0000 7fc5a17fa700 1 -- 172.21.15.87:0/2139486328 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc59c087020 con 0x7fc5a41073b0 2024-06-06T07:46:41.382 INFO:teuthology.orchestra.run.smithi087.stdout:287762808889 2024-06-06T07:46:41.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 msgr2=0x7fc58c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 0x7fc58c0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5940099d0 tx=0x7fc594008040 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 msgr2=0x7fc5a41a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0062a0 tx=0x7fc59c00bf10 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 shutdown_connections 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fc58c0a2cb0 0x7fc58c0a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5a41073b0 0x7fc5a41a3bc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a406b120 0x7fc5a419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 --2- 172.21.15.87:0/2139486328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a4104ec0 0x7fc5a419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 >> 172.21.15.87:0/2139486328 conn(0x7fc5a4078360 msgr2=0x7fc5a4103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 shutdown_connections 2024-06-06T07:46:41.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:41.384+0000 7fc5aa675700 1 -- 172.21.15.87:0/2139486328 wait complete. 2024-06-06T07:46:41.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411419 for osd.2 2024-06-06T07:46:41.508 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:41.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:41 smithi195 ceph-mon[37040]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:41 smithi181 ceph-mon[36790]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:42.186 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808884 got 287762808889 for osd.9 2024-06-06T07:46:42.186 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:42 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2139486328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T07:46:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:42 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2139486328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T07:46:42.598 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:42 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2139486328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T07:46:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 -- 172.21.15.87:0/4143381169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c1084f0 msgr2=0x7f128c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 --2- 172.21.15.87:0/4143381169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c1084f0 0x7f128c1088d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1288009990 tx=0x7f1288009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 -- 172.21.15.87:0/4143381169 shutdown_connections 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 --2- 172.21.15.87:0/4143381169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102f10 0x7f128c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 --2- 172.21.15.87:0/4143381169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c1084f0 0x7f128c1088d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 --2- 172.21.15.87:0/4143381169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c102550 0x7f128c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 -- 172.21.15.87:0/4143381169 >> 172.21.15.87:0/4143381169 conn(0x7f128c079650 msgr2=0x7f128c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 -- 172.21.15.87:0/4143381169 shutdown_connections 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.264+0000 7f12933be700 1 -- 172.21.15.87:0/4143381169 wait complete. 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 Processor -- start 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 -- start start 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 0x7f128c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c1084f0 0x7f128c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f128c112850 con 0x7f128c102f10 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f128c1126d0 con 0x7f128c102550 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f12933be700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f128c1129d0 con 0x7f128c1084f0 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40476/0 (socket says 172.21.15.87:40476) 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 -- 172.21.15.87:0/4106552353 learned_addr learned my addr 172.21.15.87:0/4106552353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f129115a700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 0x7f128c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 -- 172.21.15.87:0/4106552353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c1084f0 msgr2=0x7f128c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f129195b700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c1084f0 0x7f128c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c1084f0 0x7f128c19d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 -- 172.21.15.87:0/4106552353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 msgr2=0x7f128c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 0x7f128c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.265+0000 7f1290959700 1 -- 172.21.15.87:0/4106552353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c19dd60 con 0x7f128c102f10 2024-06-06T07:46:43.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f129115a700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 0x7f128c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f1290959700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f127c00b530 tx=0x7f127c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c015020 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c19e050 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c011d10 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c010e50 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.266+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c1a58d0 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.267+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c103ba0 con 0x7f128c102f10 2024-06-06T07:46:43.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.268+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f127c018020 con 0x7f128c102f10 2024-06-06T07:46:43.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.269+0000 7f12827fc700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 0x7f12780a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.269+0000 7f129115a700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 0x7f12780a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.269+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f127c0cf250 con 0x7f128c102f10 2024-06-06T07:46:43.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.269+0000 7f129115a700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 0x7f12780a5310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1288006010 tx=0x7f128800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:43.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.272+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f127c01f020 con 0x7f128c102f10 2024-06-06T07:46:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.501+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f128c19a0e0 con 0x7f128c102f10 2024-06-06T07:46:43.502 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:43 smithi087 ceph-mon[32276]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.502+0000 7f12827fc700 1 -- 172.21.15.87:0/4106552353 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f128c19a0e0 con 0x7f128c102f10 2024-06-06T07:46:43.502 INFO:teuthology.orchestra.run.smithi087.stdout:343597383728 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 msgr2=0x7f12780a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 0x7f12780a5310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1288006010 tx=0x7f128800b540 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 msgr2=0x7f128c1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f127c00b530 tx=0x7f127c00b8f0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 shutdown_connections 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f12780a2e50 0x7f12780a5310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102550 0x7f128c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f128c1084f0 0x7f128c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 --2- 172.21.15.87:0/4106552353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102f10 0x7f128c1992e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 >> 172.21.15.87:0/4106552353 conn(0x7f128c079650 msgr2=0x7f128c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:43.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 shutdown_connections 2024-06-06T07:46:43.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.504+0000 7f12933be700 1 -- 172.21.15.87:0/4106552353 wait complete. 2024-06-06T07:46:43.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:43 smithi195 ceph-mon[37040]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:43.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:43 smithi181 ceph-mon[36790]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- 172.21.15.87:0/1683063493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684107480 msgr2=0x7f1684107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- 172.21.15.87:0/1683063493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684107480 0x7f1684107860 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1678009990 tx=0x7f1678009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- 172.21.15.87:0/1683063493 shutdown_connections 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- 172.21.15.87:0/1683063493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- 172.21.15.87:0/1683063493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684107480 0x7f1684107860 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- 172.21.15.87:0/1683063493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684101b10 0x7f1684101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- 172.21.15.87:0/1683063493 >> 172.21.15.87:0/1683063493 conn(0x7f1684077990 msgr2=0x7f16840fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:43.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- 172.21.15.87:0/1683063493 shutdown_connections 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- 172.21.15.87:0/1683063493 wait complete. 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 Processor -- start 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- start start 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684107480 0x7f16841978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16841116c0 con 0x7f1684101b10 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1684111540 con 0x7f16841024d0 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.673+0000 7f168ab40700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1684111840 con 0x7f1684107480 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40500/0 (socket says 172.21.15.87:40500) 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 -- 172.21.15.87:0/2988765545 learned_addr learned my addr 172.21.15.87:0/2988765545 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16890dd700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684107480 0x7f16841978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f1683fff700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 -- 172.21.15.87:0/2988765545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684107480 msgr2=0x7f16841978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684107480 0x7f16841978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 -- 172.21.15.87:0/2988765545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 msgr2=0x7f168419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 -- 172.21.15.87:0/2988765545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1684198020 con 0x7f1684101b10 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f1683fff700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f16888dc700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f167800bff0 tx=0x7f1678005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678027070 con 0x7f1684101b10 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16841982b0 con 0x7f1684101b10 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16780040d0 con 0x7f1684101b10 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167800f460 con 0x7f1684101b10 2024-06-06T07:46:43.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.674+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1684100a00 con 0x7f1684101b10 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.675+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1684102b20 con 0x7f1684101b10 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.676+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f167802a070 con 0x7f1684101b10 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.677+0000 7f1681ffb700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 0x7f16740a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.677+0000 7f1683fff700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 0x7f16740a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.677+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f16780d0c60 con 0x7f1684101b10 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.678+0000 7f1683fff700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 0x7f16740a7380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1670005fd0 tx=0x7f1670005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:43.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.681+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1678031020 con 0x7f1684101b10 2024-06-06T07:46:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.890+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1684069180 con 0x7f1684101b10 2024-06-06T07:46:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.890+0000 7f1681ffb700 1 -- 172.21.15.87:0/2988765545 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1678013070 con 0x7f1684101b10 2024-06-06T07:46:43.891 INFO:teuthology.orchestra.run.smithi087.stdout:253403070527 2024-06-06T07:46:43.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 msgr2=0x7f16740a7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 0x7f16740a7380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1670005fd0 tx=0x7f1670005e30 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 msgr2=0x7f168419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f167800bff0 tx=0x7f1678005590 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 shutdown_connections 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f16740a4ec0 0x7f16740a7380 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1684107480 0x7f16841978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1684101b10 0x7f168419d700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 --2- 172.21.15.87:0/2988765545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16841024d0 0x7f168419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.892+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 >> 172.21.15.87:0/2988765545 conn(0x7f1684077990 msgr2=0x7f16840feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.893+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 shutdown_connections 2024-06-06T07:46:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:43.893+0000 7f168ab40700 1 -- 172.21.15.87:0/2988765545 wait complete. 2024-06-06T07:46:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:44 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4106552353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T07:46:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:44 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2988765545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T07:46:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:44 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4106552353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T07:46:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:44 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2988765545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T07:46:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:44 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4106552353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T07:46:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:44 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2988765545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T07:46:44.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070519 got 253403070527 for osd.8 2024-06-06T07:46:44.674 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:45.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383722 got 343597383728 for osd.11 2024-06-06T07:46:45.182 DEBUG:teuthology.parallel:result is None 2024-06-06T07:46:45.182 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T07:46:45.183 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:46:45.447 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:45 smithi195 ceph-mon[37040]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:45 smithi181 ceph-mon[36790]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:45.569 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:45 smithi087 ceph-mon[32276]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:46:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:46 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:46 smithi195 ceph-mon[37040]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:46 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:46 smithi181 ceph-mon[36790]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:46.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:46 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:46.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:46 smithi087 ceph-mon[32276]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 -- 172.21.15.87:0/1749633734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 msgr2=0x7f4d7410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 --2- 172.21.15.87:0/1749633734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7410f320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0099c0 tx=0x7f4d6c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 -- 172.21.15.87:0/1749633734 shutdown_connections 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 --2- 172.21.15.87:0/1749633734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7410f320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 --2- 172.21.15.87:0/1749633734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d741084e0 0x7f4d741088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 --2- 172.21.15.87:0/1749633734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d7406a170 0x7f4d7406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 -- 172.21.15.87:0/1749633734 >> 172.21.15.87:0/1749633734 conn(0x7f4d740782c0 msgr2=0x7f4d740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 -- 172.21.15.87:0/1749633734 shutdown_connections 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.204+0000 7f4d7b461700 1 -- 172.21.15.87:0/1749633734 wait complete. 2024-06-06T07:46:47.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 Processor -- start 2024-06-06T07:46:47.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 -- start start 2024-06-06T07:46:47.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 0x7f4d7419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:47.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:47.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d741084e0 0x7f4d74198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74112800 con 0x7f4d7406ab10 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74112680 con 0x7f4d741084e0 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d7b461700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74112980 con 0x7f4d7406a170 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d789fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d789fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43746/0 (socket says 172.21.15.87:43746) 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.205+0000 7f4d789fc700 1 -- 172.21.15.87:0/2670883613 learned_addr learned my addr 172.21.15.87:0/2670883613 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d791fd700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 0x7f4d7419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d799fe700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d741084e0 0x7f4d74198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 -- 172.21.15.87:0/2670883613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 msgr2=0x7f4d7419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 0x7f4d7419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 -- 172.21.15.87:0/2670883613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d741084e0 msgr2=0x7f4d74198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d741084e0 0x7f4d74198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 -- 172.21.15.87:0/2670883613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d74199220 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d791fd700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 0x7f4d7419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d789fc700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c9b0 tx=0x7f4d6400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64007b00 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d64007ca0 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d74199510 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64018780 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.206+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d741a5780 con 0x7f4d7406ab10 2024-06-06T07:46:47.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.207+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d74069180 con 0x7f4d7406ab10 2024-06-06T07:46:47.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.208+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d6400f450 con 0x7f4d7406ab10 2024-06-06T07:46:47.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.209+0000 7f4d6a7fc700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 0x7f4d600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:47.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.209+0000 7f4d791fd700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 0x7f4d600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:47.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.209+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4d640cf3f0 con 0x7f4d7406ab10 2024-06-06T07:46:47.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.209+0000 7f4d791fd700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 0x7f4d600a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d74107950 tx=0x7f4d70009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:47.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.213+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d64014030 con 0x7f4d7406ab10 2024-06-06T07:46:47.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.408+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d740628d0 con 0x7f4d600a2ca0 2024-06-06T07:46:47.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.417+0000 7f4d6a7fc700 1 -- 172.21.15.87:0/2670883613 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340390 (secure 0 0 0) 0x7f4d740628d0 con 0x7f4d600a2ca0 2024-06-06T07:46:47.422 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 msgr2=0x7f4d600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 0x7f4d600a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d74107950 tx=0x7f4d70009570 comp rx=0 tx=0).stop 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 msgr2=0x7f4d7419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c9b0 tx=0x7f4d6400cd70 comp rx=0 tx=0).stop 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 shutdown_connections 2024-06-06T07:46:47.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4d600a2ca0 0x7f4d600a5160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d741084e0 0x7f4d74198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d7406a170 0x7f4d7419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 --2- 172.21.15.87:0/2670883613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d7406ab10 0x7f4d7419edf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 >> 172.21.15.87:0/2670883613 conn(0x7f4d740782c0 msgr2=0x7f4d740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 shutdown_connections 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:47.421+0000 7f4d7b461700 1 -- 172.21.15.87:0/2670883613 wait complete. 2024-06-06T07:46:47.427 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:46:47.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:47 smithi195 ceph-mon[37040]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T07:46:47.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:46:47.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:46:47.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:47.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:47 smithi087 ceph-mon[32276]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T07:46:47.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:46:47.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:46:47.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:47 smithi181 ceph-mon[36790]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T07:46:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:46:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:46:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:46:48.168 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-06-06T07:46:46.182611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657062,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":758,"num_read_kb":498,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385464,"kb_used_data":15428,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556360,"statfs":{"total":1151940427776,"available":1151545712640,"internally_reserved":0,"allocated":15798272,"data_stored":8787716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003485"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228666+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:45:48.228666+0000","last_peered":"2024-06-06T07:45:48.228666+0000","last_clean":"2024-06-06T07:45:48.228666+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:45:48.228666+0000","last_undegraded":"2024-06-06T07:45:48.228666+0000","last_fullsized":"2024-06-06T07:45:48.228666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225467+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:45:48.225467+0000","last_peered":"2024-06-06T07:45:48.225467+0000","last_clean":"2024-06-06T07:45:48.225467+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:45:48.225467+0000","last_undegraded":"2024-06-06T07:45:48.225467+0000","last_fullsized":"2024-06-06T07:45:48.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225222+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:45:48.225222+0000","last_peered":"2024-06-06T07:45:48.225222+0000","last_clean":"2024-06-06T07:45:48.225222+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:45:48.225222+0000","last_undegraded":"2024-06-06T07:45:48.225222+0000","last_fullsized":"2024-06-06T07:45:48.225222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232643+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:45:48.232643+0000","last_peered":"2024-06-06T07:45:48.232643+0000","last_clean":"2024-06-06T07:45:48.232643+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:45:48.232643+0000","last_undegraded":"2024-06-06T07:45:48.232643+0000","last_fullsized":"2024-06-06T07:45:48.232643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226546+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:45:48.226546+0000","last_peered":"2024-06-06T07:45:48.226546+0000","last_clean":"2024-06-06T07:45:48.226546+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:45:48.226546+0000","last_undegraded":"2024-06-06T07:45:48.226546+0000","last_fullsized":"2024-06-06T07:45:48.226546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230082+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:45:48.230082+0000","last_peered":"2024-06-06T07:45:48.230082+0000","last_clean":"2024-06-06T07:45:48.230082+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:45:48.230082+0000","last_undegraded":"2024-06-06T07:45:48.230082+0000","last_fullsized":"2024-06-06T07:45:48.230082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226535+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:45:48.226535+0000","last_peered":"2024-06-06T07:45:48.226535+0000","last_clean":"2024-06-06T07:45:48.226535+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:45:48.226535+0000","last_undegraded":"2024-06-06T07:45:48.226535+0000","last_fullsized":"2024-06-06T07:45:48.226535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230059+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:45:48.230059+0000","last_peered":"2024-06-06T07:45:48.230059+0000","last_clean":"2024-06-06T07:45:48.230059+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:45:48.230059+0000","last_undegraded":"2024-06-06T07:45:48.230059+0000","last_fullsized":"2024-06-06T07:45:48.230059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918574+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:45:48.918574+0000","last_peered":"2024-06-06T07:45:48.918574+0000","last_clean":"2024-06-06T07:45:48.918574+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:45:48.918574+0000","last_undegraded":"2024-06-06T07:45:48.918574+0000","last_fullsized":"2024-06-06T07:45:48.918574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230150+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:45:48.230150+0000","last_peered":"2024-06-06T07:45:48.230150+0000","last_clean":"2024-06-06T07:45:48.230150+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:45:48.230150+0000","last_undegraded":"2024-06-06T07:45:48.230150+0000","last_fullsized":"2024-06-06T07:45:48.230150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227980+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:45:48.227980+0000","last_peered":"2024-06-06T07:45:48.227980+0000","last_clean":"2024-06-06T07:45:48.227980+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:45:48.227980+0000","last_undegraded":"2024-06-06T07:45:48.227980+0000","last_fullsized":"2024-06-06T07:45:48.227980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232522+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:45:48.232522+0000","last_peered":"2024-06-06T07:45:48.232522+0000","last_clean":"2024-06-06T07:45:48.232522+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:45:48.232522+0000","last_undegraded":"2024-06-06T07:45:48.232522+0000","last_fullsized":"2024-06-06T07:45:48.232522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228466+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:45:48.228466+0000","last_peered":"2024-06-06T07:45:48.228466+0000","last_clean":"2024-06-06T07:45:48.228466+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:45:48.228466+0000","last_undegraded":"2024-06-06T07:45:48.228466+0000","last_fullsized":"2024-06-06T07:45:48.228466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225246+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:45:48.225246+0000","last_peered":"2024-06-06T07:45:48.225246+0000","last_clean":"2024-06-06T07:45:48.225246+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:45:48.225246+0000","last_undegraded":"2024-06-06T07:45:48.225246+0000","last_fullsized":"2024-06-06T07:45:48.225246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225110+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:45:48.225110+0000","last_peered":"2024-06-06T07:45:48.225110+0000","last_clean":"2024-06-06T07:45:48.225110+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:45:48.225110+0000","last_undegraded":"2024-06-06T07:45:48.225110+0000","last_fullsized":"2024-06-06T07:45:48.225110+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227417+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:45:48.227417+0000","last_peered":"2024-06-06T07:45:48.227417+0000","last_clean":"2024-06-06T07:45:48.227417+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:45:48.227417+0000","last_undegraded":"2024-06-06T07:45:48.227417+0000","last_fullsized":"2024-06-06T07:45:48.227417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228389+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:45:48.228389+0000","last_peered":"2024-06-06T07:45:48.228389+0000","last_clean":"2024-06-06T07:45:48.228389+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:45:48.228389+0000","last_undegraded":"2024-06-06T07:45:48.228389+0000","last_fullsized":"2024-06-06T07:45:48.228389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918302+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:45:48.918302+0000","last_peered":"2024-06-06T07:45:48.918302+0000","last_clean":"2024-06-06T07:45:48.918302+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:45:48.918302+0000","last_undegraded":"2024-06-06T07:45:48.918302+0000","last_fullsized":"2024-06-06T07:45:48.918302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230314+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:45:48.230314+0000","last_peered":"2024-06-06T07:45:48.230314+0000","last_clean":"2024-06-06T07:45:48.230314+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:45:48.230314+0000","last_undegraded":"2024-06-06T07:45:48.230314+0000","last_fullsized":"2024-06-06T07:45:48.230314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230313+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:45:48.230313+0000","last_peered":"2024-06-06T07:45:48.230313+0000","last_clean":"2024-06-06T07:45:48.230313+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:45:48.230313+0000","last_undegraded":"2024-06-06T07:45:48.230313+0000","last_fullsized":"2024-06-06T07:45:48.230313+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918847+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:45:48.918847+0000","last_peered":"2024-06-06T07:45:48.918847+0000","last_clean":"2024-06-06T07:45:48.918847+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:45:48.918847+0000","last_undegraded":"2024-06-06T07:45:48.918847+0000","last_fullsized":"2024-06-06T07:45:48.918847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230520+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:45:48.230520+0000","last_peered":"2024-06-06T07:45:48.230520+0000","last_clean":"2024-06-06T07:45:48.230520+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:45:48.230520+0000","last_undegraded":"2024-06-06T07:45:48.230520+0000","last_fullsized":"2024-06-06T07:45:48.230520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231916+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:45:48.231916+0000","last_peered":"2024-06-06T07:45:48.231916+0000","last_clean":"2024-06-06T07:45:48.231916+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:45:48.231916+0000","last_undegraded":"2024-06-06T07:45:48.231916+0000","last_fullsized":"2024-06-06T07:45:48.231916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230636+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:45:48.230636+0000","last_peered":"2024-06-06T07:45:48.230636+0000","last_clean":"2024-06-06T07:45:48.230636+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:45:48.230636+0000","last_undegraded":"2024-06-06T07:45:48.230636+0000","last_fullsized":"2024-06-06T07:45:48.230636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225342+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:45:48.225342+0000","last_peered":"2024-06-06T07:45:48.225342+0000","last_clean":"2024-06-06T07:45:48.225342+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:45:48.225342+0000","last_undegraded":"2024-06-06T07:45:48.225342+0000","last_fullsized":"2024-06-06T07:45:48.225342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230476+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:45:48.230476+0000","last_peered":"2024-06-06T07:45:48.230476+0000","last_clean":"2024-06-06T07:45:48.230476+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:45:48.230476+0000","last_undegraded":"2024-06-06T07:45:48.230476+0000","last_fullsized":"2024-06-06T07:45:48.230476+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227304+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:45:48.227304+0000","last_peered":"2024-06-06T07:45:48.227304+0000","last_clean":"2024-06-06T07:45:48.227304+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:45:48.227304+0000","last_undegraded":"2024-06-06T07:45:48.227304+0000","last_fullsized":"2024-06-06T07:45:48.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:45:48.228889+0000","last_peered":"2024-06-06T07:45:48.228889+0000","last_clean":"2024-06-06T07:45:48.228889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:45:48.228889+0000","last_undegraded":"2024-06-06T07:45:48.228889+0000","last_fullsized":"2024-06-06T07:45:48.228889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230296+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:45:48.230296+0000","last_peered":"2024-06-06T07:45:48.230296+0000","last_clean":"2024-06-06T07:45:48.230296+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:45:48.230296+0000","last_undegraded":"2024-06-06T07:45:48.230296+0000","last_fullsized":"2024-06-06T07:45:48.230296+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226641+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:45:48.226641+0000","last_peered":"2024-06-06T07:45:48.226641+0000","last_clean":"2024-06-06T07:45:48.226641+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:45:48.226641+0000","last_undegraded":"2024-06-06T07:45:48.226641+0000","last_fullsized":"2024-06-06T07:45:48.226641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232889+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:45:48.232889+0000","last_peered":"2024-06-06T07:45:48.232889+0000","last_clean":"2024-06-06T07:45:48.232889+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:45:48.232889+0000","last_undegraded":"2024-06-06T07:45:48.232889+0000","last_fullsized":"2024-06-06T07:45:48.232889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230186+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:45:48.230186+0000","last_peered":"2024-06-06T07:45:48.230186+0000","last_clean":"2024-06-06T07:45:48.230186+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:45:48.230186+0000","last_undegraded":"2024-06-06T07:45:48.230186+0000","last_fullsized":"2024-06-06T07:45:48.230186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":154,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:42.387852+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:46:42.387852+0000","last_peered":"2024-06-06T07:46:42.387852+0000","last_clean":"2024-06-06T07:46:42.387852+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:46:42.387852+0000","last_undegraded":"2024-06-06T07:46:42.387852+0000","last_fullsized":"2024-06-06T07:46:42.387852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226391+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:45:48.226391+0000","last_peered":"2024-06-06T07:45:48.226391+0000","last_clean":"2024-06-06T07:45:48.226391+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:45:48.226391+0000","last_undegraded":"2024-06-06T07:45:48.226391+0000","last_fullsized":"2024-06-06T07:45:48.226391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230472+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:45:48.230472+0000","last_peered":"2024-06-06T07:45:48.230472+0000","last_clean":"2024-06-06T07:45:48.230472+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:45:48.230472+0000","last_undegraded":"2024-06-06T07:45:48.230472+0000","last_fullsized":"2024-06-06T07:45:48.230472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229873+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:45:48.229873+0000","last_peered":"2024-06-06T07:45:48.229873+0000","last_clean":"2024-06-06T07:45:48.229873+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:45:48.229873+0000","last_undegraded":"2024-06-06T07:45:48.229873+0000","last_fullsized":"2024-06-06T07:45:48.229873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232551+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:45:48.232551+0000","last_peered":"2024-06-06T07:45:48.232551+0000","last_clean":"2024-06-06T07:45:48.232551+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:45:48.232551+0000","last_undegraded":"2024-06-06T07:45:48.232551+0000","last_fullsized":"2024-06-06T07:45:48.232551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228208+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:45:48.228208+0000","last_peered":"2024-06-06T07:45:48.228208+0000","last_clean":"2024-06-06T07:45:48.228208+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:45:48.228208+0000","last_undegraded":"2024-06-06T07:45:48.228208+0000","last_fullsized":"2024-06-06T07:45:48.228208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228227+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:45:48.228227+0000","last_peered":"2024-06-06T07:45:48.228227+0000","last_clean":"2024-06-06T07:45:48.228227+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:45:48.228227+0000","last_undegraded":"2024-06-06T07:45:48.228227+0000","last_fullsized":"2024-06-06T07:45:48.228227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230154+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:45:48.230154+0000","last_peered":"2024-06-06T07:45:48.230154+0000","last_clean":"2024-06-06T07:45:48.230154+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:45:48.230154+0000","last_undegraded":"2024-06-06T07:45:48.230154+0000","last_fullsized":"2024-06-06T07:45:48.230154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225389+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:45:48.225389+0000","last_peered":"2024-06-06T07:45:48.225389+0000","last_clean":"2024-06-06T07:45:48.225389+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:45:48.225389+0000","last_undegraded":"2024-06-06T07:45:48.225389+0000","last_fullsized":"2024-06-06T07:45:48.225389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231868+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:45:48.231868+0000","last_peered":"2024-06-06T07:45:48.231868+0000","last_clean":"2024-06-06T07:45:48.231868+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:45:48.231868+0000","last_undegraded":"2024-06-06T07:45:48.231868+0000","last_fullsized":"2024-06-06T07:45:48.231868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.916831+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:45:48.916831+0000","last_peered":"2024-06-06T07:45:48.916831+0000","last_clean":"2024-06-06T07:45:48.916831+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:45:48.916831+0000","last_undegraded":"2024-06-06T07:45:48.916831+0000","last_fullsized":"2024-06-06T07:45:48.916831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230696+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:45:48.230696+0000","last_peered":"2024-06-06T07:45:48.230696+0000","last_clean":"2024-06-06T07:45:48.230696+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:45:48.230696+0000","last_undegraded":"2024-06-06T07:45:48.230696+0000","last_fullsized":"2024-06-06T07:45:48.230696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":145,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:40.384306+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:46:40.384306+0000","last_peered":"2024-06-06T07:46:40.384306+0000","last_clean":"2024-06-06T07:46:40.384306+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:46:40.384306+0000","last_undegraded":"2024-06-06T07:46:40.384306+0000","last_fullsized":"2024-06-06T07:46:40.384306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227698+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:45:48.227698+0000","last_peered":"2024-06-06T07:45:48.227698+0000","last_clean":"2024-06-06T07:45:48.227698+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:45:48.227698+0000","last_undegraded":"2024-06-06T07:45:48.227698+0000","last_fullsized":"2024-06-06T07:45:48.227698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226070+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:45:48.226070+0000","last_peered":"2024-06-06T07:45:48.226070+0000","last_clean":"2024-06-06T07:45:48.226070+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:45:48.226070+0000","last_undegraded":"2024-06-06T07:45:48.226070+0000","last_fullsized":"2024-06-06T07:45:48.226070+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227073+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:45:48.227073+0000","last_peered":"2024-06-06T07:45:48.227073+0000","last_clean":"2024-06-06T07:45:48.227073+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:45:48.227073+0000","last_undegraded":"2024-06-06T07:45:48.227073+0000","last_fullsized":"2024-06-06T07:45:48.227073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225392+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:45:48.225392+0000","last_peered":"2024-06-06T07:45:48.225392+0000","last_clean":"2024-06-06T07:45:48.225392+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:45:48.225392+0000","last_undegraded":"2024-06-06T07:45:48.225392+0000","last_fullsized":"2024-06-06T07:45:48.225392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230814+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:45:48.230814+0000","last_peered":"2024-06-06T07:45:48.230814+0000","last_clean":"2024-06-06T07:45:48.230814+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:45:48.230814+0000","last_undegraded":"2024-06-06T07:45:48.230814+0000","last_fullsized":"2024-06-06T07:45:48.230814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232774+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:45:48.232774+0000","last_peered":"2024-06-06T07:45:48.232774+0000","last_clean":"2024-06-06T07:45:48.232774+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:45:48.232774+0000","last_undegraded":"2024-06-06T07:45:48.232774+0000","last_fullsized":"2024-06-06T07:45:48.232774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229012+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:45:48.229012+0000","last_peered":"2024-06-06T07:45:48.229012+0000","last_clean":"2024-06-06T07:45:48.229012+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:45:48.229012+0000","last_undegraded":"2024-06-06T07:45:48.229012+0000","last_fullsized":"2024-06-06T07:45:48.229012+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.917184+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:45:48.917184+0000","last_peered":"2024-06-06T07:45:48.917184+0000","last_clean":"2024-06-06T07:45:48.917184+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:45:48.917184+0000","last_undegraded":"2024-06-06T07:45:48.917184+0000","last_fullsized":"2024-06-06T07:45:48.917184+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":282,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:41.385665+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:46:41.385665+0000","last_peered":"2024-06-06T07:46:41.385665+0000","last_clean":"2024-06-06T07:46:41.385665+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:46:41.385665+0000","last_undegraded":"2024-06-06T07:46:41.385665+0000","last_fullsized":"2024-06-06T07:46:41.385665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62070,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":163,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227058+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:45:48.227058+0000","last_peered":"2024-06-06T07:45:48.227058+0000","last_clean":"2024-06-06T07:45:48.227058+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:45:48.227058+0000","last_undegraded":"2024-06-06T07:45:48.227058+0000","last_fullsized":"2024-06-06T07:45:48.227058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233667+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:45:48.233667+0000","last_peered":"2024-06-06T07:45:48.233667+0000","last_clean":"2024-06-06T07:45:48.233667+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:45:48.233667+0000","last_undegraded":"2024-06-06T07:45:48.233667+0000","last_fullsized":"2024-06-06T07:45:48.233667+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229897+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:45:48.229897+0000","last_peered":"2024-06-06T07:45:48.229897+0000","last_clean":"2024-06-06T07:45:48.229897+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:45:48.229897+0000","last_undegraded":"2024-06-06T07:45:48.229897+0000","last_fullsized":"2024-06-06T07:45:48.229897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228513+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:45:48.228513+0000","last_peered":"2024-06-06T07:45:48.228513+0000","last_clean":"2024-06-06T07:45:48.228513+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:45:48.228513+0000","last_undegraded":"2024-06-06T07:45:48.228513+0000","last_fullsized":"2024-06-06T07:45:48.228513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227501+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:45:48.227501+0000","last_peered":"2024-06-06T07:45:48.227501+0000","last_clean":"2024-06-06T07:45:48.227501+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:45:48.227501+0000","last_undegraded":"2024-06-06T07:45:48.227501+0000","last_fullsized":"2024-06-06T07:45:48.227501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232326+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:45:48.232326+0000","last_peered":"2024-06-06T07:45:48.232326+0000","last_clean":"2024-06-06T07:45:48.232326+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:45:48.232326+0000","last_undegraded":"2024-06-06T07:45:48.232326+0000","last_fullsized":"2024-06-06T07:45:48.232326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226916+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:45:48.226916+0000","last_peered":"2024-06-06T07:45:48.226916+0000","last_clean":"2024-06-06T07:45:48.226916+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:45:48.226916+0000","last_undegraded":"2024-06-06T07:45:48.226916+0000","last_fullsized":"2024-06-06T07:45:48.226916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.919117+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:45:48.919117+0000","last_peered":"2024-06-06T07:45:48.919117+0000","last_clean":"2024-06-06T07:45:48.919117+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:45:48.919117+0000","last_undegraded":"2024-06-06T07:45:48.919117+0000","last_fullsized":"2024-06-06T07:45:48.919117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'4","reported_seq":152,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:43.425603+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:46:43.425603+0000","last_peered":"2024-06-06T07:46:43.425603+0000","last_clean":"2024-06-06T07:46:43.425603+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:46:43.425603+0000","last_undegraded":"2024-06-06T07:46:43.425603+0000","last_fullsized":"2024-06-06T07:46:43.425603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226554+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:45:48.226554+0000","last_peered":"2024-06-06T07:45:48.226554+0000","last_clean":"2024-06-06T07:45:48.226554+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:45:48.226554+0000","last_undegraded":"2024-06-06T07:45:48.226554+0000","last_fullsized":"2024-06-06T07:45:48.226554+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228891+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:45:48.228891+0000","last_peered":"2024-06-06T07:45:48.228891+0000","last_clean":"2024-06-06T07:45:48.228891+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:45:48.228891+0000","last_undegraded":"2024-06-06T07:45:48.228891+0000","last_fullsized":"2024-06-06T07:45:48.228891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.248883+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:45:48.248883+0000","last_peered":"2024-06-06T07:45:48.248883+0000","last_clean":"2024-06-06T07:45:48.248883+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:45:48.248883+0000","last_undegraded":"2024-06-06T07:45:48.248883+0000","last_fullsized":"2024-06-06T07:45:48.248883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231827+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:45:48.231827+0000","last_peered":"2024-06-06T07:45:48.231827+0000","last_clean":"2024-06-06T07:45:48.231827+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:45:48.231827+0000","last_undegraded":"2024-06-06T07:45:48.231827+0000","last_fullsized":"2024-06-06T07:45:48.231827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231768+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:45:48.231768+0000","last_peered":"2024-06-06T07:45:48.231768+0000","last_clean":"2024-06-06T07:45:48.231768+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:45:48.231768+0000","last_undegraded":"2024-06-06T07:45:48.231768+0000","last_fullsized":"2024-06-06T07:45:48.231768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.252519+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:45:48.252519+0000","last_peered":"2024-06-06T07:45:48.252519+0000","last_clean":"2024-06-06T07:45:48.252519+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:45:48.252519+0000","last_undegraded":"2024-06-06T07:45:48.252519+0000","last_fullsized":"2024-06-06T07:45:48.252519+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228071+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:45:48.228071+0000","last_peered":"2024-06-06T07:45:48.228071+0000","last_clean":"2024-06-06T07:45:48.228071+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:45:48.228071+0000","last_undegraded":"2024-06-06T07:45:48.228071+0000","last_fullsized":"2024-06-06T07:45:48.228071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226314+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:45:48.226314+0000","last_peered":"2024-06-06T07:45:48.226314+0000","last_clean":"2024-06-06T07:45:48.226314+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:45:48.226314+0000","last_undegraded":"2024-06-06T07:45:48.226314+0000","last_fullsized":"2024-06-06T07:45:48.226314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.917806+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:45:48.917806+0000","last_peered":"2024-06-06T07:45:48.917806+0000","last_clean":"2024-06-06T07:45:48.917806+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:45:48.917806+0000","last_undegraded":"2024-06-06T07:45:48.917806+0000","last_fullsized":"2024-06-06T07:45:48.917806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230403+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:45:48.230403+0000","last_peered":"2024-06-06T07:45:48.230403+0000","last_clean":"2024-06-06T07:45:48.230403+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:45:48.230403+0000","last_undegraded":"2024-06-06T07:45:48.230403+0000","last_fullsized":"2024-06-06T07:45:48.230403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228576+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:45:48.228576+0000","last_peered":"2024-06-06T07:45:48.228576+0000","last_clean":"2024-06-06T07:45:48.228576+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:45:48.228576+0000","last_undegraded":"2024-06-06T07:45:48.228576+0000","last_fullsized":"2024-06-06T07:45:48.228576+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226836+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:45:48.226836+0000","last_peered":"2024-06-06T07:45:48.226836+0000","last_clean":"2024-06-06T07:45:48.226836+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:45:48.226836+0000","last_undegraded":"2024-06-06T07:45:48.226836+0000","last_fullsized":"2024-06-06T07:45:48.226836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229726+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:45:48.229726+0000","last_peered":"2024-06-06T07:45:48.229726+0000","last_clean":"2024-06-06T07:45:48.229726+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:45:48.229726+0000","last_undegraded":"2024-06-06T07:45:48.229726+0000","last_fullsized":"2024-06-06T07:45:48.229726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228351+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:45:48.228351+0000","last_peered":"2024-06-06T07:45:48.228351+0000","last_clean":"2024-06-06T07:45:48.228351+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:45:48.228351+0000","last_undegraded":"2024-06-06T07:45:48.228351+0000","last_fullsized":"2024-06-06T07:45:48.228351+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231753+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:45:48.231753+0000","last_peered":"2024-06-06T07:45:48.231753+0000","last_clean":"2024-06-06T07:45:48.231753+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:45:48.231753+0000","last_undegraded":"2024-06-06T07:45:48.231753+0000","last_fullsized":"2024-06-06T07:45:48.231753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:43.424826+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:46:43.424826+0000","last_peered":"2024-06-06T07:46:43.424826+0000","last_clean":"2024-06-06T07:46:43.424826+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:46:43.424826+0000","last_undegraded":"2024-06-06T07:46:43.424826+0000","last_fullsized":"2024-06-06T07:46:43.424826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227786+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:45:48.227786+0000","last_peered":"2024-06-06T07:45:48.227786+0000","last_clean":"2024-06-06T07:45:48.227786+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:45:48.227786+0000","last_undegraded":"2024-06-06T07:45:48.227786+0000","last_fullsized":"2024-06-06T07:45:48.227786+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230413+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:45:48.230413+0000","last_peered":"2024-06-06T07:45:48.230413+0000","last_clean":"2024-06-06T07:45:48.230413+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:45:48.230413+0000","last_undegraded":"2024-06-06T07:45:48.230413+0000","last_fullsized":"2024-06-06T07:45:48.230413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230682+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:45:48.230682+0000","last_peered":"2024-06-06T07:45:48.230682+0000","last_clean":"2024-06-06T07:45:48.230682+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:45:48.230682+0000","last_undegraded":"2024-06-06T07:45:48.230682+0000","last_fullsized":"2024-06-06T07:45:48.230682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":379,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226354+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:45:48.226354+0000","last_peered":"2024-06-06T07:45:48.226354+0000","last_clean":"2024-06-06T07:45:48.226354+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:45:48.226354+0000","last_undegraded":"2024-06-06T07:45:48.226354+0000","last_fullsized":"2024-06-06T07:45:48.226354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233325+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:45:48.233325+0000","last_peered":"2024-06-06T07:45:48.233325+0000","last_clean":"2024-06-06T07:45:48.233325+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:45:48.233325+0000","last_undegraded":"2024-06-06T07:45:48.233325+0000","last_fullsized":"2024-06-06T07:45:48.233325+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227945+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:45:48.227945+0000","last_peered":"2024-06-06T07:45:48.227945+0000","last_clean":"2024-06-06T07:45:48.227945+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:45:48.227945+0000","last_undegraded":"2024-06-06T07:45:48.227945+0000","last_fullsized":"2024-06-06T07:45:48.227945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226537+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:45:48.226537+0000","last_peered":"2024-06-06T07:45:48.226537+0000","last_clean":"2024-06-06T07:45:48.226537+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:45:48.226537+0000","last_undegraded":"2024-06-06T07:45:48.226537+0000","last_fullsized":"2024-06-06T07:45:48.226537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227366+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:45:48.227366+0000","last_peered":"2024-06-06T07:45:48.227366+0000","last_clean":"2024-06-06T07:45:48.227366+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:45:48.227366+0000","last_undegraded":"2024-06-06T07:45:48.227366+0000","last_fullsized":"2024-06-06T07:45:48.227366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226961+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:45:48.226961+0000","last_peered":"2024-06-06T07:45:48.226961+0000","last_clean":"2024-06-06T07:45:48.226961+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:45:48.226961+0000","last_undegraded":"2024-06-06T07:45:48.226961+0000","last_fullsized":"2024-06-06T07:45:48.226961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228377+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:45:48.228377+0000","last_peered":"2024-06-06T07:45:48.228377+0000","last_clean":"2024-06-06T07:45:48.228377+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:45:48.228377+0000","last_undegraded":"2024-06-06T07:45:48.228377+0000","last_fullsized":"2024-06-06T07:45:48.228377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230094+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:45:48.230094+0000","last_peered":"2024-06-06T07:45:48.230094+0000","last_clean":"2024-06-06T07:45:48.230094+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:45:48.230094+0000","last_undegraded":"2024-06-06T07:45:48.230094+0000","last_fullsized":"2024-06-06T07:45:48.230094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918407+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:45:48.918407+0000","last_peered":"2024-06-06T07:45:48.918407+0000","last_clean":"2024-06-06T07:45:48.918407+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:45:48.918407+0000","last_undegraded":"2024-06-06T07:45:48.918407+0000","last_fullsized":"2024-06-06T07:45:48.918407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230746+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:45:48.230746+0000","last_peered":"2024-06-06T07:45:48.230746+0000","last_clean":"2024-06-06T07:45:48.230746+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:45:48.230746+0000","last_undegraded":"2024-06-06T07:45:48.230746+0000","last_fullsized":"2024-06-06T07:45:48.230746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230069+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:45:48.230069+0000","last_peered":"2024-06-06T07:45:48.230069+0000","last_clean":"2024-06-06T07:45:48.230069+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:45:48.230069+0000","last_undegraded":"2024-06-06T07:45:48.230069+0000","last_fullsized":"2024-06-06T07:45:48.230069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233142+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:45:48.233142+0000","last_peered":"2024-06-06T07:45:48.233142+0000","last_clean":"2024-06-06T07:45:48.233142+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:45:48.233142+0000","last_undegraded":"2024-06-06T07:45:48.233142+0000","last_fullsized":"2024-06-06T07:45:48.233142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225523+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:45:48.225523+0000","last_peered":"2024-06-06T07:45:48.225523+0000","last_clean":"2024-06-06T07:45:48.225523+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:45:48.225523+0000","last_undegraded":"2024-06-06T07:45:48.225523+0000","last_fullsized":"2024-06-06T07:45:48.225523+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232360+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:45:48.232360+0000","last_peered":"2024-06-06T07:45:48.232360+0000","last_clean":"2024-06-06T07:45:48.232360+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:45:48.232360+0000","last_undegraded":"2024-06-06T07:45:48.232360+0000","last_fullsized":"2024-06-06T07:45:48.232360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228080+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:45:48.228080+0000","last_peered":"2024-06-06T07:45:48.228080+0000","last_clean":"2024-06-06T07:45:48.228080+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:45:48.228080+0000","last_undegraded":"2024-06-06T07:45:48.228080+0000","last_fullsized":"2024-06-06T07:45:48.228080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226391+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:45:48.226391+0000","last_peered":"2024-06-06T07:45:48.226391+0000","last_clean":"2024-06-06T07:45:48.226391+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:45:48.226391+0000","last_undegraded":"2024-06-06T07:45:48.226391+0000","last_fullsized":"2024-06-06T07:45:48.226391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918950+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:45:48.918950+0000","last_peered":"2024-06-06T07:45:48.918950+0000","last_clean":"2024-06-06T07:45:48.918950+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:45:48.918950+0000","last_undegraded":"2024-06-06T07:45:48.918950+0000","last_fullsized":"2024-06-06T07:45:48.918950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230785+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:45:48.230785+0000","last_peered":"2024-06-06T07:45:48.230785+0000","last_clean":"2024-06-06T07:45:48.230785+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:45:48.230785+0000","last_undegraded":"2024-06-06T07:45:48.230785+0000","last_fullsized":"2024-06-06T07:45:48.230785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228017+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:45:48.228017+0000","last_peered":"2024-06-06T07:45:48.228017+0000","last_clean":"2024-06-06T07:45:48.228017+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:45:48.228017+0000","last_undegraded":"2024-06-06T07:45:48.228017+0000","last_fullsized":"2024-06-06T07:45:48.228017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232384+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:45:48.232384+0000","last_peered":"2024-06-06T07:45:48.232384+0000","last_clean":"2024-06-06T07:45:48.232384+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:45:48.232384+0000","last_undegraded":"2024-06-06T07:45:48.232384+0000","last_fullsized":"2024-06-06T07:45:48.232384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229794+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:45:48.229794+0000","last_peered":"2024-06-06T07:45:48.229794+0000","last_clean":"2024-06-06T07:45:48.229794+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:45:48.229794+0000","last_undegraded":"2024-06-06T07:45:48.229794+0000","last_fullsized":"2024-06-06T07:45:48.229794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230754+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:45:48.230754+0000","last_peered":"2024-06-06T07:45:48.230754+0000","last_clean":"2024-06-06T07:45:48.230754+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:45:48.230754+0000","last_undegraded":"2024-06-06T07:45:48.230754+0000","last_fullsized":"2024-06-06T07:45:48.230754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232360+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:45:48.232360+0000","last_peered":"2024-06-06T07:45:48.232360+0000","last_clean":"2024-06-06T07:45:48.232360+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:45:48.232360+0000","last_undegraded":"2024-06-06T07:45:48.232360+0000","last_fullsized":"2024-06-06T07:45:48.232360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226643+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:45:48.226643+0000","last_peered":"2024-06-06T07:45:48.226643+0000","last_clean":"2024-06-06T07:45:48.226643+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:45:48.226643+0000","last_undegraded":"2024-06-06T07:45:48.226643+0000","last_fullsized":"2024-06-06T07:45:48.226643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227808+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:45:48.227808+0000","last_peered":"2024-06-06T07:45:48.227808+0000","last_clean":"2024-06-06T07:45:48.227808+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:45:48.227808+0000","last_undegraded":"2024-06-06T07:45:48.227808+0000","last_fullsized":"2024-06-06T07:45:48.227808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227636+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:45:48.227636+0000","last_peered":"2024-06-06T07:45:48.227636+0000","last_clean":"2024-06-06T07:45:48.227636+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:45:48.227636+0000","last_undegraded":"2024-06-06T07:45:48.227636+0000","last_fullsized":"2024-06-06T07:45:48.227636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.237168+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:45:48.237168+0000","last_peered":"2024-06-06T07:45:48.237168+0000","last_clean":"2024-06-06T07:45:48.237168+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:45:48.237168+0000","last_undegraded":"2024-06-06T07:45:48.237168+0000","last_fullsized":"2024-06-06T07:45:48.237168+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228151+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:45:48.228151+0000","last_peered":"2024-06-06T07:45:48.228151+0000","last_clean":"2024-06-06T07:45:48.228151+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:45:48.228151+0000","last_undegraded":"2024-06-06T07:45:48.228151+0000","last_fullsized":"2024-06-06T07:45:48.228151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225124+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:45:48.225124+0000","last_peered":"2024-06-06T07:45:48.225124+0000","last_clean":"2024-06-06T07:45:48.225124+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:45:48.225124+0000","last_undegraded":"2024-06-06T07:45:48.225124+0000","last_fullsized":"2024-06-06T07:45:48.225124+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233008+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:45:48.233008+0000","last_peered":"2024-06-06T07:45:48.233008+0000","last_clean":"2024-06-06T07:45:48.233008+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:45:48.233008+0000","last_undegraded":"2024-06-06T07:45:48.233008+0000","last_fullsized":"2024-06-06T07:45:48.233008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226927+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:45:48.226927+0000","last_peered":"2024-06-06T07:45:48.226927+0000","last_clean":"2024-06-06T07:45:48.226927+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:45:48.226927+0000","last_undegraded":"2024-06-06T07:45:48.226927+0000","last_fullsized":"2024-06-06T07:45:48.226927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231333+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:45:48.231333+0000","last_peered":"2024-06-06T07:45:48.231333+0000","last_clean":"2024-06-06T07:45:48.231333+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:45:48.231333+0000","last_undegraded":"2024-06-06T07:45:48.231333+0000","last_fullsized":"2024-06-06T07:45:48.231333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.235572+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:45:48.235572+0000","last_peered":"2024-06-06T07:45:48.235572+0000","last_clean":"2024-06-06T07:45:48.235572+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:45:48.235572+0000","last_undegraded":"2024-06-06T07:45:48.235572+0000","last_fullsized":"2024-06-06T07:45:48.235572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227270+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:45:48.227270+0000","last_peered":"2024-06-06T07:45:48.227270+0000","last_clean":"2024-06-06T07:45:48.227270+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:45:48.227270+0000","last_undegraded":"2024-06-06T07:45:48.227270+0000","last_fullsized":"2024-06-06T07:45:48.227270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228190+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:45:48.228190+0000","last_peered":"2024-06-06T07:45:48.228190+0000","last_clean":"2024-06-06T07:45:48.228190+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:45:48.228190+0000","last_undegraded":"2024-06-06T07:45:48.228190+0000","last_fullsized":"2024-06-06T07:45:48.228190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230518+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:45:48.230518+0000","last_peered":"2024-06-06T07:45:48.230518+0000","last_clean":"2024-06-06T07:45:48.230518+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:45:48.230518+0000","last_undegraded":"2024-06-06T07:45:48.230518+0000","last_fullsized":"2024-06-06T07:45:48.230518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229982+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:45:48.229982+0000","last_peered":"2024-06-06T07:45:48.229982+0000","last_clean":"2024-06-06T07:45:48.229982+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:45:48.229982+0000","last_undegraded":"2024-06-06T07:45:48.229982+0000","last_fullsized":"2024-06-06T07:45:48.229982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225324+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:45:48.225324+0000","last_peered":"2024-06-06T07:45:48.225324+0000","last_clean":"2024-06-06T07:45:48.225324+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:45:48.225324+0000","last_undegraded":"2024-06-06T07:45:48.225324+0000","last_fullsized":"2024-06-06T07:45:48.225324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228158+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:45:48.228158+0000","last_peered":"2024-06-06T07:45:48.228158+0000","last_clean":"2024-06-06T07:45:48.228158+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:45:48.228158+0000","last_undegraded":"2024-06-06T07:45:48.228158+0000","last_fullsized":"2024-06-06T07:45:48.228158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228024+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:45:48.228024+0000","last_peered":"2024-06-06T07:45:48.228024+0000","last_clean":"2024-06-06T07:45:48.228024+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:45:48.228024+0000","last_undegraded":"2024-06-06T07:45:48.228024+0000","last_fullsized":"2024-06-06T07:45:48.228024+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232169+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:45:48.232169+0000","last_peered":"2024-06-06T07:45:48.232169+0000","last_clean":"2024-06-06T07:45:48.232169+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:45:48.232169+0000","last_undegraded":"2024-06-06T07:45:48.232169+0000","last_fullsized":"2024-06-06T07:45:48.232169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230950+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:45:48.230950+0000","last_peered":"2024-06-06T07:45:48.230950+0000","last_clean":"2024-06-06T07:45:48.230950+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:45:48.230950+0000","last_undegraded":"2024-06-06T07:45:48.230950+0000","last_fullsized":"2024-06-06T07:45:48.230950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231576+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:45:48.231576+0000","last_peered":"2024-06-06T07:45:48.231576+0000","last_clean":"2024-06-06T07:45:48.231576+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:45:48.231576+0000","last_undegraded":"2024-06-06T07:45:48.231576+0000","last_fullsized":"2024-06-06T07:45:48.231576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230833+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:45:48.230833+0000","last_peered":"2024-06-06T07:45:48.230833+0000","last_clean":"2024-06-06T07:45:48.230833+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:45:48.230833+0000","last_undegraded":"2024-06-06T07:45:48.230833+0000","last_fullsized":"2024-06-06T07:45:48.230833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228006+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:45:48.228006+0000","last_peered":"2024-06-06T07:45:48.228006+0000","last_clean":"2024-06-06T07:45:48.228006+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:45:48.228006+0000","last_undegraded":"2024-06-06T07:45:48.228006+0000","last_fullsized":"2024-06-06T07:45:48.228006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228406+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:45:48.228406+0000","last_peered":"2024-06-06T07:45:48.228406+0000","last_clean":"2024-06-06T07:45:48.228406+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:45:48.228406+0000","last_undegraded":"2024-06-06T07:45:48.228406+0000","last_fullsized":"2024-06-06T07:45:48.228406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227679+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:45:48.227679+0000","last_peered":"2024-06-06T07:45:48.227679+0000","last_clean":"2024-06-06T07:45:48.227679+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:45:48.227679+0000","last_undegraded":"2024-06-06T07:45:48.227679+0000","last_fullsized":"2024-06-06T07:45:48.227679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66460,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":622,"num_read_kb":379,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383729,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":570184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645366,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28812,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716340,"statfs":{"total":95995035648,"available":95965532160,"internally_reserved":0,"allocated":1830912,"data_stored":1223874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808890,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":570286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":570072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32892,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712260,"statfs":{"total":95995035648,"available":95961354240,"internally_reserved":0,"allocated":1748992,"data_stored":1166504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":565462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28200,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716952,"statfs":{"total":95995035648,"available":95966158848,"internally_reserved":0,"allocated":1204224,"data_stored":628024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":566850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40696,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704456,"statfs":{"total":95995035648,"available":95953362944,"internally_reserved":0,"allocated":1220608,"data_stored":628600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1146880,"data_stored":565656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":570502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37056,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708096,"statfs":{"total":95995035648,"available":95957090304,"internally_reserved":0,"allocated":1757184,"data_stored":1161702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:46:48.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:46:48.417 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:48.709 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:48 smithi087 ceph-mon[32276]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:48.710 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:48 smithi087 ceph-mon[32276]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T07:46:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:48 smithi195 ceph-mon[37040]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:48 smithi195 ceph-mon[37040]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T07:46:48.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:48 smithi181 ceph-mon[36790]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:48.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:48 smithi181 ceph-mon[36790]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T07:46:50.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.358+0000 7f86283de700 1 -- 172.21.15.87:0/1317356467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8620102520 msgr2=0x7f86201029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.358+0000 7f86283de700 1 --2- 172.21.15.87:0/1317356467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8620102520 0x7f86201029a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f861c0099c0 tx=0x7f861c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 -- 172.21.15.87:0/1317356467 shutdown_connections 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 --2- 172.21.15.87:0/1317356467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86201085c0 0x7f86201089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 --2- 172.21.15.87:0/1317356467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102ee0 0x7f862010f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 --2- 172.21.15.87:0/1317356467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8620102520 0x7f86201029a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 -- 172.21.15.87:0/1317356467 >> 172.21.15.87:0/1317356467 conn(0x7f8620079680 msgr2=0x7f8620079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 -- 172.21.15.87:0/1317356467 shutdown_connections 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 -- 172.21.15.87:0/1317356467 wait complete. 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 Processor -- start 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 -- start start 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.359+0000 7f86283de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102520 0x7f862019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:50.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8620102ee0 0x7f862019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8620112860 con 0x7f86201085c0 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86201126e0 con 0x7f8620102ee0 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86201129e0 con 0x7f8620102520 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43764/0 (socket says 172.21.15.87:43764) 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 -- 172.21.15.87:0/390917218 learned_addr learned my addr 172.21.15.87:0/390917218 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862617a700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102520 0x7f862019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f8625979700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8620102ee0 0x7f862019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 -- 172.21.15.87:0/390917218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102520 msgr2=0x7f862019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102520 0x7f862019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 -- 172.21.15.87:0/390917218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8620102ee0 msgr2=0x7f862019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8620102ee0 0x7f862019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 -- 172.21.15.87:0/390917218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862019fe80 con 0x7f86201085c0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f862697b700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8618007ee0 tx=0x7f861800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861800dbd0 con 0x7f86201085c0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861800dd70 con 0x7f86201085c0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618005a60 con 0x7f86201085c0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86201a0170 con 0x7f86201085c0 2024-06-06T07:46:50.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.360+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201a79f0 con 0x7f86201085c0 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.362+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8620069180 con 0x7f86201085c0 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.363+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f861801a020 con 0x7f86201085c0 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.363+0000 7f86177fe700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 0x7f860c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.363+0000 7f862617a700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 0x7f860c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.363+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f86180cead0 con 0x7f86201085c0 2024-06-06T07:46:50.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.364+0000 7f862617a700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 0x7f860c0a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8610005950 tx=0x7f861000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:50.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.368+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8618010070 con 0x7f86201085c0 2024-06-06T07:46:50.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.560+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8620076070 con 0x7f860c0a2ca0 2024-06-06T07:46:50.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.569+0000 7f86177fe700 1 -- 172.21.15.87:0/390917218 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340604 (secure 0 0 0) 0x7f8620076070 con 0x7f860c0a2ca0 2024-06-06T07:46:50.573 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 msgr2=0x7f860c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 0x7f860c0a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8610005950 tx=0x7f861000b410 comp rx=0 tx=0).stop 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 msgr2=0x7f862019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8618007ee0 tx=0x7f861800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 shutdown_connections 2024-06-06T07:46:50.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f860c0a2ca0 0x7f860c0a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201085c0 0x7f862019f740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620102520 0x7f862019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 --2- 172.21.15.87:0/390917218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8620102ee0 0x7f862019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 >> 172.21.15.87:0/390917218 conn(0x7f8620079680 msgr2=0x7f8620074950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 shutdown_connections 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:50.572+0000 7f86283de700 1 -- 172.21.15.87:0/390917218 wait complete. 2024-06-06T07:46:50.577 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:46:51.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:51 smithi087 ceph-mon[32276]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T07:46:51.490 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-06-06T07:46:50.183814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657593,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":786,"num_read_kb":511,"num_write":358,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":699,"ondisk_log_size":699,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389568,"kb_used_data":15436,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552256,"statfs":{"total":1151940427776,"available":1151541510144,"internally_reserved":0,"allocated":15806464,"data_stored":8788919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":33,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003567"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.167728+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:46:46.167728+0000","last_peered":"2024-06-06T07:46:46.167728+0000","last_clean":"2024-06-06T07:46:46.167728+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:46:46.167728+0000","last_undegraded":"2024-06-06T07:46:46.167728+0000","last_fullsized":"2024-06-06T07:46:46.167728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172518+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:46:46.172518+0000","last_peered":"2024-06-06T07:46:46.172518+0000","last_clean":"2024-06-06T07:46:46.172518+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:46:46.172518+0000","last_undegraded":"2024-06-06T07:46:46.172518+0000","last_fullsized":"2024-06-06T07:46:46.172518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.176562+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:46:46.176562+0000","last_peered":"2024-06-06T07:46:46.176562+0000","last_clean":"2024-06-06T07:46:46.176562+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:46:46.176562+0000","last_undegraded":"2024-06-06T07:46:46.176562+0000","last_fullsized":"2024-06-06T07:46:46.176562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.159459+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:46:46.159459+0000","last_peered":"2024-06-06T07:46:46.159459+0000","last_clean":"2024-06-06T07:46:46.159459+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:46:46.159459+0000","last_undegraded":"2024-06-06T07:46:46.159459+0000","last_fullsized":"2024-06-06T07:46:46.159459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.152216+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:46:46.152216+0000","last_peered":"2024-06-06T07:46:46.152216+0000","last_clean":"2024-06-06T07:46:46.152216+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:46:46.152216+0000","last_undegraded":"2024-06-06T07:46:46.152216+0000","last_fullsized":"2024-06-06T07:46:46.152216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169999+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:46:46.169999+0000","last_peered":"2024-06-06T07:46:46.169999+0000","last_clean":"2024-06-06T07:46:46.169999+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:46:46.169999+0000","last_undegraded":"2024-06-06T07:46:46.169999+0000","last_fullsized":"2024-06-06T07:46:46.169999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.175153+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:46:46.175153+0000","last_peered":"2024-06-06T07:46:46.175153+0000","last_clean":"2024-06-06T07:46:46.175153+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:46:46.175153+0000","last_undegraded":"2024-06-06T07:46:46.175153+0000","last_fullsized":"2024-06-06T07:46:46.175153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.165886+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:46:46.165886+0000","last_peered":"2024-06-06T07:46:46.165886+0000","last_clean":"2024-06-06T07:46:46.165886+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:46:46.165886+0000","last_undegraded":"2024-06-06T07:46:46.165886+0000","last_fullsized":"2024-06-06T07:46:46.165886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":26,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.151010+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:46:46.151010+0000","last_peered":"2024-06-06T07:46:46.151010+0000","last_clean":"2024-06-06T07:46:46.151010+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:46:46.151010+0000","last_undegraded":"2024-06-06T07:46:46.151010+0000","last_fullsized":"2024-06-06T07:46:46.151010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171681+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:46:46.171681+0000","last_peered":"2024-06-06T07:46:46.171681+0000","last_clean":"2024-06-06T07:46:46.171681+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:46:46.171681+0000","last_undegraded":"2024-06-06T07:46:46.171681+0000","last_fullsized":"2024-06-06T07:46:46.171681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.174610+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:46:46.174610+0000","last_peered":"2024-06-06T07:46:46.174610+0000","last_clean":"2024-06-06T07:46:46.174610+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:46:46.174610+0000","last_undegraded":"2024-06-06T07:46:46.174610+0000","last_fullsized":"2024-06-06T07:46:46.174610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.164197+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:46:46.164197+0000","last_peered":"2024-06-06T07:46:46.164197+0000","last_clean":"2024-06-06T07:46:46.164197+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:46:46.164197+0000","last_undegraded":"2024-06-06T07:46:46.164197+0000","last_fullsized":"2024-06-06T07:46:46.164197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228466+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:45:48.228466+0000","last_peered":"2024-06-06T07:45:48.228466+0000","last_clean":"2024-06-06T07:45:48.228466+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:45:48.228466+0000","last_undegraded":"2024-06-06T07:45:48.228466+0000","last_fullsized":"2024-06-06T07:45:48.228466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172484+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:46:46.172484+0000","last_peered":"2024-06-06T07:46:46.172484+0000","last_clean":"2024-06-06T07:46:46.172484+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:46:46.172484+0000","last_undegraded":"2024-06-06T07:46:46.172484+0000","last_fullsized":"2024-06-06T07:46:46.172484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168160+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:46:46.168160+0000","last_peered":"2024-06-06T07:46:46.168160+0000","last_clean":"2024-06-06T07:46:46.168160+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:46:46.168160+0000","last_undegraded":"2024-06-06T07:46:46.168160+0000","last_fullsized":"2024-06-06T07:46:46.168160+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.177855+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:46:46.177855+0000","last_peered":"2024-06-06T07:46:46.177855+0000","last_clean":"2024-06-06T07:46:46.177855+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:46:46.177855+0000","last_undegraded":"2024-06-06T07:46:46.177855+0000","last_fullsized":"2024-06-06T07:46:46.177855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168125+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:46:46.168125+0000","last_peered":"2024-06-06T07:46:46.168125+0000","last_clean":"2024-06-06T07:46:46.168125+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:46:46.168125+0000","last_undegraded":"2024-06-06T07:46:46.168125+0000","last_fullsized":"2024-06-06T07:46:46.168125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171661+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:46:46.171661+0000","last_peered":"2024-06-06T07:46:46.171661+0000","last_clean":"2024-06-06T07:46:46.171661+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:46:46.171661+0000","last_undegraded":"2024-06-06T07:46:46.171661+0000","last_fullsized":"2024-06-06T07:46:46.171661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230314+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:45:48.230314+0000","last_peered":"2024-06-06T07:45:48.230314+0000","last_clean":"2024-06-06T07:45:48.230314+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:45:48.230314+0000","last_undegraded":"2024-06-06T07:45:48.230314+0000","last_fullsized":"2024-06-06T07:45:48.230314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230313+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:45:48.230313+0000","last_peered":"2024-06-06T07:45:48.230313+0000","last_clean":"2024-06-06T07:45:48.230313+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:45:48.230313+0000","last_undegraded":"2024-06-06T07:45:48.230313+0000","last_fullsized":"2024-06-06T07:45:48.230313+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918847+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:45:48.918847+0000","last_peered":"2024-06-06T07:45:48.918847+0000","last_clean":"2024-06-06T07:45:48.918847+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:45:48.918847+0000","last_undegraded":"2024-06-06T07:45:48.918847+0000","last_fullsized":"2024-06-06T07:45:48.918847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.165253+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:46:46.165253+0000","last_peered":"2024-06-06T07:46:46.165253+0000","last_clean":"2024-06-06T07:46:46.165253+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:46:46.165253+0000","last_undegraded":"2024-06-06T07:46:46.165253+0000","last_fullsized":"2024-06-06T07:46:46.165253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231916+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:45:48.231916+0000","last_peered":"2024-06-06T07:45:48.231916+0000","last_clean":"2024-06-06T07:45:48.231916+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:45:48.231916+0000","last_undegraded":"2024-06-06T07:45:48.231916+0000","last_fullsized":"2024-06-06T07:45:48.231916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230636+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:45:48.230636+0000","last_peered":"2024-06-06T07:45:48.230636+0000","last_clean":"2024-06-06T07:45:48.230636+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:45:48.230636+0000","last_undegraded":"2024-06-06T07:45:48.230636+0000","last_fullsized":"2024-06-06T07:45:48.230636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.176492+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:46:46.176492+0000","last_peered":"2024-06-06T07:46:46.176492+0000","last_clean":"2024-06-06T07:46:46.176492+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:46:46.176492+0000","last_undegraded":"2024-06-06T07:46:46.176492+0000","last_fullsized":"2024-06-06T07:46:46.176492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230476+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:45:48.230476+0000","last_peered":"2024-06-06T07:45:48.230476+0000","last_clean":"2024-06-06T07:45:48.230476+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:45:48.230476+0000","last_undegraded":"2024-06-06T07:45:48.230476+0000","last_fullsized":"2024-06-06T07:45:48.230476+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227304+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:45:48.227304+0000","last_peered":"2024-06-06T07:45:48.227304+0000","last_clean":"2024-06-06T07:45:48.227304+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:45:48.227304+0000","last_undegraded":"2024-06-06T07:45:48.227304+0000","last_fullsized":"2024-06-06T07:45:48.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.178128+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:46:46.178128+0000","last_peered":"2024-06-06T07:46:46.178128+0000","last_clean":"2024-06-06T07:46:46.178128+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:46:46.178128+0000","last_undegraded":"2024-06-06T07:46:46.178128+0000","last_fullsized":"2024-06-06T07:46:46.178128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230296+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:45:48.230296+0000","last_peered":"2024-06-06T07:45:48.230296+0000","last_clean":"2024-06-06T07:45:48.230296+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:45:48.230296+0000","last_undegraded":"2024-06-06T07:45:48.230296+0000","last_fullsized":"2024-06-06T07:45:48.230296+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226641+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:45:48.226641+0000","last_peered":"2024-06-06T07:45:48.226641+0000","last_clean":"2024-06-06T07:45:48.226641+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:45:48.226641+0000","last_undegraded":"2024-06-06T07:45:48.226641+0000","last_fullsized":"2024-06-06T07:45:48.226641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.170434+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:46:46.170434+0000","last_peered":"2024-06-06T07:46:46.170434+0000","last_clean":"2024-06-06T07:46:46.170434+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:46:46.170434+0000","last_undegraded":"2024-06-06T07:46:46.170434+0000","last_fullsized":"2024-06-06T07:46:46.170434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230186+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:45:48.230186+0000","last_peered":"2024-06-06T07:45:48.230186+0000","last_clean":"2024-06-06T07:45:48.230186+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:45:48.230186+0000","last_undegraded":"2024-06-06T07:45:48.230186+0000","last_fullsized":"2024-06-06T07:45:48.230186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:47.395730+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:46:47.395730+0000","last_peered":"2024-06-06T07:46:47.395730+0000","last_clean":"2024-06-06T07:46:47.395730+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:46:47.395730+0000","last_undegraded":"2024-06-06T07:46:47.395730+0000","last_fullsized":"2024-06-06T07:46:47.395730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.156518+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:46:46.156518+0000","last_peered":"2024-06-06T07:46:46.156518+0000","last_clean":"2024-06-06T07:46:46.156518+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:46:46.156518+0000","last_undegraded":"2024-06-06T07:46:46.156518+0000","last_fullsized":"2024-06-06T07:46:46.156518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230472+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:45:48.230472+0000","last_peered":"2024-06-06T07:45:48.230472+0000","last_clean":"2024-06-06T07:45:48.230472+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:45:48.230472+0000","last_undegraded":"2024-06-06T07:45:48.230472+0000","last_fullsized":"2024-06-06T07:45:48.230472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229873+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:45:48.229873+0000","last_peered":"2024-06-06T07:45:48.229873+0000","last_clean":"2024-06-06T07:45:48.229873+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:45:48.229873+0000","last_undegraded":"2024-06-06T07:45:48.229873+0000","last_fullsized":"2024-06-06T07:45:48.229873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.166179+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:46:46.166179+0000","last_peered":"2024-06-06T07:46:46.166179+0000","last_clean":"2024-06-06T07:46:46.166179+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:46:46.166179+0000","last_undegraded":"2024-06-06T07:46:46.166179+0000","last_fullsized":"2024-06-06T07:46:46.166179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228208+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:45:48.228208+0000","last_peered":"2024-06-06T07:45:48.228208+0000","last_clean":"2024-06-06T07:45:48.228208+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:45:48.228208+0000","last_undegraded":"2024-06-06T07:45:48.228208+0000","last_fullsized":"2024-06-06T07:45:48.228208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228227+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:45:48.228227+0000","last_peered":"2024-06-06T07:45:48.228227+0000","last_clean":"2024-06-06T07:45:48.228227+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:45:48.228227+0000","last_undegraded":"2024-06-06T07:45:48.228227+0000","last_fullsized":"2024-06-06T07:45:48.228227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169838+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:46:46.169838+0000","last_peered":"2024-06-06T07:46:46.169838+0000","last_clean":"2024-06-06T07:46:46.169838+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:46:46.169838+0000","last_undegraded":"2024-06-06T07:46:46.169838+0000","last_fullsized":"2024-06-06T07:46:46.169838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225389+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:45:48.225389+0000","last_peered":"2024-06-06T07:45:48.225389+0000","last_clean":"2024-06-06T07:45:48.225389+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:45:48.225389+0000","last_undegraded":"2024-06-06T07:45:48.225389+0000","last_fullsized":"2024-06-06T07:45:48.225389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231868+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:45:48.231868+0000","last_peered":"2024-06-06T07:45:48.231868+0000","last_clean":"2024-06-06T07:45:48.231868+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:45:48.231868+0000","last_undegraded":"2024-06-06T07:45:48.231868+0000","last_fullsized":"2024-06-06T07:45:48.231868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.159086+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:46:46.159086+0000","last_peered":"2024-06-06T07:46:46.159086+0000","last_clean":"2024-06-06T07:46:46.159086+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:46:46.159086+0000","last_undegraded":"2024-06-06T07:46:46.159086+0000","last_fullsized":"2024-06-06T07:46:46.159086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230696+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:45:48.230696+0000","last_peered":"2024-06-06T07:45:48.230696+0000","last_clean":"2024-06-06T07:45:48.230696+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:45:48.230696+0000","last_undegraded":"2024-06-06T07:45:48.230696+0000","last_fullsized":"2024-06-06T07:45:48.230696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":151,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.122832+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:46:46.122832+0000","last_peered":"2024-06-06T07:46:46.122832+0000","last_clean":"2024-06-06T07:46:46.122832+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:46:46.122832+0000","last_undegraded":"2024-06-06T07:46:46.122832+0000","last_fullsized":"2024-06-06T07:46:46.122832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.175777+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:46:46.175777+0000","last_peered":"2024-06-06T07:46:46.175777+0000","last_clean":"2024-06-06T07:46:46.175777+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:46:46.175777+0000","last_undegraded":"2024-06-06T07:46:46.175777+0000","last_fullsized":"2024-06-06T07:46:46.175777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226070+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:45:48.226070+0000","last_peered":"2024-06-06T07:45:48.226070+0000","last_clean":"2024-06-06T07:45:48.226070+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:45:48.226070+0000","last_undegraded":"2024-06-06T07:45:48.226070+0000","last_fullsized":"2024-06-06T07:45:48.226070+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227073+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:45:48.227073+0000","last_peered":"2024-06-06T07:45:48.227073+0000","last_clean":"2024-06-06T07:45:48.227073+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:45:48.227073+0000","last_undegraded":"2024-06-06T07:45:48.227073+0000","last_fullsized":"2024-06-06T07:45:48.227073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168649+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:46:46.168649+0000","last_peered":"2024-06-06T07:46:46.168649+0000","last_clean":"2024-06-06T07:46:46.168649+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:46:46.168649+0000","last_undegraded":"2024-06-06T07:46:46.168649+0000","last_fullsized":"2024-06-06T07:46:46.168649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230814+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:45:48.230814+0000","last_peered":"2024-06-06T07:45:48.230814+0000","last_clean":"2024-06-06T07:45:48.230814+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:45:48.230814+0000","last_undegraded":"2024-06-06T07:45:48.230814+0000","last_fullsized":"2024-06-06T07:45:48.230814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232774+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:45:48.232774+0000","last_peered":"2024-06-06T07:45:48.232774+0000","last_clean":"2024-06-06T07:45:48.232774+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:45:48.232774+0000","last_undegraded":"2024-06-06T07:45:48.232774+0000","last_fullsized":"2024-06-06T07:45:48.232774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.178185+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:46:46.178185+0000","last_peered":"2024-06-06T07:46:46.178185+0000","last_clean":"2024-06-06T07:46:46.178185+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:46:46.178185+0000","last_undegraded":"2024-06-06T07:46:46.178185+0000","last_fullsized":"2024-06-06T07:46:46.178185+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.917184+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:45:48.917184+0000","last_peered":"2024-06-06T07:45:48.917184+0000","last_clean":"2024-06-06T07:45:48.917184+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:45:48.917184+0000","last_undegraded":"2024-06-06T07:45:48.917184+0000","last_fullsized":"2024-06-06T07:45:48.917184+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":296,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.393724+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:46:46.393724+0000","last_peered":"2024-06-06T07:46:46.393724+0000","last_clean":"2024-06-06T07:46:46.393724+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:46:46.393724+0000","last_undegraded":"2024-06-06T07:46:46.393724+0000","last_fullsized":"2024-06-06T07:46:46.393724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62319,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":216,"num_read_kb":169,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172717+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:46:46.172717+0000","last_peered":"2024-06-06T07:46:46.172717+0000","last_clean":"2024-06-06T07:46:46.172717+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:46:46.172717+0000","last_undegraded":"2024-06-06T07:46:46.172717+0000","last_fullsized":"2024-06-06T07:46:46.172717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233667+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:45:48.233667+0000","last_peered":"2024-06-06T07:45:48.233667+0000","last_clean":"2024-06-06T07:45:48.233667+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:45:48.233667+0000","last_undegraded":"2024-06-06T07:45:48.233667+0000","last_fullsized":"2024-06-06T07:45:48.233667+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229897+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:45:48.229897+0000","last_peered":"2024-06-06T07:45:48.229897+0000","last_clean":"2024-06-06T07:45:48.229897+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:45:48.229897+0000","last_undegraded":"2024-06-06T07:45:48.229897+0000","last_fullsized":"2024-06-06T07:45:48.229897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228513+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:45:48.228513+0000","last_peered":"2024-06-06T07:45:48.228513+0000","last_clean":"2024-06-06T07:45:48.228513+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:45:48.228513+0000","last_undegraded":"2024-06-06T07:45:48.228513+0000","last_fullsized":"2024-06-06T07:45:48.228513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227501+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:45:48.227501+0000","last_peered":"2024-06-06T07:45:48.227501+0000","last_clean":"2024-06-06T07:45:48.227501+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:45:48.227501+0000","last_undegraded":"2024-06-06T07:45:48.227501+0000","last_fullsized":"2024-06-06T07:45:48.227501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232326+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:45:48.232326+0000","last_peered":"2024-06-06T07:45:48.232326+0000","last_clean":"2024-06-06T07:45:48.232326+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:45:48.232326+0000","last_undegraded":"2024-06-06T07:45:48.232326+0000","last_fullsized":"2024-06-06T07:45:48.232326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.177675+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:46:46.177675+0000","last_peered":"2024-06-06T07:46:46.177675+0000","last_clean":"2024-06-06T07:46:46.177675+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:46:46.177675+0000","last_undegraded":"2024-06-06T07:46:46.177675+0000","last_fullsized":"2024-06-06T07:46:46.177675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.919117+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:45:48.919117+0000","last_peered":"2024-06-06T07:45:48.919117+0000","last_clean":"2024-06-06T07:45:48.919117+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:45:48.919117+0000","last_undegraded":"2024-06-06T07:45:48.919117+0000","last_fullsized":"2024-06-06T07:45:48.919117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'5","reported_seq":159,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:48.434555+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:46:48.434555+0000","last_peered":"2024-06-06T07:46:48.434555+0000","last_clean":"2024-06-06T07:46:48.434555+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:46:48.434555+0000","last_undegraded":"2024-06-06T07:46:48.434555+0000","last_fullsized":"2024-06-06T07:46:48.434555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.175310+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:46:46.175310+0000","last_peered":"2024-06-06T07:46:46.175310+0000","last_clean":"2024-06-06T07:46:46.175310+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:46:46.175310+0000","last_undegraded":"2024-06-06T07:46:46.175310+0000","last_fullsized":"2024-06-06T07:46:46.175310+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228891+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:45:48.228891+0000","last_peered":"2024-06-06T07:45:48.228891+0000","last_clean":"2024-06-06T07:45:48.228891+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:45:48.228891+0000","last_undegraded":"2024-06-06T07:45:48.228891+0000","last_fullsized":"2024-06-06T07:45:48.228891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.248883+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:45:48.248883+0000","last_peered":"2024-06-06T07:45:48.248883+0000","last_clean":"2024-06-06T07:45:48.248883+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:45:48.248883+0000","last_undegraded":"2024-06-06T07:45:48.248883+0000","last_fullsized":"2024-06-06T07:45:48.248883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.165669+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:46:46.165669+0000","last_peered":"2024-06-06T07:46:46.165669+0000","last_clean":"2024-06-06T07:46:46.165669+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:46:46.165669+0000","last_undegraded":"2024-06-06T07:46:46.165669+0000","last_fullsized":"2024-06-06T07:46:46.165669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231768+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:45:48.231768+0000","last_peered":"2024-06-06T07:45:48.231768+0000","last_clean":"2024-06-06T07:45:48.231768+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:45:48.231768+0000","last_undegraded":"2024-06-06T07:45:48.231768+0000","last_fullsized":"2024-06-06T07:45:48.231768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.252519+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:45:48.252519+0000","last_peered":"2024-06-06T07:45:48.252519+0000","last_clean":"2024-06-06T07:45:48.252519+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:45:48.252519+0000","last_undegraded":"2024-06-06T07:45:48.252519+0000","last_fullsized":"2024-06-06T07:45:48.252519+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171950+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:46:46.171950+0000","last_peered":"2024-06-06T07:46:46.171950+0000","last_clean":"2024-06-06T07:46:46.171950+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:46:46.171950+0000","last_undegraded":"2024-06-06T07:46:46.171950+0000","last_fullsized":"2024-06-06T07:46:46.171950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226314+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:45:48.226314+0000","last_peered":"2024-06-06T07:45:48.226314+0000","last_clean":"2024-06-06T07:45:48.226314+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:45:48.226314+0000","last_undegraded":"2024-06-06T07:45:48.226314+0000","last_fullsized":"2024-06-06T07:45:48.226314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.917806+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:45:48.917806+0000","last_peered":"2024-06-06T07:45:48.917806+0000","last_clean":"2024-06-06T07:45:48.917806+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:45:48.917806+0000","last_undegraded":"2024-06-06T07:45:48.917806+0000","last_fullsized":"2024-06-06T07:45:48.917806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.157924+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:46:46.157924+0000","last_peered":"2024-06-06T07:46:46.157924+0000","last_clean":"2024-06-06T07:46:46.157924+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:46:46.157924+0000","last_undegraded":"2024-06-06T07:46:46.157924+0000","last_fullsized":"2024-06-06T07:46:46.157924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228576+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:45:48.228576+0000","last_peered":"2024-06-06T07:45:48.228576+0000","last_clean":"2024-06-06T07:45:48.228576+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:45:48.228576+0000","last_undegraded":"2024-06-06T07:45:48.228576+0000","last_fullsized":"2024-06-06T07:45:48.228576+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226836+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:45:48.226836+0000","last_peered":"2024-06-06T07:45:48.226836+0000","last_clean":"2024-06-06T07:45:48.226836+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:45:48.226836+0000","last_undegraded":"2024-06-06T07:45:48.226836+0000","last_fullsized":"2024-06-06T07:45:48.226836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168437+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:46:46.168437+0000","last_peered":"2024-06-06T07:46:46.168437+0000","last_clean":"2024-06-06T07:46:46.168437+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:46:46.168437+0000","last_undegraded":"2024-06-06T07:46:46.168437+0000","last_fullsized":"2024-06-06T07:46:46.168437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228351+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:45:48.228351+0000","last_peered":"2024-06-06T07:45:48.228351+0000","last_clean":"2024-06-06T07:45:48.228351+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:45:48.228351+0000","last_undegraded":"2024-06-06T07:45:48.228351+0000","last_fullsized":"2024-06-06T07:45:48.228351+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.165384+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:46:46.165384+0000","last_peered":"2024-06-06T07:46:46.165384+0000","last_clean":"2024-06-06T07:46:46.165384+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:46:46.165384+0000","last_undegraded":"2024-06-06T07:46:46.165384+0000","last_fullsized":"2024-06-06T07:46:46.165384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:43.424826+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:46:43.424826+0000","last_peered":"2024-06-06T07:46:43.424826+0000","last_clean":"2024-06-06T07:46:43.424826+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:46:43.424826+0000","last_undegraded":"2024-06-06T07:46:43.424826+0000","last_fullsized":"2024-06-06T07:46:43.424826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227786+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:45:48.227786+0000","last_peered":"2024-06-06T07:45:48.227786+0000","last_clean":"2024-06-06T07:45:48.227786+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:45:48.227786+0000","last_undegraded":"2024-06-06T07:45:48.227786+0000","last_fullsized":"2024-06-06T07:45:48.227786+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171617+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:46:46.171617+0000","last_peered":"2024-06-06T07:46:46.171617+0000","last_clean":"2024-06-06T07:46:46.171617+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:46:46.171617+0000","last_undegraded":"2024-06-06T07:46:46.171617+0000","last_fullsized":"2024-06-06T07:46:46.171617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230682+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:45:48.230682+0000","last_peered":"2024-06-06T07:45:48.230682+0000","last_clean":"2024-06-06T07:45:48.230682+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:45:48.230682+0000","last_undegraded":"2024-06-06T07:45:48.230682+0000","last_fullsized":"2024-06-06T07:45:48.230682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":379,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226354+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:45:48.226354+0000","last_peered":"2024-06-06T07:45:48.226354+0000","last_clean":"2024-06-06T07:45:48.226354+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:45:48.226354+0000","last_undegraded":"2024-06-06T07:45:48.226354+0000","last_fullsized":"2024-06-06T07:45:48.226354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233325+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:45:48.233325+0000","last_peered":"2024-06-06T07:45:48.233325+0000","last_clean":"2024-06-06T07:45:48.233325+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:45:48.233325+0000","last_undegraded":"2024-06-06T07:45:48.233325+0000","last_fullsized":"2024-06-06T07:45:48.233325+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227945+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:45:48.227945+0000","last_peered":"2024-06-06T07:45:48.227945+0000","last_clean":"2024-06-06T07:45:48.227945+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:45:48.227945+0000","last_undegraded":"2024-06-06T07:45:48.227945+0000","last_fullsized":"2024-06-06T07:45:48.227945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226537+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:45:48.226537+0000","last_peered":"2024-06-06T07:45:48.226537+0000","last_clean":"2024-06-06T07:45:48.226537+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:45:48.226537+0000","last_undegraded":"2024-06-06T07:45:48.226537+0000","last_fullsized":"2024-06-06T07:45:48.226537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227366+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:45:48.227366+0000","last_peered":"2024-06-06T07:45:48.227366+0000","last_clean":"2024-06-06T07:45:48.227366+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:45:48.227366+0000","last_undegraded":"2024-06-06T07:45:48.227366+0000","last_fullsized":"2024-06-06T07:45:48.227366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226961+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:45:48.226961+0000","last_peered":"2024-06-06T07:45:48.226961+0000","last_clean":"2024-06-06T07:45:48.226961+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:45:48.226961+0000","last_undegraded":"2024-06-06T07:45:48.226961+0000","last_fullsized":"2024-06-06T07:45:48.226961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228377+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:45:48.228377+0000","last_peered":"2024-06-06T07:45:48.228377+0000","last_clean":"2024-06-06T07:45:48.228377+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:45:48.228377+0000","last_undegraded":"2024-06-06T07:45:48.228377+0000","last_fullsized":"2024-06-06T07:45:48.228377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230094+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:45:48.230094+0000","last_peered":"2024-06-06T07:45:48.230094+0000","last_clean":"2024-06-06T07:45:48.230094+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:45:48.230094+0000","last_undegraded":"2024-06-06T07:45:48.230094+0000","last_fullsized":"2024-06-06T07:45:48.230094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'11","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918407+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:45:48.918407+0000","last_peered":"2024-06-06T07:45:48.918407+0000","last_clean":"2024-06-06T07:45:48.918407+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:45:48.918407+0000","last_undegraded":"2024-06-06T07:45:48.918407+0000","last_fullsized":"2024-06-06T07:45:48.918407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169795+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:46:46.169795+0000","last_peered":"2024-06-06T07:46:46.169795+0000","last_clean":"2024-06-06T07:46:46.169795+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:46:46.169795+0000","last_undegraded":"2024-06-06T07:46:46.169795+0000","last_fullsized":"2024-06-06T07:46:46.169795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230069+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:45:48.230069+0000","last_peered":"2024-06-06T07:45:48.230069+0000","last_clean":"2024-06-06T07:45:48.230069+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:45:48.230069+0000","last_undegraded":"2024-06-06T07:45:48.230069+0000","last_fullsized":"2024-06-06T07:45:48.230069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233142+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:45:48.233142+0000","last_peered":"2024-06-06T07:45:48.233142+0000","last_clean":"2024-06-06T07:45:48.233142+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:45:48.233142+0000","last_undegraded":"2024-06-06T07:45:48.233142+0000","last_fullsized":"2024-06-06T07:45:48.233142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.177251+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:46:46.177251+0000","last_peered":"2024-06-06T07:46:46.177251+0000","last_clean":"2024-06-06T07:46:46.177251+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:46:46.177251+0000","last_undegraded":"2024-06-06T07:46:46.177251+0000","last_fullsized":"2024-06-06T07:46:46.177251+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232360+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:45:48.232360+0000","last_peered":"2024-06-06T07:45:48.232360+0000","last_clean":"2024-06-06T07:45:48.232360+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:45:48.232360+0000","last_undegraded":"2024-06-06T07:45:48.232360+0000","last_fullsized":"2024-06-06T07:45:48.232360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228080+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:45:48.228080+0000","last_peered":"2024-06-06T07:45:48.228080+0000","last_clean":"2024-06-06T07:45:48.228080+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:45:48.228080+0000","last_undegraded":"2024-06-06T07:45:48.228080+0000","last_fullsized":"2024-06-06T07:45:48.228080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.130829+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:46:46.130829+0000","last_peered":"2024-06-06T07:46:46.130829+0000","last_clean":"2024-06-06T07:46:46.130829+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:46:46.130829+0000","last_undegraded":"2024-06-06T07:46:46.130829+0000","last_fullsized":"2024-06-06T07:46:46.130829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.918950+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:45:48.918950+0000","last_peered":"2024-06-06T07:45:48.918950+0000","last_clean":"2024-06-06T07:45:48.918950+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:45:48.918950+0000","last_undegraded":"2024-06-06T07:45:48.918950+0000","last_fullsized":"2024-06-06T07:45:48.918950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230785+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:45:48.230785+0000","last_peered":"2024-06-06T07:45:48.230785+0000","last_clean":"2024-06-06T07:45:48.230785+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:45:48.230785+0000","last_undegraded":"2024-06-06T07:45:48.230785+0000","last_fullsized":"2024-06-06T07:45:48.230785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171863+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:46:46.171863+0000","last_peered":"2024-06-06T07:46:46.171863+0000","last_clean":"2024-06-06T07:46:46.171863+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:46:46.171863+0000","last_undegraded":"2024-06-06T07:46:46.171863+0000","last_fullsized":"2024-06-06T07:46:46.171863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232384+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:45:48.232384+0000","last_peered":"2024-06-06T07:45:48.232384+0000","last_clean":"2024-06-06T07:45:48.232384+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:45:48.232384+0000","last_undegraded":"2024-06-06T07:45:48.232384+0000","last_fullsized":"2024-06-06T07:45:48.232384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229794+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:45:48.229794+0000","last_peered":"2024-06-06T07:45:48.229794+0000","last_clean":"2024-06-06T07:45:48.229794+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:45:48.229794+0000","last_undegraded":"2024-06-06T07:45:48.229794+0000","last_fullsized":"2024-06-06T07:45:48.229794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.159189+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:46:46.159189+0000","last_peered":"2024-06-06T07:46:46.159189+0000","last_clean":"2024-06-06T07:46:46.159189+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:46:46.159189+0000","last_undegraded":"2024-06-06T07:46:46.159189+0000","last_fullsized":"2024-06-06T07:46:46.159189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232360+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:45:48.232360+0000","last_peered":"2024-06-06T07:45:48.232360+0000","last_clean":"2024-06-06T07:45:48.232360+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:45:48.232360+0000","last_undegraded":"2024-06-06T07:45:48.232360+0000","last_fullsized":"2024-06-06T07:45:48.232360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.226643+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:45:48.226643+0000","last_peered":"2024-06-06T07:45:48.226643+0000","last_clean":"2024-06-06T07:45:48.226643+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:45:48.226643+0000","last_undegraded":"2024-06-06T07:45:48.226643+0000","last_fullsized":"2024-06-06T07:45:48.226643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.176511+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:46:46.176511+0000","last_peered":"2024-06-06T07:46:46.176511+0000","last_clean":"2024-06-06T07:46:46.176511+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:46:46.176511+0000","last_undegraded":"2024-06-06T07:46:46.176511+0000","last_fullsized":"2024-06-06T07:46:46.176511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227636+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:45:48.227636+0000","last_peered":"2024-06-06T07:45:48.227636+0000","last_clean":"2024-06-06T07:45:48.227636+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:45:48.227636+0000","last_undegraded":"2024-06-06T07:45:48.227636+0000","last_fullsized":"2024-06-06T07:45:48.227636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.237168+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:45:48.237168+0000","last_peered":"2024-06-06T07:45:48.237168+0000","last_clean":"2024-06-06T07:45:48.237168+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:45:48.237168+0000","last_undegraded":"2024-06-06T07:45:48.237168+0000","last_fullsized":"2024-06-06T07:45:48.237168+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.175658+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:46:46.175658+0000","last_peered":"2024-06-06T07:46:46.175658+0000","last_clean":"2024-06-06T07:46:46.175658+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:46:46.175658+0000","last_undegraded":"2024-06-06T07:46:46.175658+0000","last_fullsized":"2024-06-06T07:46:46.175658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.225124+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:45:48.225124+0000","last_peered":"2024-06-06T07:45:48.225124+0000","last_clean":"2024-06-06T07:45:48.225124+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:45:48.225124+0000","last_undegraded":"2024-06-06T07:45:48.225124+0000","last_fullsized":"2024-06-06T07:45:48.225124+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.233008+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:45:48.233008+0000","last_peered":"2024-06-06T07:45:48.233008+0000","last_clean":"2024-06-06T07:45:48.233008+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:45:48.233008+0000","last_undegraded":"2024-06-06T07:45:48.233008+0000","last_fullsized":"2024-06-06T07:45:48.233008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168017+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:46:46.168017+0000","last_peered":"2024-06-06T07:46:46.168017+0000","last_clean":"2024-06-06T07:46:46.168017+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:46:46.168017+0000","last_undegraded":"2024-06-06T07:46:46.168017+0000","last_fullsized":"2024-06-06T07:46:46.168017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.231333+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:45:48.231333+0000","last_peered":"2024-06-06T07:45:48.231333+0000","last_clean":"2024-06-06T07:45:48.231333+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:45:48.231333+0000","last_undegraded":"2024-06-06T07:45:48.231333+0000","last_fullsized":"2024-06-06T07:45:48.231333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.235572+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:45:48.235572+0000","last_peered":"2024-06-06T07:45:48.235572+0000","last_clean":"2024-06-06T07:45:48.235572+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:45:48.235572+0000","last_undegraded":"2024-06-06T07:45:48.235572+0000","last_fullsized":"2024-06-06T07:45:48.235572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.175753+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:46:46.175753+0000","last_peered":"2024-06-06T07:46:46.175753+0000","last_clean":"2024-06-06T07:46:46.175753+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:46:46.175753+0000","last_undegraded":"2024-06-06T07:46:46.175753+0000","last_fullsized":"2024-06-06T07:46:46.175753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172843+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:46:46.172843+0000","last_peered":"2024-06-06T07:46:46.172843+0000","last_clean":"2024-06-06T07:46:46.172843+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:46:46.172843+0000","last_undegraded":"2024-06-06T07:46:46.172843+0000","last_fullsized":"2024-06-06T07:46:46.172843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.230518+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:45:48.230518+0000","last_peered":"2024-06-06T07:45:48.230518+0000","last_clean":"2024-06-06T07:45:48.230518+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:45:48.230518+0000","last_undegraded":"2024-06-06T07:45:48.230518+0000","last_fullsized":"2024-06-06T07:45:48.230518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.229982+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:45:48.229982+0000","last_peered":"2024-06-06T07:45:48.229982+0000","last_clean":"2024-06-06T07:45:48.229982+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:45:48.229982+0000","last_undegraded":"2024-06-06T07:45:48.229982+0000","last_fullsized":"2024-06-06T07:45:48.229982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.162725+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:46:46.162725+0000","last_peered":"2024-06-06T07:46:46.162725+0000","last_clean":"2024-06-06T07:46:46.162725+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:46:46.162725+0000","last_undegraded":"2024-06-06T07:46:46.162725+0000","last_fullsized":"2024-06-06T07:46:46.162725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228158+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:45:48.228158+0000","last_peered":"2024-06-06T07:45:48.228158+0000","last_clean":"2024-06-06T07:45:48.228158+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:45:48.228158+0000","last_undegraded":"2024-06-06T07:45:48.228158+0000","last_fullsized":"2024-06-06T07:45:48.228158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.170081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:46:46.170081+0000","last_peered":"2024-06-06T07:46:46.170081+0000","last_clean":"2024-06-06T07:46:46.170081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:46:46.170081+0000","last_undegraded":"2024-06-06T07:46:46.170081+0000","last_fullsized":"2024-06-06T07:46:46.170081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.232169+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:45:48.232169+0000","last_peered":"2024-06-06T07:45:48.232169+0000","last_clean":"2024-06-06T07:45:48.232169+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:45:48.232169+0000","last_undegraded":"2024-06-06T07:45:48.232169+0000","last_fullsized":"2024-06-06T07:45:48.232169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.159248+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:46:46.159248+0000","last_peered":"2024-06-06T07:46:46.159248+0000","last_clean":"2024-06-06T07:46:46.159248+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:46:46.159248+0000","last_undegraded":"2024-06-06T07:46:46.159248+0000","last_fullsized":"2024-06-06T07:46:46.159248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.164435+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:46:46.164435+0000","last_peered":"2024-06-06T07:46:46.164435+0000","last_clean":"2024-06-06T07:46:46.164435+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:46:46.164435+0000","last_undegraded":"2024-06-06T07:46:46.164435+0000","last_fullsized":"2024-06-06T07:46:46.164435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169810+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:46:46.169810+0000","last_peered":"2024-06-06T07:46:46.169810+0000","last_clean":"2024-06-06T07:46:46.169810+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:46:46.169810+0000","last_undegraded":"2024-06-06T07:46:46.169810+0000","last_fullsized":"2024-06-06T07:46:46.169810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.177388+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:46:46.177388+0000","last_peered":"2024-06-06T07:46:46.177388+0000","last_clean":"2024-06-06T07:46:46.177388+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:46:46.177388+0000","last_undegraded":"2024-06-06T07:46:46.177388+0000","last_fullsized":"2024-06-06T07:46:46.177388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228406+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:45:48.228406+0000","last_peered":"2024-06-06T07:45:48.228406+0000","last_clean":"2024-06-06T07:45:48.228406+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:45:48.228406+0000","last_undegraded":"2024-06-06T07:45:48.228406+0000","last_fullsized":"2024-06-06T07:45:48.228406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169888+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:46:46.169888+0000","last_peered":"2024-06-06T07:46:46.169888+0000","last_clean":"2024-06-06T07:46:46.169888+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:46:46.169888+0000","last_undegraded":"2024-06-06T07:46:46.169888+0000","last_fullsized":"2024-06-06T07:46:46.169888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":410,"ondisk_log_size":410,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66991,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":650,"num_read_kb":392,"num_write":175,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383730,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":570184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645367,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28808,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716344,"statfs":{"total":95995035648,"available":95965536256,"internally_reserved":0,"allocated":1835008,"data_stored":1224264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":570286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070529,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":570072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32884,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712268,"statfs":{"total":95995035648,"available":95961362432,"internally_reserved":0,"allocated":1748992,"data_stored":1166786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":565462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":628273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":566850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40696,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704456,"statfs":{"total":95995035648,"available":95953362944,"internally_reserved":0,"allocated":1220608,"data_stored":628600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":565656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":570643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37052,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708100,"statfs":{"total":95995035648,"available":95957094400,"internally_reserved":0,"allocated":1757184,"data_stored":1161843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:46:51.495 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T07:46:51.495 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-06T07:46:51.495 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-06T07:46:51.495 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-06T07:46:51.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:51 smithi195 ceph-mon[37040]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T07:46:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:51 smithi181 ceph-mon[36790]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T07:46:51.738 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:52 smithi195 ceph-mon[37040]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:52 smithi181 ceph-mon[36790]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:52.587 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:52 smithi087 ceph-mon[32276]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:46:53.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:53 smithi087 ceph-mon[32276]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:53.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:53 smithi195 ceph-mon[37040]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:53 smithi181 ceph-mon[36790]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 -- 172.21.15.87:0/3305480500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c102030 msgr2=0x7fe03c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 --2- 172.21.15.87:0/3305480500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c102030 0x7fe03c105ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe0300099c0 tx=0x7fe030009cd0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 -- 172.21.15.87:0/3305480500 shutdown_connections 2024-06-06T07:46:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 --2- 172.21.15.87:0/3305480500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c1066d0 0x7fe03c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 --2- 172.21.15.87:0/3305480500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c101680 0x7fe03c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 --2- 172.21.15.87:0/3305480500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c102030 0x7fe03c105ff0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 -- 172.21.15.87:0/3305480500 >> 172.21.15.87:0/3305480500 conn(0x7fe03c0773e0 msgr2=0x7fe03c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 -- 172.21.15.87:0/3305480500 shutdown_connections 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.595+0000 7fe043b08700 1 -- 172.21.15.87:0/3305480500 wait complete. 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 Processor -- start 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 -- start start 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 0x7fe03c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c1066d0 0x7fe03c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1116b0 con 0x7fe03c101680 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c111530 con 0x7fe03c102030 2024-06-06T07:46:53.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe043b08700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c111830 con 0x7fe03c1066d0 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43778/0 (socket says 172.21.15.87:43778) 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 -- 172.21.15.87:0/358752074 learned_addr learned my addr 172.21.15.87:0/358752074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0410a3700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 0x7fe03c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0420a5700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c1066d0 0x7fe03c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 -- 172.21.15.87:0/358752074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c1066d0 msgr2=0x7fe03c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c1066d0 0x7fe03c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 -- 172.21.15.87:0/358752074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 msgr2=0x7fe03c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 0x7fe03c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0418a4700 1 -- 172.21.15.87:0/358752074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c1a5170 con 0x7fe03c101680 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.596+0000 7fe0410a3700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 0x7fe03c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe0418a4700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe02c007f10 tx=0x7fe02c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00dc90 con 0x7fe03c101680 2024-06-06T07:46:53.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c1a5340 con 0x7fe03c101680 2024-06-06T07:46:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02c00de30 con 0x7fe03c101680 2024-06-06T07:46:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0133f0 con 0x7fe03c101680 2024-06-06T07:46:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.597+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a5850 con 0x7fe03c101680 2024-06-06T07:46:53.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.598+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03c069180 con 0x7fe03c101680 2024-06-06T07:46:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.599+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe02c01a020 con 0x7fe03c101680 2024-06-06T07:46:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.600+0000 7fe03affd700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 0x7fe0240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.600+0000 7fe0410a3700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 0x7fe0240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.600+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fe02c0cf3e0 con 0x7fe03c101680 2024-06-06T07:46:53.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.601+0000 7fe0410a3700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 0x7fe0240a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe030000c00 tx=0x7fe030005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:53.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.604+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe02c010070 con 0x7fe03c101680 2024-06-06T07:46:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.846+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe03c02dca0 con 0x7fe03c101680 2024-06-06T07:46:53.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.846+0000 7fe03affd700 1 -- 172.21.15.87:0/358752074 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe02c09bef0 con 0x7fe03c101680 2024-06-06T07:46:53.846 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:46:53.846 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-06T07:46:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 msgr2=0x7fe0240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:53.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 0x7fe0240a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe030000c00 tx=0x7fe030005fd0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 msgr2=0x7fe03c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe02c007f10 tx=0x7fe02c00d3b0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 shutdown_connections 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fe0240a2c20 0x7fe0240a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe03c1066d0 0x7fe03c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe03c101680 0x7fe03c199bb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 --2- 172.21.15.87:0/358752074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c102030 0x7fe03c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 >> 172.21.15.87:0/358752074 conn(0x7fe03c0773e0 msgr2=0x7fe03c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 shutdown_connections 2024-06-06T07:46:53.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:53.848+0000 7fe043b08700 1 -- 172.21.15.87:0/358752074 wait complete. 2024-06-06T07:46:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:54 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/358752074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:46:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:54 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/358752074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:46:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:54 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/358752074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:46:54.682 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-06T07:46:54.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:46:54.922 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:55.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:55 smithi087 ceph-mon[32276]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:55.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:55 smithi195 ceph-mon[37040]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:55.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:55 smithi181 ceph-mon[36790]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:56.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 -- 172.21.15.87:0/1488312811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c881084f0 msgr2=0x7f3c881088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/1488312811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c881084f0 0x7f3c881088d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009990 tx=0x7f3c78009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 -- 172.21.15.87:0/1488312811 shutdown_connections 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/1488312811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c88102f10 0x7f3c8810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/1488312811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c881084f0 0x7f3c881088d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/1488312811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c88102550 0x7f3c881029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 -- 172.21.15.87:0/1488312811 >> 172.21.15.87:0/1488312811 conn(0x7f3c88079650 msgr2=0x7f3c88079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 -- 172.21.15.87:0/1488312811 shutdown_connections 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.800+0000 7f3c8ea11700 1 -- 172.21.15.87:0/1488312811 wait complete. 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 Processor -- start 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 -- start start 2024-06-06T07:46:56.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c88102f10 0x7f3c8819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c881084f0 0x7f3c881989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88112770 con 0x7f3c88102550 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881125f0 con 0x7f3c881084f0 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8ea11700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881128f0 con 0x7f3c88102f10 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c87fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c87fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43784/0 (socket says 172.21.15.87:43784) 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c87fff700 1 -- 172.21.15.87:0/3326492256 learned_addr learned my addr 172.21.15.87:0/3326492256 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.801+0000 7f3c8cfae700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c881084f0 0x7f3c881989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 -- 172.21.15.87:0/3326492256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c88102f10 msgr2=0x7f3c8819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:56.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c88102f10 0x7f3c8819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 -- 172.21.15.87:0/3326492256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c881084f0 msgr2=0x7f3c881989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c881084f0 0x7f3c881989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 -- 172.21.15.87:0/3326492256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c88199150 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c87fff700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c78000c00 tx=0x7f3c780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78027070 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881993e0 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c78004050 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7800f460 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.802+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881a56b0 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.803+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c88069180 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.804+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c7802a070 con 0x7f3c88102550 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.805+0000 7f3c857fa700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 0x7f3c700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:56.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.805+0000 7f3c877fe700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 0x7f3c700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:56.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.805+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f3c780d03a0 con 0x7f3c88102550 2024-06-06T07:46:56.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.805+0000 7f3c877fe700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 0x7f3c700a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c005fd0 tx=0x7f3c7c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:56.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:56.809+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c7805e850 con 0x7f3c88102550 2024-06-06T07:46:57.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.050+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8802dca0 con 0x7f3c88102550 2024-06-06T07:46:57.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.051+0000 7f3c857fa700 1 -- 172.21.15.87:0/3326492256 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f3c78013070 con 0x7f3c88102550 2024-06-06T07:46:57.051 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:46:57.052 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:46:57.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 msgr2=0x7f3c700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:57.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 0x7f3c700a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c005fd0 tx=0x7f3c7c005d00 comp rx=0 tx=0).stop 2024-06-06T07:46:57.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 msgr2=0x7f3c8819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c78000c00 tx=0x7f3c780047b0 comp rx=0 tx=0).stop 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 shutdown_connections 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f3c700a2c20 0x7f3c700a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c881084f0 0x7f3c881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c88102550 0x7f3c8819e7e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 --2- 172.21.15.87:0/3326492256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c88102f10 0x7f3c8819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 >> 172.21.15.87:0/3326492256 conn(0x7f3c88079650 msgr2=0x7f3c881005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 shutdown_connections 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:57.053+0000 7f3c8ea11700 1 -- 172.21.15.87:0/3326492256 wait complete. 2024-06-06T07:46:57.054 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:46:57.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:57 smithi087 ceph-mon[32276]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:57.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3326492256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:46:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:57 smithi195 ceph-mon[37040]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:57.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3326492256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:46:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:57 smithi181 ceph-mon[36790]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T07:46:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3326492256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:46:57.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:46:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:46:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:46:59 smithi195 ceph-mon[37040]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:59.535 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:46:59 smithi087 ceph-mon[32276]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:46:59 smithi181 ceph-mon[36790]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T07:46:59.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1258338680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08107570 msgr2=0x7f0b08107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1258338680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08107570 0x7f0b08107950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009990 tx=0x7f0af8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1258338680 shutdown_connections 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1258338680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08101810 0x7f0b0810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1258338680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08107570 0x7f0b08107950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1258338680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 0x7f0b081012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1258338680 >> 172.21.15.87:0/1258338680 conn(0x7f0b080766b0 msgr2=0x7f0b08076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1258338680 shutdown_connections 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.941+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1258338680 wait complete. 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 Processor -- start 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 -- start start 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 0x7f0b0819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:59.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 0x7f0b08198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112830 con 0x7f0b08101810 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b081126b0 con 0x7f0b08100e50 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0ea4c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b081129b0 con 0x7f0b08107570 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57258/0 (socket says 172.21.15.87:57258) 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 -- 172.21.15.87:0/1166309049 learned_addr learned my addr 172.21.15.87:0/1166309049 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b07fff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 0x7f0b0819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0cfe9700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 0x7f0b08198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 -- 172.21.15.87:0/1166309049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 msgr2=0x7f0b08198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 0x7f0b08198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 -- 172.21.15.87:0/1166309049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 msgr2=0x7f0b0819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 0x7f0b0819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0afffff700 1 -- 172.21.15.87:0/1166309049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b08199250 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.942+0000 7f0b0cfe9700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 0x7f0b08198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0afffff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0af000d950 tx=0x7f0af000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0016070 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0009b40 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b08199540 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af000f620 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.943+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b081a5800 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.944+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08069180 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.945+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0af0019070 con 0x7f0b08101810 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.945+0000 7f0b05ffb700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 0x7f0af40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:46:59.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.946+0000 7f0b07fff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 0x7f0af40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:46:59.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.946+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0af00cf3d0 con 0x7f0b08101810 2024-06-06T07:46:59.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.946+0000 7f0b07fff700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 0x7f0af40a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0af800b5c0 tx=0x7f0af8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:46:59.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:46:59.950+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0af009bfe0 con 0x7f0b08101810 2024-06-06T07:47:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.192+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b0802dca0 con 0x7f0b08101810 2024-06-06T07:47:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.193+0000 7f0b05ffb700 1 -- 172.21.15.87:0/1166309049 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f0af009b8e0 con 0x7f0b08101810 2024-06-06T07:47:00.193 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:00.194 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:47:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 msgr2=0x7f0af40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 0x7f0af40a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0af800b5c0 tx=0x7f0af8005fb0 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 msgr2=0x7f0b0819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0af000d950 tx=0x7f0af000dd10 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 shutdown_connections 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0af40a2c20 0x7f0af40a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b08100e50 0x7f0b0819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b08107570 0x7f0b08198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 --2- 172.21.15.87:0/1166309049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b08101810 0x7f0b0819ee70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.195+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 >> 172.21.15.87:0/1166309049 conn(0x7f0b080766b0 msgr2=0x7f0b080ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:00.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.196+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 shutdown_connections 2024-06-06T07:47:00.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:00.196+0000 7f0b0ea4c700 1 -- 172.21.15.87:0/1166309049 wait complete. 2024-06-06T07:47:00.197 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:47:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:00 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1166309049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:00.568 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1166309049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:00 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1166309049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:00.956 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:47:01.196 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:01.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:01 smithi181 ceph-mon[36790]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:47:01.597 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:01 smithi087 ceph-mon[32276]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:47:01.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:01 smithi195 ceph-mon[37040]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:47:02.362 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:02 smithi087 ceph-mon[32276]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T07:47:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:02 smithi181 ceph-mon[36790]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T07:47:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:02 smithi195 ceph-mon[37040]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T07:47:03.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 -- 172.21.15.87:0/2935800581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294104e90 msgr2=0x7fa294105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:03.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/2935800581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294104e90 0x7fa294105320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009990 tx=0x7fa28c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 -- 172.21.15.87:0/2935800581 shutdown_connections 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/2935800581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa294107380 0x7fa29406a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/2935800581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294104e90 0x7fa294105320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/2935800581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29406a6b0 0x7fa29406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 -- 172.21.15.87:0/2935800581 >> 172.21.15.87:0/2935800581 conn(0x7fa2940782c0 msgr2=0x7fa2940786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 -- 172.21.15.87:0/2935800581 shutdown_connections 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 -- 172.21.15.87:0/2935800581 wait complete. 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.025+0000 7fa29bd2e700 1 Processor -- start 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 -- start start 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 0x7fa29419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:03.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa294104e90 0x7fa29419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa294118d10 con 0x7fa294107380 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa294118b90 con 0x7fa29406a6b0 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29bd2e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa294118e90 con 0x7fa294104e90 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57278/0 (socket says 172.21.15.87:57278) 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 -- 172.21.15.87:0/3734913199 learned_addr learned my addr 172.21.15.87:0/3734913199 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa2992c9700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa294104e90 0x7fa29419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa299aca700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 0x7fa29419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 -- 172.21.15.87:0/3734913199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa294104e90 msgr2=0x7fa29419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:03.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa294104e90 0x7fa29419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 -- 172.21.15.87:0/3734913199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 msgr2=0x7fa29419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 0x7fa29419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 -- 172.21.15.87:0/3734913199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2941a4110 con 0x7fa294107380 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa299aca700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 0x7fa29419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.026+0000 7fa29a2cb700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00c010 tx=0x7fa28c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.027+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c027070 con 0x7fa294107380 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.027+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2941a43a0 con 0x7fa294107380 2024-06-06T07:47:03.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.027+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c004050 con 0x7fa294107380 2024-06-06T07:47:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.027+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c022da0 con 0x7fa294107380 2024-06-06T07:47:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.027+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2941abc80 con 0x7fa294107380 2024-06-06T07:47:03.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.028+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa294069180 con 0x7fa294107380 2024-06-06T07:47:03.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.029+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa28c02a070 con 0x7fa294107380 2024-06-06T07:47:03.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.029+0000 7fa292ffd700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 0x7fa27c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:03.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.030+0000 7fa299aca700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 0x7fa27c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:03.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.030+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fa28c0d0550 con 0x7fa294107380 2024-06-06T07:47:03.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.030+0000 7fa299aca700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 0x7fa27c0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e350 tx=0x7fa284009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:03.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.033+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa28c013070 con 0x7fa294107380 2024-06-06T07:47:03.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.229+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2941abe50 con 0x7fa294107380 2024-06-06T07:47:03.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.231+0000 7fa292ffd700 1 -- 172.21.15.87:0/3734913199 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24916 (secure 0 0 0) 0x7fa28c09d150 con 0x7fa294107380 2024-06-06T07:47:03.231 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:03.231 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":99,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:45:48.222161+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:47:03.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 msgr2=0x7fa27c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:03.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 0x7fa27c0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e350 tx=0x7fa284009040 comp rx=0 tx=0).stop 2024-06-06T07:47:03.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 msgr2=0x7fa2941a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:03.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00c010 tx=0x7fa28c0038b0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 shutdown_connections 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fa27c0a2c20 0x7fa27c0a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29406a6b0 0x7fa29419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa294107380 0x7fa2941a39d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 --2- 172.21.15.87:0/3734913199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa294104e90 0x7fa29419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 >> 172.21.15.87:0/3734913199 conn(0x7fa2940782c0 msgr2=0x7fa294103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 shutdown_connections 2024-06-06T07:47:03.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:03.233+0000 7fa29bd2e700 1 -- 172.21.15.87:0/3734913199 wait complete. 2024-06-06T07:47:03.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3734913199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:03.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3734913199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3734913199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:03.965 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:47:04.206 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:04 smithi181 ceph-mon[36790]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:04.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:04 smithi087 ceph-mon[32276]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:04 smithi195 ceph-mon[37040]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- 172.21.15.87:0/1582238995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 msgr2=0x7f577c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 --2- 172.21.15.87:0/1582238995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c102b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f576c0099c0 tx=0x7f576c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- 172.21.15.87:0/1582238995 shutdown_connections 2024-06-06T07:47:06.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 --2- 172.21.15.87:0/1582238995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 0x7f577c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 --2- 172.21.15.87:0/1582238995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c102b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 --2- 172.21.15.87:0/1582238995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 0x7f577c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- 172.21.15.87:0/1582238995 >> 172.21.15.87:0/1582238995 conn(0x7f577c0795f0 msgr2=0x7f577c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- 172.21.15.87:0/1582238995 shutdown_connections 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- 172.21.15.87:0/1582238995 wait complete. 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 Processor -- start 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.058+0000 7f57824b1700 1 -- start start 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 0x7f577c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 0x7f577c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f577c112760 con 0x7f577c102740 2024-06-06T07:47:06.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1125e0 con 0x7f577c107840 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57824b1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1128e0 con 0x7f577c1030f0 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57298/0 (socket says 172.21.15.87:57298) 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 -- 172.21.15.87:0/156527153 learned_addr learned my addr 172.21.15.87:0/156527153 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577b7fe700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 0x7f577c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f5780a4e700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 0x7f577c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 -- 172.21.15.87:0/156527153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 msgr2=0x7f577c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 0x7f577c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 -- 172.21.15.87:0/156527153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 msgr2=0x7f577c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 0x7f577c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 -- 172.21.15.87:0/156527153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c199140 con 0x7f577c102740 2024-06-06T07:47:06.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f577bfff700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f576c0059d0 tx=0x7f576c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:06.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.059+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c027070 con 0x7f577c102740 2024-06-06T07:47:06.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.060+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c004050 con 0x7f577c102740 2024-06-06T07:47:06.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.060+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1993d0 con 0x7f577c102740 2024-06-06T07:47:06.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.060+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c00f460 con 0x7f577c102740 2024-06-06T07:47:06.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.060+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a56f0 con 0x7f577c102740 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.061+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c069180 con 0x7f577c102740 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.062+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f576c02a070 con 0x7f577c102740 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.063+0000 7f57797fa700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 0x7f57640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.063+0000 7f577b7fe700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 0x7f57640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.063+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f576c0d0ac0 con 0x7f577c102740 2024-06-06T07:47:06.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.064+0000 7f577b7fe700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 0x7f57640a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5770005fd0 tx=0x7f577000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:06.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.067+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f576c031020 con 0x7f577c102740 2024-06-06T07:47:06.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.318+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577c02dca0 con 0x7f577c102740 2024-06-06T07:47:06.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.318+0000 7f57797fa700 1 -- 172.21.15.87:0/156527153 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f576c013070 con 0x7f577c102740 2024-06-06T07:47:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:06.319 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:47:06.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 msgr2=0x7f57640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:06.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 0x7f57640a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5770005fd0 tx=0x7f577000a560 comp rx=0 tx=0).stop 2024-06-06T07:47:06.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 msgr2=0x7f577c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:06.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f576c0059d0 tx=0x7f576c0038b0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 shutdown_connections 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f57640a2c20 0x7f57640a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f577c107840 0x7f577c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c102740 0x7f577c19e820 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 --2- 172.21.15.87:0/156527153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f577c1030f0 0x7f577c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 >> 172.21.15.87:0/156527153 conn(0x7f577c0795f0 msgr2=0x7f577c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 shutdown_connections 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:06.322+0000 7f57824b1700 1 -- 172.21.15.87:0/156527153 wait complete. 2024-06-06T07:47:06.323 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:47:07.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T07:44:02.139989+0000', 'modified': '2024-06-06T07:45:07.358878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24467, 'mds_1': 24448, 'mds_2': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/922595027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 922595027}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 922595027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/1343084853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1343084853}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 1343084853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/494555841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 494555841}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 494555841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.181:6837/3077028235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3077028235}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3077028235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.195:6839/203076359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 203076359}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 203076359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/851837058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 851837058}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 851837058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24467, 'qdb_cluster': [24448, 24446, 24467]} 2024-06-06T07:47:07.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T07:47:07.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T07:47:07.278 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:07.330 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:07 smithi087 ceph-mon[32276]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:07.330 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:07 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/156527153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:07 smithi195 ceph-mon[37040]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:07 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/156527153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:07 smithi181 ceph-mon[36790]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:07 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/156527153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/4285533412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed010ef70 msgr2=0x7f2ed0111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/4285533412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed010ef70 0x7f2ed0111360 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009990 tx=0x7f2eb8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/4285533412 shutdown_connections 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/4285533412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed010ef70 0x7f2ed0111360 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/4285533412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 0x7f2ed006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/4285533412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed006b500 0x7f2ed010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/4285533412 >> 172.21.15.87:0/4285533412 conn(0x7f2ed0078360 msgr2=0x7f2ed0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/4285533412 shutdown_connections 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/4285533412 wait complete. 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 Processor -- start 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.122+0000 7f2ed6ecc700 1 -- start start 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 0x7f2ed019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed010ef70 0x7f2ed019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0114970 con 0x7f2ed006b500 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01147f0 con 0x7f2ed010ef70 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0114af0 con 0x7f2ed006abe0 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57214/0 (socket says 172.21.15.87:57214) 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 -- 172.21.15.87:0/1626244383 learned_addr learned my addr 172.21.15.87:0/1626244383 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:09.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 0x7f2ed019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed5469700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed010ef70 0x7f2ed019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 -- 172.21.15.87:0/1626244383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 msgr2=0x7f2ed019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 0x7f2ed019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 -- 172.21.15.87:0/1626244383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed010ef70 msgr2=0x7f2ed019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed010ef70 0x7f2ed019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 -- 172.21.15.87:0/1626244383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed019fd80 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecffff700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ada0 tx=0x7f2ec000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecdffb700 1 -- 172.21.15.87:0/1626244383 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0017020 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecdffb700 1 -- 172.21.15.87:0/1626244383 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec0004bb0 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ecdffb700 1 -- 172.21.15.87:0/1626244383 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0005460 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.123+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed01a0070 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.124+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed01a78f0 con 0x7f2ed006b500 2024-06-06T07:47:09.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.125+0000 7f2ecdffb700 1 -- 172.21.15.87:0/1626244383 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ec000c980 con 0x7f2ed006b500 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.126+0000 7f2ecdffb700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 0x7f2ebc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.126+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 0x7f2ebc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.126+0000 7f2ecdffb700 1 -- 172.21.15.87:0/1626244383 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2ec00ceaf0 con 0x7f2ed006b500 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 0x7f2ebc0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005ea0 tx=0x7f2ec8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 Processor -- start 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 -- start start 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 0x55eb43e799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e79f10 0x55eb43e73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eb43dc57c0 con 0x55eb43e73ed0 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb43d6d6d0 con 0x55eb43e795f0 2024-06-06T07:47:09.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb43dc1780 con 0x55eb43e79f10 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57222/0 (socket says 172.21.15.87:57222) 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.127+0000 7f2ed5469700 1 -- 172.21.15.87:0/3096164719 learned_addr learned my addr 172.21.15.87:0/3096164719 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ecffff700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e79f10 0x55eb43e73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 0x55eb43e799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 -- 172.21.15.87:0/3096164719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e79f10 msgr2=0x55eb43e73990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e79f10 0x55eb43e73990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 -- 172.21.15.87:0/3096164719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 msgr2=0x55eb43e799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 0x55eb43e799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 -- 172.21.15.87:0/3096164719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb43d63f00 con 0x55eb43e73ed0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 0x55eb43e799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009990 tx=0x7f2eb8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e962ba00fc028f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ec6ffd700 1 -- 172.21.15.87:0/3096164719 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8028070 con 0x55eb43e73ed0 2024-06-06T07:47:09.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ec6ffd700 1 -- 172.21.15.87:0/3096164719 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb80055b0 con 0x55eb43e73ed0 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.128+0000 7f2ec6ffd700 1 -- 172.21.15.87:0/3096164719 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800bc70 con 0x55eb43e73ed0 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3096164719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 msgr2=0x55eb43e742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009990 tx=0x7f2eb8009f90 comp rx=0 tx=0).stop 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3096164719 shutdown_connections 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e795f0 0x55eb43e799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e742f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3096164719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e79f10 0x55eb43e73990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3096164719 >> 172.21.15.87:0/3096164719 conn(0x55eb43d9ec70 msgr2=0x55eb43e6e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3096164719 shutdown_connections 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3096164719 wait complete. 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 -- start start 2024-06-06T07:47:09.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.129+0000 7f2ee174fb80 1 Processor -- start 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e80920 con 0x55eb43e79f10 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb43d65ba0 con 0x55eb43e73ed0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e80aa0 con 0x55eb43e795f0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ee174fb80 10 client.0 _ll_register_callbacks cb 0x55eb43d5f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed4c68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57234/0 (socket says 172.21.15.87:57234) 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 -- 172.21.15.87:0/3248791204 learned_addr learned my addr 172.21.15.87:0/3248791204 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ecffff700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 -- 172.21.15.87:0/3248791204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 msgr2=0x55eb43e7cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 -- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 msgr2=0x55eb43e7fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 -- 172.21.15.87:0/3248791204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb43d14880 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2ed5469700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ce0 tx=0x7f2eb8005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c01326a1685ba server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.130+0000 7f2eaffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8030020 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2eb800ba70 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb80055b0 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2eb803b050 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 10 client.15063 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb43e8b1c0 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb80055b0 con 0x55eb43e79f10 2024-06-06T07:47:09.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2eb802b030 con 0x55eb43e79f10 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 20 client.15063 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43e8c510 0x55eb43e8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 4 client.15063 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.131+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb43dd2cc0 con 0x55eb43e8c510 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.132+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43e8c510 0x55eb43e8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.132+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43e8c510 0x55eb43e8e6d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=180af2288d151385 server_cookie=a79e009dcdff7f26 in_seq=0 out_seq=0 2024-06-06T07:47:09.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.132+0000 7f2eaffff700 10 client.15063 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2eaffff700 1 -- 172.21.15.87:0/3248791204 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eb43dd2cc0 con 0x55eb43e8c510 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2eaffff700 10 client.15063 handle_command_reply: tid=0 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 client.15063 shutdown 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 msgr2=0x55eb43e7d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ce0 tx=0x7f2eb8005300 comp rx=0 tx=0).stop 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 shutdown_connections 2024-06-06T07:47:09.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43e8c510 0x55eb43e8e6d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/3248791204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2eaffff700 0 client.15063 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 >> 172.21.15.87:0/3248791204 conn(0x55eb43d9ec70 msgr2=0x55eb43e72980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 shutdown_connections 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 1 -- 172.21.15.87:0/3248791204 wait complete. 2024-06-06T07:47:09.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.133+0000 7f2ee174fb80 20 client.15063 trim_cache size 0 max 16384 2024-06-06T07:47:09.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 Processor -- start 2024-06-06T07:47:09.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 -- start start 2024-06-06T07:47:09.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e68ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8ca10 con 0x55eb43e73ed0 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8d0e0 con 0x55eb43e79f10 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.154+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8d7b0 con 0x55eb43e795f0 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57244/0 (socket says 172.21.15.87:57244) 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 -- 172.21.15.87:0/902777728 learned_addr learned my addr 172.21.15.87:0/902777728 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed5469700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ecffff700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e68ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 -- 172.21.15.87:0/902777728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 msgr2=0x55eb43e68ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e68ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 -- 172.21.15.87:0/902777728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 msgr2=0x55eb43e7d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 -- 172.21.15.87:0/902777728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb43e8e370 con 0x55eb43e73ed0 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed5469700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ecffff700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e68ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800ee20 tx=0x7f2ec8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fdc899f7041bfc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ec67fc700 1 -- 172.21.15.87:0/902777728 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec801f8d0 con 0x55eb43e73ed0 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ec67fc700 1 -- 172.21.15.87:0/902777728 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec802b070 con 0x55eb43e73ed0 2024-06-06T07:47:09.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.155+0000 7f2ec67fc700 1 -- 172.21.15.87:0/902777728 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8026b80 con 0x55eb43e73ed0 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- 172.21.15.87:0/902777728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 msgr2=0x55eb43e73990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800ee20 tx=0x7f2ec8014000 comp rx=0 tx=0).stop 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- 172.21.15.87:0/902777728 shutdown_connections 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e79f10 0x55eb43e7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e73ed0 0x55eb43e73990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/902777728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e68ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- 172.21.15.87:0/902777728 >> 172.21.15.87:0/902777728 conn(0x55eb43d9ec70 msgr2=0x55eb43e7b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- 172.21.15.87:0/902777728 shutdown_connections 2024-06-06T07:47:09.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- 172.21.15.87:0/902777728 wait complete. 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 -- start start 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.156+0000 7f2ee174fb80 1 Processor -- start 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8e550 con 0x55eb43e79f10 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8de80 con 0x55eb43e73ed0 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb43e8ec20 con 0x55eb43e795f0 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 10 client.0 _ll_register_callbacks cb 0x55eb43d5f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57246/0 (socket says 172.21.15.87:57246) 2024-06-06T07:47:09.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 -- 172.21.15.87:0/327032068 learned_addr learned my addr 172.21.15.87:0/327032068 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ecffff700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 -- 172.21.15.87:0/327032068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 msgr2=0x55eb43e7d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 -- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 msgr2=0x55eb43e7cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 -- 172.21.15.87:0/327032068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb43e96250 con 0x55eb43e79f10 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ed5469700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005f00 tx=0x7f2eb8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8166885cdf7d7b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2eaffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8030070 con 0x55eb43e79f10 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f2eb800f660 con 0x55eb43e79f10 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.157+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec802b070 con 0x55eb43e79f10 2024-06-06T07:47:09.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2eb8028040 con 0x55eb43e79f10 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 10 client.15069 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb43ef8e50 con 0x55eb43e79f10 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec802b070 con 0x55eb43e79f10 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2eb802b040 con 0x55eb43e79f10 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 20 client.15069 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43efd8a0 0x55eb43efc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 4 client.15069 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "status"}) -- 0x55eb43dcd240 con 0x55eb43efd8a0 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.158+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43efd8a0 0x55eb43efc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ed4c68700 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43efd8a0 0x55eb43efc5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c092f010db552c4a server_cookie=7f7491997e356dda in_seq=0 out_seq=0 2024-06-06T07:47:09.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2eaffff700 10 client.15069 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2eaffff700 1 -- 172.21.15.87:0/327032068 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55eb43dcd240 con 0x55eb43efd8a0 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2eaffff700 10 client.15069 handle_command_reply: tid=0 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 client.15069 shutdown 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 msgr2=0x55eb43e7d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005f00 tx=0x7f2eb8009310 comp rx=0 tx=0).stop 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 shutdown_connections 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb43e73ed0 0x55eb43e7cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eb43efd8a0 0x55eb43efc5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eb43e79f10 0x55eb43e7d540 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 --2- 172.21.15.87:0/327032068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb43e795f0 0x55eb43e7d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2eaffff700 0 client.15069 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 >> 172.21.15.87:0/327032068 conn(0x55eb43d9ec70 msgr2=0x55eb43e67db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 shutdown_connections 2024-06-06T07:47:09.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 1 -- 172.21.15.87:0/327032068 wait complete. 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.159+0000 7f2ee174fb80 20 client.15069 trim_cache size 0 max 16384 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24446, 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:09.166 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 126.936239722, 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 216.352319733 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:09.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 msgr2=0x7f2ebc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 0x7f2ebc0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005ea0 tx=0x7f2ec8005e30 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 msgr2=0x7f2ed019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ada0 tx=0x7f2ec000c330 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 shutdown_connections 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f2ebc0a2ca0 0x7f2ebc0a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ed010ef70 0x7f2ed019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed006abe0 0x7f2ed019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 --2- 172.21.15.87:0/1626244383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006b500 0x7f2ed019b300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 >> 172.21.15.87:0/1626244383 conn(0x7f2ed0078360 msgr2=0x7f2ed0110320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 shutdown_connections 2024-06-06T07:47:09.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:09.160+0000 7f2ed6ecc700 1 -- 172.21.15.87:0/1626244383 wait complete. 2024-06-06T07:47:09.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:09 smithi087 ceph-mon[32276]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:09.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:09 smithi195 ceph-mon[37040]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:09.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:09 smithi181 ceph-mon[36790]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:09.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T07:47:10.207 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:11 smithi195 ceph-mon[37040]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:11.539 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:11 smithi087 ceph-mon[32276]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:11 smithi181 ceph-mon[36790]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 -- 172.21.15.87:0/1279410582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 msgr2=0x7f38cc115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/1279410582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc115a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38c400b0a0 tx=0x7f38c400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 -- 172.21.15.87:0/1279410582 shutdown_connections 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/1279410582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc115a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/1279410582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc0746c0 0x7f38cc074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/1279410582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc074fe0 0x7f38cc113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 -- 172.21.15.87:0/1279410582 >> 172.21.15.87:0/1279410582 conn(0x7f38cc0795f0 msgr2=0x7f38cc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 -- 172.21.15.87:0/1279410582 shutdown_connections 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.946+0000 7f38d2bc2700 1 -- 172.21.15.87:0/1279410582 wait complete. 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 Processor -- start 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 -- start start 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0746c0 0x7f38cc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 0x7f38cc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc118d90 con 0x7f38cc113660 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc118c10 con 0x7f38cc0746c0 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d2bc2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc118f10 con 0x7f38cc074fe0 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57260/0 (socket says 172.21.15.87:57260) 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 -- 172.21.15.87:0/3121089328 learned_addr learned my addr 172.21.15.87:0/3121089328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d095e700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0746c0 0x7f38cc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38cbfff700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 0x7f38cc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 -- 172.21.15.87:0/3121089328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 msgr2=0x7f38cc19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 0x7f38cc19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 -- 172.21.15.87:0/3121089328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0746c0 msgr2=0x7f38cc19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0746c0 0x7f38cc19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 -- 172.21.15.87:0/3121089328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc1a4160 con 0x7f38cc113660 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38cbfff700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 0x7f38cc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.947+0000 7f38d115f700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f38c40062a0 tx=0x7f38c4004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.948+0000 7f38c9ffb700 1 -- 172.21.15.87:0/3121089328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4021070 con 0x7f38cc113660 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.948+0000 7f38c9ffb700 1 -- 172.21.15.87:0/3121089328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c4007c00 con 0x7f38cc113660 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.948+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc1a43f0 con 0x7f38cc113660 2024-06-06T07:47:11.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.948+0000 7f38c9ffb700 1 -- 172.21.15.87:0/3121089328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c401d450 con 0x7f38cc113660 2024-06-06T07:47:11.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.948+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc1abcd0 con 0x7f38cc113660 2024-06-06T07:47:11.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.950+0000 7f38c9ffb700 1 -- 172.21.15.87:0/3121089328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f38c4004020 con 0x7f38cc113660 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.951+0000 7f38c9ffb700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 0x7f38bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.951+0000 7f38d095e700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 0x7f38bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.951+0000 7f38c9ffb700 1 -- 172.21.15.87:0/3121089328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38c40d03d0 con 0x7f38cc113660 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.951+0000 7f38d095e700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 0x7f38bc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38c0004630 tx=0x7f38c000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 Processor -- start 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 -- start start 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a151210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56149a043f10 con 0x56149a151750 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56149a0a2800 con 0x56149a156f10 2024-06-06T07:47:11.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38dd445b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56149a00a410 con 0x56149a157830 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57268/0 (socket says 172.21.15.87:57268) 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 -- 172.21.15.87:0/724889810 learned_addr learned my addr 172.21.15.87:0/724889810 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38cbfff700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 -- 172.21.15.87:0/724889810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 msgr2=0x56149a151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d095e700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 -- 172.21.15.87:0/724889810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 msgr2=0x56149a1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a1572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 -- 172.21.15.87:0/724889810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561499ff1880 con 0x56149a151750 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38cbfff700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.952+0000 7f38d115f700 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38c40a5020 tx=0x7f38c4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187382aa2d43f01 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38b6ffd700 1 -- 172.21.15.87:0/724889810 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c402a070 con 0x56149a151750 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38b6ffd700 1 -- 172.21.15.87:0/724889810 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c40a0d20 con 0x56149a151750 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38b6ffd700 1 -- 172.21.15.87:0/724889810 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c409cbd0 con 0x56149a151750 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 -- 172.21.15.87:0/724889810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 msgr2=0x56149a151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38c40a5020 tx=0x7f38c4003f70 comp rx=0 tx=0).stop 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 -- 172.21.15.87:0/724889810 shutdown_connections 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151b70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 --2- 172.21.15.87:0/724889810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 -- 172.21.15.87:0/724889810 >> 172.21.15.87:0/724889810 conn(0x56149a07e030 msgr2=0x56149a14c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 -- 172.21.15.87:0/724889810 shutdown_connections 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.953+0000 7f38dd445b80 1 -- 172.21.15.87:0/724889810 wait complete. 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 -- start start 2024-06-06T07:47:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 Processor -- start 2024-06-06T07:47:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 0x56149a15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56149a042f70 con 0x56149a156f10 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56149a09e7c0 con 0x56149a151750 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56149a15dfc0 con 0x56149a157830 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38dd445b80 10 client.0 _ll_register_callbacks cb 0x56149a038560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57284/0 (socket says 172.21.15.87:57284) 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38cbfff700 1 -- 172.21.15.87:0/3999627977 learned_addr learned my addr 172.21.15.87:0/3999627977 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.954+0000 7f38d115f700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38d095e700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 0x56149a15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 -- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 msgr2=0x56149a15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 -- 172.21.15.87:0/3999627977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 msgr2=0x56149a15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 0x56149a15d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 -- 172.21.15.87:0/3999627977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149a068700 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38d095e700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 0x56149a15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38cbfff700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38b8005910 tx=0x7f38b800f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f38c67b73fbea0f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38abfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004a30 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38b8004bd0 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c40a0d20 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38b8004dd0 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38dd445b80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56149a168710 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.955+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c40a0d20 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f38b8011b60 con 0x56149a156f10 2024-06-06T07:47:11.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 20 client.15081 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a16ad60 0x56149a16c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 4 client.15081 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56149a0afd00 con 0x56149a16ad60 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38d095e700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a16ad60 0x56149a16c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38abfff700 10 client.15081 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.956+0000 7f38d095e700 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a16ad60 0x56149a16c1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3c2a7aa7214728c server_cookie=6a3702eeda68db48 in_seq=0 out_seq=0 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38abfff700 1 -- 172.21.15.87:0/3999627977 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56149a0afd00 con 0x56149a16ad60 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38abfff700 10 client.15081 handle_command_reply: tid=0 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 client.15081 shutdown 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 msgr2=0x56149a15a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38b8005910 tx=0x7f38b800f940 comp rx=0 tx=0).stop 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 shutdown_connections 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a16ad60 0x56149a16c1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a151750 0x56149a15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 --2- 172.21.15.87:0/3999627977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a15a900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 >> 172.21.15.87:0/3999627977 conn(0x56149a07e030 msgr2=0x56149a14b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.957+0000 7f38abfff700 0 client.15081 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.958+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 shutdown_connections 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.958+0000 7f38dd445b80 1 -- 172.21.15.87:0/3999627977 wait complete. 2024-06-06T07:47:11.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.958+0000 7f38dd445b80 20 client.15081 trim_cache size 0 max 16384 2024-06-06T07:47:11.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.978+0000 7f38dd445b80 1 Processor -- start 2024-06-06T07:47:11.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 -- start start 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a14be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56149a16a510 con 0x56149a151750 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56149a16abe0 con 0x56149a156f10 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38dd445b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56149a16b2b0 con 0x56149a157830 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57296/0 (socket says 172.21.15.87:57296) 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 -- 172.21.15.87:0/4101039657 learned_addr learned my addr 172.21.15.87:0/4101039657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 -- 172.21.15.87:0/4101039657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 msgr2=0x56149a14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 -- 172.21.15.87:0/4101039657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 msgr2=0x56149a158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 -- 172.21.15.87:0/4101039657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149a159880 con 0x56149a151750 2024-06-06T07:47:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38d095e700 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f38c000bef0 tx=0x7f38c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dda89561599de8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38b67fc700 1 -- 172.21.15.87:0/4101039657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0003dd0 con 0x56149a151750 2024-06-06T07:47:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38b67fc700 1 -- 172.21.15.87:0/4101039657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0011d30 con 0x56149a151750 2024-06-06T07:47:11.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.979+0000 7f38b67fc700 1 -- 172.21.15.87:0/4101039657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00053a0 con 0x56149a151750 2024-06-06T07:47:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- 172.21.15.87:0/4101039657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 msgr2=0x56149a151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f38c000bef0 tx=0x7f38c0013000 comp rx=0 tx=0).stop 2024-06-06T07:47:11.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- 172.21.15.87:0/4101039657 shutdown_connections 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a157830 0x56149a14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a151750 0x56149a151210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 --2- 172.21.15.87:0/4101039657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a156f10 0x56149a158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- 172.21.15.87:0/4101039657 >> 172.21.15.87:0/4101039657 conn(0x56149a07e030 msgr2=0x56149a157c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- 172.21.15.87:0/4101039657 shutdown_connections 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- 172.21.15.87:0/4101039657 wait complete. 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.980+0000 7f38dd445b80 1 -- start start 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 Processor -- start 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a151750 0x56149a175650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a157830 0x56149a170600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56149a16c050 con 0x56149a156f10 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56149a16b980 con 0x56149a157830 2024-06-06T07:47:11.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56149a16c720 con 0x56149a151750 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38dd445b80 10 client.0 _ll_register_callbacks cb 0x56149a038560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57300/0 (socket says 172.21.15.87:57300) 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 -- 172.21.15.87:0/108308366 learned_addr learned my addr 172.21.15.87:0/108308366 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38d095e700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a151750 0x56149a175650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 -- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a151750 msgr2=0x56149a175650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38d115f700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a157830 0x56149a170600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a151750 0x56149a175650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 -- 172.21.15.87:0/108308366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a157830 msgr2=0x56149a170600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a157830 0x56149a170600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.981+0000 7f38cbfff700 1 -- 172.21.15.87:0/108308366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149a170b40 con 0x56149a156f10 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38cbfff700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f38b8011f80 tx=0x7f38b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab9a8617aef786 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:11.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8011850 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f38b80119f0 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c0011d30 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38b8011bf0 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 10 client.15087 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56149a1d6ec0 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0011d30 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f38b8003a90 con 0x56149a156f10 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 20 client.15087 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a1dbab0 0x56149a1da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 4 client.15087 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.982+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "status"}) -- 0x56149a0aa280 con 0x56149a1dbab0 2024-06-06T07:47:11.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38d095e700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a1dbab0 0x56149a1da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38d095e700 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a1dbab0 0x56149a1da7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d6b621b743ece47 server_cookie=797fecfc154139b7 in_seq=0 out_seq=0 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38abfff700 10 client.15087 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38abfff700 1 -- 172.21.15.87:0/108308366 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56149a0aa280 con 0x56149a1dbab0 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38abfff700 10 client.15087 handle_command_reply: tid=0 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 client.15087 shutdown 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 msgr2=0x56149a175b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f38b8011f80 tx=0x7f38b8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 shutdown_connections 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x56149a1dbab0 0x56149a1da7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56149a157830 0x56149a170600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56149a151750 0x56149a175650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 --2- 172.21.15.87:0/108308366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56149a156f10 0x56149a175b90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 >> 172.21.15.87:0/108308366 conn(0x56149a07e030 msgr2=0x56149a15a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.983+0000 7f38abfff700 0 client.15087 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 shutdown_connections 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38dd445b80 1 -- 172.21.15.87:0/108308366 wait complete. 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38dd445b80 20 client.15087 trim_cache size 0 max 16384 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24448, 2024-06-06T07:47:11.989 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 130.76952356199999, 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 216.318887241 2024-06-06T07:47:11.990 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 msgr2=0x7f38bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 0x7f38bc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38c0004630 tx=0x7f38c000b3c0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 msgr2=0x7f38cc1a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f38c40062a0 tx=0x7f38c4004d70 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 shutdown_connections 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f38bc0a2c20 0x7f38bc0a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0746c0 0x7f38cc19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38cc113660 0x7f38cc1a3a20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 --2- 172.21.15.87:0/3121089328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38cc074fe0 0x7f38cc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 >> 172.21.15.87:0/3121089328 conn(0x7f38cc0795f0 msgr2=0x7f38cc1123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:11.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 shutdown_connections 2024-06-06T07:47:11.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:11.984+0000 7f38d2bc2700 1 -- 172.21.15.87:0/3121089328 wait complete. 2024-06-06T07:47:12.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:47:12.956 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:13.342 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:13 smithi087 ceph-mon[32276]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:13 smithi195 ceph-mon[37040]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:13 smithi181 ceph-mon[36790]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:14.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 -- 172.21.15.87:0/1487583253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294106370 msgr2=0x7fb294106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 --2- 172.21.15.87:0/1487583253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294106370 0x7fb294106750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb284009990 tx=0x7fb284009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 -- 172.21.15.87:0/1487583253 shutdown_connections 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 --2- 172.21.15.87:0/1487583253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 0x7fb29410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 --2- 172.21.15.87:0/1487583253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294106370 0x7fb294106750 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 --2- 172.21.15.87:0/1487583253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb294101b90 0x7fb294102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 -- 172.21.15.87:0/1487583253 >> 172.21.15.87:0/1487583253 conn(0x7fb294077a00 msgr2=0x7fb2940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 -- 172.21.15.87:0/1487583253 shutdown_connections 2024-06-06T07:47:14.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.686+0000 7fb298f67700 1 -- 172.21.15.87:0/1487583253 wait complete. 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 Processor -- start 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 -- start start 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 0x7fb294199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb294106370 0x7fb29419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb294111760 con 0x7fb294101b90 2024-06-06T07:47:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941115e0 con 0x7fb294102550 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb298f67700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941118e0 con 0x7fb294106370 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb29259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb29259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57330/0 (socket says 172.21.15.87:57330) 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.687+0000 7fb29259c700 1 -- 172.21.15.87:0/4001496671 learned_addr learned my addr 172.21.15.87:0/4001496671 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb291d9b700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 0x7fb294199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 -- 172.21.15.87:0/4001496671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb294106370 msgr2=0x7fb29419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb294106370 0x7fb29419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 -- 172.21.15.87:0/4001496671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 msgr2=0x7fb294199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 0x7fb294199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 -- 172.21.15.87:0/4001496671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29419dd50 con 0x7fb294101b90 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb29259c700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb284000c00 tx=0x7fb284003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb28b7fe700 1 -- 172.21.15.87:0/4001496671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284027070 con 0x7fb294101b90 2024-06-06T07:47:14.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29419dfe0 con 0x7fb294101b90 2024-06-06T07:47:14.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb28b7fe700 1 -- 172.21.15.87:0/4001496671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb284004190 con 0x7fb294101b90 2024-06-06T07:47:14.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb28b7fe700 1 -- 172.21.15.87:0/4001496671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28400f460 con 0x7fb294101b90 2024-06-06T07:47:14.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.688+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2941a58c0 con 0x7fb294101b90 2024-06-06T07:47:14.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.690+0000 7fb28b7fe700 1 -- 172.21.15.87:0/4001496671 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb28402a070 con 0x7fb294101b90 2024-06-06T07:47:14.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.691+0000 7fb28b7fe700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 0x7fb2800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.691+0000 7fb291d9b700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 0x7fb2800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.691+0000 7fb28b7fe700 1 -- 172.21.15.87:0/4001496671 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb2840d0300 con 0x7fb294101b90 2024-06-06T07:47:14.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.691+0000 7fb291d9b700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 0x7fb2800a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006fd0 tx=0x7fb27c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 Processor -- start 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 -- start start 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x56275982aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982afe0 0x5627598371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562759718f10 con 0x562759830420 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562759777800 con 0x56275982a620 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627596df410 con 0x56275982afe0 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb29259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb29259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57338/0 (socket says 172.21.15.87:57338) 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb29259c700 1 -- 172.21.15.87:0/2538587967 learned_addr learned my addr 172.21.15.87:0/2538587967 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.692+0000 7fb291d9b700 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x56275982aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb292d9d700 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982afe0 0x5627598371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 -- 172.21.15.87:0/2538587967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982afe0 msgr2=0x5627598371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982afe0 0x5627598371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 -- 172.21.15.87:0/2538587967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 msgr2=0x56275982aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x56275982aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 -- 172.21.15.87:0/2538587967 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627596c6880 con 0x562759830420 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb29259c700 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb28409c4a0 tx=0x7fb284009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1224544dd5d69e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb288ff9700 1 -- 172.21.15.87:0/2538587967 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28409c7a0 con 0x562759830420 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb288ff9700 1 -- 172.21.15.87:0/2538587967 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2840a0b50 con 0x562759830420 2024-06-06T07:47:14.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.693+0000 7fb288ff9700 1 -- 172.21.15.87:0/2538587967 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28409aaa0 con 0x562759830420 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2538587967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 msgr2=0x562759830800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb28409c4a0 tx=0x7fb284009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2538587967 shutdown_connections 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982afe0 0x5627598371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562759830420 0x562759830800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2538587967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x56275982aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2538587967 >> 172.21.15.87:0/2538587967 conn(0x562759753030 msgr2=0x5627598211d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2538587967 shutdown_connections 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2538587967 wait complete. 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 -- start start 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.694+0000 7fb2a37eab80 1 Processor -- start 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x562759839d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x562759834090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562759717f70 con 0x56275982afe0 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5627597737c0 con 0x562759830420 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562759830e00 con 0x56275982a620 2024-06-06T07:47:14.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 10 client.0 _ll_register_callbacks cb 0x56275970d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57346/0 (socket says 172.21.15.87:57346) 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 -- 172.21.15.87:0/3132434362 learned_addr learned my addr 172.21.15.87:0/3132434362 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb29259c700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x562759839d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb292d9d700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x562759834090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 -- 172.21.15.87:0/3132434362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 msgr2=0x562759839d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x562759839d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 -- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 msgr2=0x562759834090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x562759834090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 -- 172.21.15.87:0/3132434362 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275973d700 con 0x56275982afe0 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb292d9d700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x562759834090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb29259c700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x562759839d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:14.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00dd70 tx=0x7fb27c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d0407e8b372c4b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c024020 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb27c00f040 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.695+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2840a0b50 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb27c02f050 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 10 client.15099 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562759841ad0 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2840a0b50 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb27c013070 con 0x56275982afe0 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:47:14.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 20 client.15099 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x562759845600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 4 client.15099 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.696+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562759784d00 con 0x562759844190 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.697+0000 7fb29259c700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x562759845600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.697+0000 7fb29259c700 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x562759845600 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e613750af6883451 server_cookie=57a1484cdc9fadbe in_seq=0 out_seq=0 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.697+0000 7fb279ffb700 10 client.15099 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb279ffb700 1 -- 172.21.15.87:0/3132434362 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562759784d00 con 0x562759844190 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb279ffb700 10 client.15099 handle_command_reply: tid=0 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 client.15099 shutdown 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 msgr2=0x56275983a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00dd70 tx=0x7fb27c004860 comp rx=0 tx=0).stop 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 shutdown_connections 2024-06-06T07:47:14.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x562759845600 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x562759834090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb279ffb700 0 client.15099 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x562759839d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3132434362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275983a250 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 >> 172.21.15.87:0/3132434362 conn(0x562759753030 msgr2=0x5627598206b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 shutdown_connections 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3132434362 wait complete. 2024-06-06T07:47:14.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.698+0000 7fb2a37eab80 20 client.15099 trim_cache size 0 max 16384 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 Processor -- start 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 -- start start 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x562759833070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562759830420 0x562759820e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562759843940 con 0x56275982afe0 2024-06-06T07:47:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562759844010 con 0x56275982a620 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.719+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627598446e0 con 0x562759830420 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57358/0 (socket says 172.21.15.87:57358) 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 -- 172.21.15.87:0/2189839585 learned_addr learned my addr 172.21.15.87:0/2189839585 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb29259c700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x562759833070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb292d9d700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562759830420 0x562759820e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 -- 172.21.15.87:0/2189839585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562759830420 msgr2=0x562759820e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562759830420 0x562759820e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 -- 172.21.15.87:0/2189839585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 msgr2=0x562759833070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x562759833070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 -- 172.21.15.87:0/2189839585 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562759835640 con 0x56275982afe0 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb29259c700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x562759833070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb291d9b700 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009730 tx=0x7fb27c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d6bb85aa25256 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb27bfff700 1 -- 172.21.15.87:0/2189839585 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c024030 con 0x56275982afe0 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb27bfff700 1 -- 172.21.15.87:0/2189839585 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27c014aa0 con 0x56275982afe0 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.720+0000 7fb27bfff700 1 -- 172.21.15.87:0/2189839585 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c020890 con 0x56275982afe0 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2189839585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 msgr2=0x56275982e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009730 tx=0x7fb27c004750 comp rx=0 tx=0).stop 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2189839585 shutdown_connections 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56275982a620 0x562759833070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562759830420 0x562759820e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/2189839585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275982e520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2189839585 >> 172.21.15.87:0/2189839585 conn(0x562759753030 msgr2=0x56275982de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2189839585 shutdown_connections 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- 172.21.15.87:0/2189839585 wait complete. 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 -- start start 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.721+0000 7fb2a37eab80 1 Processor -- start 2024-06-06T07:47:14.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x56275984e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x5627598ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562759845480 con 0x56275982afe0 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562759844db0 con 0x562759830420 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562759845b50 con 0x56275982a620 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 10 client.0 _ll_register_callbacks cb 0x56275970d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57360/0 (socket says 172.21.15.87:57360) 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 -- 172.21.15.87:0/3962715112 learned_addr learned my addr 172.21.15.87:0/3962715112 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb29259c700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x56275984e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb292d9d700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x5627598ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 -- 172.21.15.87:0/3962715112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 msgr2=0x56275984e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x56275984e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 -- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 msgr2=0x5627598ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x5627598ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 -- 172.21.15.87:0/3962715112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627598adf50 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb291d9b700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb27c035040 tx=0x7fb27c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d861c09293157d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb279ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c024040 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb27c0205f0 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27c014aa0 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.722+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb27c012670 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 10 client.15105 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627598af200 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27c014aa0 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb27c0060b0 con 0x56275982afe0 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:14.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:47:14.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 20 client.15105 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:14.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x5627598b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:14.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 4 client.15105 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:14.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "status"}) -- 0x56275977f280 con 0x562759844190 2024-06-06T07:47:14.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.723+0000 7fb29259c700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x5627598b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb279ffb700 10 client.15105 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb29259c700 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x5627598b2690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa715a986625134 server_cookie=8c65e8e33ba3d62c in_seq=0 out_seq=0 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb279ffb700 1 -- 172.21.15.87:0/3962715112 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56275977f280 con 0x562759844190 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb279ffb700 10 client.15105 handle_command_reply: tid=0 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 client.15105 shutdown 2024-06-06T07:47:14.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 msgr2=0x56275984ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb27c035040 tx=0x7fb27c010000 comp rx=0 tx=0).stop 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 shutdown_connections 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562759844190 0x5627598b2690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562759830420 0x5627598ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb279ffb700 0 client.15105 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56275982a620 0x56275984e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 --2- 172.21.15.87:0/3962715112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56275982afe0 0x56275984ea40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 >> 172.21.15.87:0/3962715112 conn(0x562759753030 msgr2=0x56275982de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 shutdown_connections 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 1 -- 172.21.15.87:0/3962715112 wait complete. 2024-06-06T07:47:14.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.724+0000 7fb2a37eab80 20 client.15105 trim_cache size 0 max 16384 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24467, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 192.393593431, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 209.15251391300001 2024-06-06T07:47:14.731 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 msgr2=0x7fb2800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 0x7fb2800a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006fd0 tx=0x7fb27c008040 comp rx=0 tx=0).stop 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 msgr2=0x7fb294198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb284000c00 tx=0x7fb284003730 comp rx=0 tx=0).stop 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 shutdown_connections 2024-06-06T07:47:14.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2800a2c20 0x7fb2800a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb294106370 0x7fb29419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb294101b90 0x7fb294198d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 --2- 172.21.15.87:0/4001496671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb294102550 0x7fb294199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 >> 172.21.15.87:0/4001496671 conn(0x7fb294077a00 msgr2=0x7fb294100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 shutdown_connections 2024-06-06T07:47:14.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:14.725+0000 7fb298f67700 1 -- 172.21.15.87:0/4001496671 wait complete. 2024-06-06T07:47:15.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:15 smithi087 ceph-mon[32276]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:15.501 INFO:tasks.cephadm:Setup complete, yielding 2024-06-06T07:47:15.502 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T07:47:15.518 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:47:15.518 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-06T07:47:15.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:15 smithi195 ceph-mon[37040]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:15.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:15 smithi181 ceph-mon[36790]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:17.497 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:17 smithi087 ceph-mon[32276]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 -- 172.21.15.87:0/2760386476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 msgr2=0x7f7cd8107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/2760386476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8107cd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009990 tx=0x7f7cc4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 -- 172.21.15.87:0/2760386476 shutdown_connections 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/2760386476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8108210 0x7f7cd810a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/2760386476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/2760386476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8107cd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 -- 172.21.15.87:0/2760386476 >> 172.21.15.87:0/2760386476 conn(0x7f7cd80fc9e0 msgr2=0x7f7cd80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 -- 172.21.15.87:0/2760386476 shutdown_connections 2024-06-06T07:47:17.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.504+0000 7f7cdea9b700 1 -- 172.21.15.87:0/2760386476 wait complete. 2024-06-06T07:47:17.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 Processor -- start 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 -- start start 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 0x7f7cd819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810e190 con 0x7f7cd81058e0 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810e010 con 0x7f7cd8102fb0 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdea9b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810e310 con 0x7f7cd8108210 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60652/0 (socket says 172.21.15.87:60652) 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 -- 172.21.15.87:0/53126889 learned_addr learned my addr 172.21.15.87:0/53126889 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdc837700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:17.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdd038700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 0x7f7cd819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 -- 172.21.15.87:0/53126889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 msgr2=0x7f7cd819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 0x7f7cd819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 -- 172.21.15.87:0/53126889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 msgr2=0x7f7cd81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cd7fff700 1 -- 172.21.15.87:0/53126889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd81a03c0 con 0x7f7cd81058e0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdd038700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 0x7f7cd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.505+0000 7f7cdc837700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cd7fff700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4000c00 tx=0x7f7cc4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4027070 con 0x7f7cd81058e0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd81a0650 con 0x7f7cd81058e0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc4004150 con 0x7f7cd81058e0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400f6c0 con 0x7f7cd81058e0 2024-06-06T07:47:17.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.506+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a0b20 con 0x7f7cd81058e0 2024-06-06T07:47:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:17 smithi195 ceph-mon[37040]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.507+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd8069180 con 0x7f7cd81058e0 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.508+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7cc402a070 con 0x7f7cd81058e0 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.509+0000 7f7cd5ffb700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 0x7f7cc80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.509+0000 7f7cdc837700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 0x7f7cc80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.509+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7cc40d0cd0 con 0x7f7cd81058e0 2024-06-06T07:47:17.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.510+0000 7f7cdc837700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 0x7f7cc80a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005fd0 tx=0x7f7ccc005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:17.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.513+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7cc4031020 con 0x7f7cd81058e0 2024-06-06T07:47:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:17 smithi181 ceph-mon[36790]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:17.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.726+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7cd80628d0 con 0x7f7cc80a2c20 2024-06-06T07:47:17.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.727+0000 7f7cd5ffb700 1 -- 172.21.15.87:0/53126889 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7cd80628d0 con 0x7f7cc80a2c20 2024-06-06T07:47:17.728 INFO:teuthology.orchestra.run.smithi087.stdout:Backend: cephadm 2024-06-06T07:47:17.728 INFO:teuthology.orchestra.run.smithi087.stdout:Available: Yes 2024-06-06T07:47:17.728 INFO:teuthology.orchestra.run.smithi087.stdout:Paused: No 2024-06-06T07:47:17.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 msgr2=0x7f7cc80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:17.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 0x7f7cc80a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005fd0 tx=0x7f7ccc005dc0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 msgr2=0x7f7cd8198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:17.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4000c00 tx=0x7f7cc4009ee0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 shutdown_connections 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7cc80a2c20 0x7f7cc80a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cd8102fb0 0x7f7cd81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cd8108210 0x7f7cd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 --2- 172.21.15.87:0/53126889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cd81058e0 0x7f7cd8198cf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 >> 172.21.15.87:0/53126889 conn(0x7f7cd80fc9e0 msgr2=0x7f7cd81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 shutdown_connections 2024-06-06T07:47:17.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:17.729+0000 7f7cdea9b700 1 -- 172.21.15.87:0/53126889 wait complete. 2024-06-06T07:47:18.447 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-06T07:47:18.688 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:19 smithi195 ceph-mon[37040]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:19.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:19 smithi195 ceph-mon[37040]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:19.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:19 smithi181 ceph-mon[36790]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:19.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:19 smithi181 ceph-mon[36790]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:19.600 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:19 smithi087 ceph-mon[32276]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:19.600 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:19 smithi087 ceph-mon[32276]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:20.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/1553718909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc108240 msgr2=0x7fcbdc10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/1553718909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc108240 0x7fcbdc10a650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400b0a0 tx=0x7fcbd400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/1553718909 shutdown_connections 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/1553718909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc108240 0x7fcbdc10a650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/1553718909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 0x7fcbdc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/1553718909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc105910 0x7fcbdc107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/1553718909 >> 172.21.15.87:0/1553718909 conn(0x7fcbdc0fc990 msgr2=0x7fcbdc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.472+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/1553718909 shutdown_connections 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/1553718909 wait complete. 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 Processor -- start 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 -- start start 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 0x7fcbdc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 0x7fcbdc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10e100 con 0x7fcbdc105910 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10df80 con 0x7fcbdc108240 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.473+0000 7fcbe1bb2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10e280 con 0x7fcbdc102fe0 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdb7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 0x7fcbdc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60672/0 (socket says 172.21.15.87:60672) 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 -- 172.21.15.87:0/868094978 learned_addr learned my addr 172.21.15.87:0/868094978 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdbfff700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 0x7fcbdc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 -- 172.21.15.87:0/868094978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 msgr2=0x7fcbdc1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 0x7fcbdc1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 -- 172.21.15.87:0/868094978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 msgr2=0x7fcbdc19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 0x7fcbdc19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 -- 172.21.15.87:0/868094978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc1a0320 con 0x7fcbdc105910 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdbfff700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 0x7fcbdc19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbdaffd700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c950 tx=0x7fcbd000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0007ac0 con 0x7fcbdc105910 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0004d10 con 0x7fcbdc105910 2024-06-06T07:47:20.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc1a0610 con 0x7fcbdc105910 2024-06-06T07:47:20.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000f450 con 0x7fcbdc105910 2024-06-06T07:47:20.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.474+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc1a0b50 con 0x7fcbdc105910 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.475+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbdc069180 con 0x7fcbdc105910 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.477+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbd0004750 con 0x7fcbdc105910 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.478+0000 7fcbd8ff9700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 0x7fcbc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.478+0000 7fcbdb7fe700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 0x7fcbc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.478+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fcbd00cf320 con 0x7fcbdc105910 2024-06-06T07:47:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.478+0000 7fcbdb7fe700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 0x7fcbc40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005950 tx=0x7fcbcc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:20.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.481+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcbd0014030 con 0x7fcbdc105910 2024-06-06T07:47:20.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.686+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fcbdc0628d0 con 0x7fcbc40a2c20 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.700+0000 7fcbd8ff9700 1 -- 172.21.15.87:0/868094978 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fcbdc0628d0 con 0x7fcbc40a2c20 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.a smithi087 running (3m) 35s ago 3m 14.2M - 19.0.0-2946-g3d89accf b045ac545f82 6563e6d28f65 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.b smithi181 running (3m) 80s ago 3m 28.6M - 19.0.0-2946-g3d89accf b045ac545f82 b6416607acde 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.c smithi195 running (3m) 80s ago 3m 18.2M - 19.0.0-2946-g3d89accf b045ac545f82 1291b2f08147 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.d smithi087 running (3m) 35s ago 3m 17.0M - 19.0.0-2946-g3d89accf b045ac545f82 16f81635e3bd 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.e smithi181 running (3m) 80s ago 3m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 7c6028ecd49f 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.f smithi195 running (3m) 80s ago 3m 17.1M - 19.0.0-2946-g3d89accf b045ac545f82 e585d789d9b2 2024-06-06T07:47:20.700 INFO:teuthology.orchestra.run.smithi087.stdout:mds.g smithi087 running (3m) 35s ago 3m 13.7M - 19.0.0-2946-g3d89accf b045ac545f82 7c814bdfef9d 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mds.h smithi181 running (3m) 80s ago 3m 17.6M - 19.0.0-2946-g3d89accf b045ac545f82 f1dd816f140b 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mds.i smithi195 running (3m) 80s ago 3m 20.5M - 19.0.0-2946-g3d89accf b045ac545f82 c4bbaed80256 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mds.j smithi087 running (4m) 35s ago 4m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 2d054ce0c32f 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mds.k smithi181 running (4m) 80s ago 4m 14.4M - 19.0.0-2946-g3d89accf b045ac545f82 57008d0756ab 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mds.l smithi195 running (4m) 80s ago 4m 15.0M - 19.0.0-2946-g3d89accf b045ac545f82 6b718d9adcf4 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.x smithi087 *:9283,8765 running (11m) 35s ago 11m 518M - 19.0.0-2946-g3d89accf b045ac545f82 596db31ad094 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.y smithi181 *:8443,8765 running (9m) 80s ago 9m 455M - 19.0.0-2946-g3d89accf b045ac545f82 330057fe4e47 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.z smithi195 *:8443,8765 running (9m) 80s ago 9m 453M - 19.0.0-2946-g3d89accf b045ac545f82 f0c623244cee 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mon.a smithi087 running (11m) 35s ago 11m 51.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 d7ad1a2e106c 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mon.b smithi181 running (9m) 80s ago 9m 41.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 39452bd0d8b2 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:mon.c smithi195 running (9m) 80s ago 9m 39.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 a034b190713c 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 smithi087 running (9m) 35s ago 9m 47.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 41a24b68d17b 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 smithi181 running (8m) 80s ago 8m 48.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 1de2d1bcb2ec 2024-06-06T07:47:20.701 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 smithi195 running (8m) 80s ago 8m 54.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6f6233c306f3 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 smithi087 running (7m) 35s ago 7m 55.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 243bd9af5736 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 smithi181 running (7m) 80s ago 7m 42.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d594e75dd16e 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 smithi195 running (7m) 80s ago 7m 53.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6546d8a218fd 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 smithi087 running (6m) 35s ago 6m 51.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 53b1e07a4526 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 smithi181 running (6m) 80s ago 6m 52.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 8a1dab867c0e 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.8 smithi195 running (5m) 80s ago 5m 44.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 390b739675e0 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.9 smithi087 running (5m) 35s ago 5m 43.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 968e02a61d2d 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.10 smithi181 running (4m) 80s ago 4m 44.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6b813d6a614d 2024-06-06T07:47:20.702 INFO:teuthology.orchestra.run.smithi087.stdout:osd.11 smithi195 running (4m) 80s ago 4m 49.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 2feb2349b4ca 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 msgr2=0x7fcbc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 0x7fcbc40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005950 tx=0x7fcbcc00b410 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 msgr2=0x7fcbdc198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c950 tx=0x7fcbd000cc60 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 shutdown_connections 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fcbc40a2c20 0x7fcbc40a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbdc108240 0x7fcbdc19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbdc102fe0 0x7fcbdc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 --2- 172.21.15.87:0/868094978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbdc105910 0x7fcbdc198c00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:20.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 >> 172.21.15.87:0/868094978 conn(0x7fcbdc0fc990 msgr2=0x7fcbdc103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 shutdown_connections 2024-06-06T07:47:20.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:20.702+0000 7fcbe1bb2700 1 -- 172.21.15.87:0/868094978 wait complete. 2024-06-06T07:47:21.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:21 smithi087 ceph-mon[32276]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:21.420 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-06T07:47:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:21 smithi195 ceph-mon[37040]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:21.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:21 smithi181 ceph-mon[36790]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:21.662 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:22.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:22 smithi087 ceph-mon[32276]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:22 smithi181 ceph-mon[36790]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:22.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:22 smithi195 ceph-mon[37040]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:23.393 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:23 smithi087 ceph-mon[32276]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3453210726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268105900 msgr2=0x7fb268107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3453210726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268105900 0x7fb268107cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009990 tx=0x7fb25c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3453210726 shutdown_connections 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3453210726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268102fd0 0x7fb2681053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3453210726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb26810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3453210726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268105900 0x7fb268107cf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3453210726 >> 172.21.15.87:0/3453210726 conn(0x7fb2680fc980 msgr2=0x7fb2680fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3453210726 shutdown_connections 2024-06-06T07:47:23.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.395+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3453210726 wait complete. 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 Processor -- start 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 -- start start 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268105900 0x7fb268100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb2681014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810e130 con 0x7fb268102fd0 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810dfb0 con 0x7fb268105900 2024-06-06T07:47:23.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26ce6b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810e2b0 con 0x7fb268108230 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60682/0 (socket says 172.21.15.87:60682) 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb26659c700 1 -- 172.21.15.87:0/3935589408 learned_addr learned my addr 172.21.15.87:0/3935589408 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb265d9b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268105900 0x7fb268100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.396+0000 7fb266d9d700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb2681014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 -- 172.21.15.87:0/3935589408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 msgr2=0x7fb2681014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb2681014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 -- 172.21.15.87:0/3935589408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268105900 msgr2=0x7fb268100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268105900 0x7fb268100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 -- 172.21.15.87:0/3935589408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268101d70 con 0x7fb268102fd0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb266d9d700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb2681014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26659c700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c9b0 tx=0x7fb25800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258007b00 con 0x7fb268102fd0 2024-06-06T07:47:23.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681a2830 con 0x7fb268102fd0 2024-06-06T07:47:23.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb258007ca0 con 0x7fb268102fd0 2024-06-06T07:47:23.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258018730 con 0x7fb268102fd0 2024-06-06T07:47:23.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.397+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681a2df0 con 0x7fb268102fd0 2024-06-06T07:47:23.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.398+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268069180 con 0x7fb268102fd0 2024-06-06T07:47:23.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.399+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb2580188d0 con 0x7fb268102fd0 2024-06-06T07:47:23.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.400+0000 7fb2577fe700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 0x7fb2500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:23.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.400+0000 7fb265d9b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 0x7fb2500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:23.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.400+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fb2580cf690 con 0x7fb268102fd0 2024-06-06T07:47:23.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.401+0000 7fb265d9b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 0x7fb2500a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00b5c0 tx=0x7fb25c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:23.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.404+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb258014030 con 0x7fb268102fd0 2024-06-06T07:47:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:23 smithi181 ceph-mon[36790]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:23.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.611+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb2680628d0 con 0x7fb2500a2c20 2024-06-06T07:47:23.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.617+0000 7fb2577fe700 1 -- 172.21.15.87:0/3935589408 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb2680628d0 con 0x7fb2500a2c20 2024-06-06T07:47:23.617 INFO:teuthology.orchestra.run.smithi087.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-06T07:47:23.618 INFO:teuthology.orchestra.run.smithi087.stdout:mds.all 12/12 83s ago 4m smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-06-06T07:47:23.618 INFO:teuthology.orchestra.run.smithi087.stdout:mgr 3/3 83s ago 9m smithi087=x;smithi181=y;smithi195=z;count:3 2024-06-06T07:47:23.618 INFO:teuthology.orchestra.run.smithi087.stdout:mon 3/3 83s ago 9m smithi087:172.21.15.87=a;smithi181:172.21.15.181=b;smithi195:172.21.15.195=c;count:3 2024-06-06T07:47:23.618 INFO:teuthology.orchestra.run.smithi087.stdout:osd 12 83s ago - 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 msgr2=0x7fb2500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 0x7fb2500a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00b5c0 tx=0x7fb25c005fd0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 msgr2=0x7fb268102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c9b0 tx=0x7fb25800ccc0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 shutdown_connections 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fb2500a2c20 0x7fb2500a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb268108230 0x7fb2681014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb268102fd0 0x7fb268102930 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 --2- 172.21.15.87:0/3935589408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb268105900 0x7fb268100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 >> 172.21.15.87:0/3935589408 conn(0x7fb2680fc980 msgr2=0x7fb268103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:23.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 shutdown_connections 2024-06-06T07:47:23.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:23.620+0000 7fb26ce6b700 1 -- 172.21.15.87:0/3935589408 wait complete. 2024-06-06T07:47:23.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:23 smithi195 ceph-mon[37040]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:24.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-06T07:47:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:25.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:25 smithi087 ceph-mon[32276]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:25.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:25 smithi087 ceph-mon[32276]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:25 smithi181 ceph-mon[36790]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:25.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:25 smithi181 ceph-mon[36790]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:25.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:25 smithi195 ceph-mon[37040]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:25.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:25 smithi195 ceph-mon[37040]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 -- 172.21.15.87:0/3978501176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800102510 msgr2=0x7f0800102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 --2- 172.21.15.87:0/3978501176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800102510 0x7f0800102930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f07f40099c0 tx=0x7f07f4009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 -- 172.21.15.87:0/3978501176 shutdown_connections 2024-06-06T07:47:26.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 --2- 172.21.15.87:0/3978501176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08001040b0 0x7f080010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 --2- 172.21.15.87:0/3978501176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800102510 0x7f0800102930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 --2- 172.21.15.87:0/3978501176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0800103710 0x7f0800103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 -- 172.21.15.87:0/3978501176 >> 172.21.15.87:0/3978501176 conn(0x7f08000fdab0 msgr2=0x7f08000ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 -- 172.21.15.87:0/3978501176 shutdown_connections 2024-06-06T07:47:26.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.265+0000 7f0805832700 1 -- 172.21.15.87:0/3978501176 wait complete. 2024-06-06T07:47:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.266+0000 7f0805832700 1 Processor -- start 2024-06-06T07:47:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.266+0000 7f0805832700 1 -- start start 2024-06-06T07:47:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.266+0000 7f0805832700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0800102510 0x7f08001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08001040b0 0x7f080019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e0f0 con 0x7f0800103710 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f080010df70 con 0x7f0800102510 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e270 con 0x7f08001040b0 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60706/0 (socket says 172.21.15.87:60706) 2024-06-06T07:47:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 -- 172.21.15.87:0/1980790198 learned_addr learned my addr 172.21.15.87:0/1980790198 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07feffd700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0800102510 0x7f08001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07ff7fe700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08001040b0 0x7f080019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 -- 172.21.15.87:0/1980790198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08001040b0 msgr2=0x7f080019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08001040b0 0x7f080019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 -- 172.21.15.87:0/1980790198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0800102510 msgr2=0x7f08001986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0800102510 0x7f08001986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 -- 172.21.15.87:0/1980790198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001a0310 con 0x7f0800103710 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f07fe7fc700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c9e0 tx=0x7f07f000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0007ac0 con 0x7f0800103710 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f0007c60 con 0x7f0800103710 2024-06-06T07:47:26.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0018730 con 0x7f0800103710 2024-06-06T07:47:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001a0600 con 0x7f0800103710 2024-06-06T07:47:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.267+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a0af0 con 0x7f0800103710 2024-06-06T07:47:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.269+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800069180 con 0x7f0800103710 2024-06-06T07:47:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.270+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07f00188d0 con 0x7f0800103710 2024-06-06T07:47:26.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.270+0000 7f0804830700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 0x7f07e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:26.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.270+0000 7f07feffd700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 0x7f07e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:26.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.271+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f07f00cf3a0 con 0x7f0800103710 2024-06-06T07:47:26.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.271+0000 7f07feffd700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 0x7f07e80a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07f40039c0 tx=0x7f07f4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:26.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.277+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f07f0014030 con 0x7f0800103710 2024-06-06T07:47:26.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.481+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f08000628d0 con 0x7f07e80a2c20 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.483+0000 7f0804830700 1 -- 172.21.15.87:0/1980790198 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f08000628d0 con 0x7f07e80a2c20 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stdout:HOST ADDR LABELS STATUS 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 172.21.15.87 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stdout:smithi181 172.21.15.181 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stdout:smithi195 172.21.15.195 2024-06-06T07:47:26.483 INFO:teuthology.orchestra.run.smithi087.stdout:3 hosts in cluster 2024-06-06T07:47:26.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 msgr2=0x7f07e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:26.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 0x7f07e80a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07f40039c0 tx=0x7f07f4005fb0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 msgr2=0x7f0800198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c9e0 tx=0x7f07f000ccf0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 shutdown_connections 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f07e80a2c20 0x7f07e80a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0800102510 0x7f08001986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08001040b0 0x7f080019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 --2- 172.21.15.87:0/1980790198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0800103710 0x7f0800198bf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 >> 172.21.15.87:0/1980790198 conn(0x7f08000fdab0 msgr2=0x7f0800106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 shutdown_connections 2024-06-06T07:47:26.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:26.485+0000 7f0805832700 1 -- 172.21.15.87:0/1980790198 wait complete. 2024-06-06T07:47:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:26 smithi181 ceph-mon[36790]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:26 smithi087 ceph-mon[32276]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:26 smithi195 ceph-mon[37040]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:27.113 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-06T07:47:27.357 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:27 smithi181 ceph-mon[36790]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:27.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:27 smithi087 ceph-mon[32276]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:27.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:27 smithi195 ceph-mon[37040]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:28.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:28 smithi087 ceph-mon[32276]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:28 smithi181 ceph-mon[36790]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:28 smithi195 ceph-mon[37040]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:29.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 -- 172.21.15.87:0/3579100587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 msgr2=0x7ff954103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/3579100587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954103b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff9440099c0 tx=0x7ff944009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 -- 172.21.15.87:0/3579100587 shutdown_connections 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/3579100587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9541024f0 0x7ff954102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/3579100587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff954104090 0x7ff95410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/3579100587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954103b50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.033+0000 7ff95b7d3700 1 -- 172.21.15.87:0/3579100587 >> 172.21.15.87:0/3579100587 conn(0x7ff9540fda90 msgr2=0x7ff9540ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- 172.21.15.87:0/3579100587 shutdown_connections 2024-06-06T07:47:29.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- 172.21.15.87:0/3579100587 wait complete. 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 Processor -- start 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- start start 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9541024f0 0x7ff954076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff954104090 0x7ff954074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410e1d0 con 0x7ff9541036f0 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410e050 con 0x7ff954104090 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff95b7d3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410e350 con 0x7ff9541024f0 2024-06-06T07:47:29.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff958d6e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.034+0000 7ff958d6e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40284/0 (socket says 172.21.15.87:40284) 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 -- 172.21.15.87:0/303447363 learned_addr learned my addr 172.21.15.87:0/303447363 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff959d70700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff954104090 0x7ff954074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff95956f700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9541024f0 0x7ff954076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 -- 172.21.15.87:0/303447363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9541024f0 msgr2=0x7ff954076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9541024f0 0x7ff954076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 -- 172.21.15.87:0/303447363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff954104090 msgr2=0x7ff954074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff954104090 0x7ff954074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 -- 172.21.15.87:0/303447363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff954075670 con 0x7ff9541036f0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff958d6e700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff944003890 tx=0x7ff9440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944027070 con 0x7ff9541036f0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff944004150 con 0x7ff9541036f0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a2990 con 0x7ff9541036f0 2024-06-06T07:47:29.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400f460 con 0x7ff9541036f0 2024-06-06T07:47:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.035+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a2e10 con 0x7ff9541036f0 2024-06-06T07:47:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.036+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff954069180 con 0x7ff9541036f0 2024-06-06T07:47:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.037+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff94402a070 con 0x7ff9541036f0 2024-06-06T07:47:29.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.038+0000 7ff94a7fc700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 0x7ff9400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.038+0000 7ff95956f700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 0x7ff9400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.038+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff9440d0500 con 0x7ff9541036f0 2024-06-06T07:47:29.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.039+0000 7ff95956f700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 0x7ff9400a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff954103550 tx=0x7ff950009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:29.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.041+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff944013070 con 0x7ff9541036f0 2024-06-06T07:47:29.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.246+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff9540628d0 con 0x7ff9400a2c20 2024-06-06T07:47:29.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.247+0000 7ff94a7fc700 1 -- 172.21.15.87:0/303447363 <== mgr.14150 v2:172.21.15.87:6800/2460003046 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff9540628d0 con 0x7ff9400a2c20 2024-06-06T07:47:29.248 INFO:teuthology.orchestra.run.smithi087.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-06T07:47:29.248 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NQ400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-06T07:47:29.248 INFO:teuthology.orchestra.run.smithi087.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T07:47:29.248 INFO:teuthology.orchestra.run.smithi087.stdout:smithi195 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300FS480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-06-06T07:47:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 msgr2=0x7ff9400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 0x7ff9400a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff954103550 tx=0x7ff950009190 comp rx=0 tx=0).stop 2024-06-06T07:47:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 msgr2=0x7ff954074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:29.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff944003890 tx=0x7ff9440047b0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 shutdown_connections 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7ff9400a2c20 0x7ff9400a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff954104090 0x7ff954074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9541024f0 0x7ff954076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 --2- 172.21.15.87:0/303447363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9541036f0 0x7ff954074880 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 >> 172.21.15.87:0/303447363 conn(0x7ff9540fda90 msgr2=0x7ff954106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 shutdown_connections 2024-06-06T07:47:29.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:29.250+0000 7ff95b7d3700 1 -- 172.21.15.87:0/303447363 wait complete. 2024-06-06T07:47:29.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:29 smithi087 ceph-mon[32276]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:29.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:29 smithi181 ceph-mon[36790]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:29.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:29 smithi195 ceph-mon[37040]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:47:29.968 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T07:47:29.978 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:47:29.978 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-06T07:47:30.216 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:30 smithi181 ceph-mon[36790]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:30.568 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:30 smithi087 ceph-mon[32276]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:30.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:30 smithi195 ceph-mon[37040]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:31.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2194299104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec4104090 msgr2=0x7f9ec410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2194299104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec4104090 0x7f9ec410a940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b0d0 tx=0x7f9ebc00b3e0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2194299104 shutdown_connections 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2194299104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2194299104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec4104090 0x7f9ec410a940 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2194299104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec41036f0 0x7f9ec4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2194299104 >> 172.21.15.87:0/2194299104 conn(0x7f9ec40fda90 msgr2=0x7f9ec40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2194299104 shutdown_connections 2024-06-06T07:47:31.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.912+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2194299104 wait complete. 2024-06-06T07:47:31.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 Processor -- start 2024-06-06T07:47:31.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 -- start start 2024-06-06T07:47:31.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:31.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:31.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec4104090 0x7f9ec419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410e180 con 0x7f9ec41036f0 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410e000 con 0x7f9ec41024f0 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec8e9d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec410e300 con 0x7f9ec4104090 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40304/0 (socket says 172.21.15.87:40304) 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.913+0000 7f9ec1d9b700 1 -- 172.21.15.87:0/2487933220 learned_addr learned my addr 172.21.15.87:0/2487933220 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec259c700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec2d9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec4104090 0x7f9ec419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 -- 172.21.15.87:0/2487933220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec4104090 msgr2=0x7f9ec419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec4104090 0x7f9ec419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 -- 172.21.15.87:0/2487933220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 msgr2=0x7f9ec4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 -- 172.21.15.87:0/2487933220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a0390 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec259c700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec1d9b700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c9a0 tx=0x7f9eb400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4007a60 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004d10 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a0680 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400f450 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.914+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a0c40 con 0x7f9ec41036f0 2024-06-06T07:47:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.915+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec4069180 con 0x7f9ec41036f0 2024-06-06T07:47:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.916+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9eb4004750 con 0x7f9ec41036f0 2024-06-06T07:47:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.917+0000 7f9eb37fe700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 0x7f9eac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.917+0000 7f9ec259c700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 0x7f9eac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.917+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9eb40cf4c0 con 0x7f9ec41036f0 2024-06-06T07:47:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.918+0000 7f9ec259c700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 0x7f9eac0a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4103550 tx=0x7f9eb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:31.921+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9eb409c150 con 0x7f9ec41036f0 2024-06-06T07:47:32.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.164+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9ec402dcd0 con 0x7f9ec41036f0 2024-06-06T07:47:32.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.164+0000 7f9eb37fe700 1 -- 172.21.15.87:0/2487933220 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3052 (secure 0 0 0) 0x7f9eb409ba50 con 0x7f9ec41036f0 2024-06-06T07:47:32.164 INFO:teuthology.orchestra.run.smithi087.stdout:e26 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:legacy client fscid: 1 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:Filesystem 'cephfs' (1) 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:fs_name cephfs 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 26 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-06-06T07:44:02.139989+0000 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:modified 2024-06-06T07:45:07.358878+0000 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:tableserver 0 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:root 0 2024-06-06T07:47:32.165 INFO:teuthology.orchestra.run.smithi087.stdout:session_timeout 300 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:session_autoclose 300 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:max_file_size 1099511627776 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:max_xattr_size 65536 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:required_client_features {} 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:last_failure 0 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:last_failure_osd_epoch 0 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:max_mds 3 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:in 0,1,2 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:up {0=24467,1=24448,2=24446} 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:failed 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:damaged 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:stopped 2024-06-06T07:47:32.166 INFO:teuthology.orchestra.run.smithi087.stdout:data_pools [3,4] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:metadata_pool 2 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:inline_data disabled 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:balancer 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:bal_rank_mask -1 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:standby_count_wanted 0 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:qdb_cluster leader: 24467 members: 24448,24446,24467 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b{0:24467} state up:active seq 6 addr [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c{0:24430} state up:standby-replay seq 1 addr [v2:172.21.15.195:6836/494555841,v1:172.21.15.195:6837/494555841] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.i{1:24448} state up:active seq 23 addr [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.e{1:24422} state up:standby-replay seq 1 addr [v2:172.21.15.181:6834/1343084853,v1:172.21.15.181:6835/1343084853] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.h{2:24446} state up:active seq 24 addr [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.f{2:24421} state up:standby-replay seq 1 addr [v2:172.21.15.195:6834/922595027,v1:172.21.15.195:6835/922595027] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:32.167 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:Standby daemons: 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.87:6834/2593068869,v1:172.21.15.87:6835/2593068869] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.87:6836/2477022406,v1:172.21.15.87:6837/2477022406] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.87:6838/200596410,v1:172.21.15.87:6839/200596410] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a{-1:14607} state up:standby seq 1 addr [v2:172.21.15.87:6840/3223638025,v1:172.21.15.87:6841/3223638025] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.l{-1:24394} state up:standby seq 1 addr [v2:172.21.15.195:6832/2222795089,v1:172.21.15.195:6833/2222795089] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.168 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.k{-1:24401} state up:standby seq 1 addr [v2:172.21.15.181:6832/2618702621,v1:172.21.15.181:6833/2618702621] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 msgr2=0x7f9eac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 0x7f9eac0a5160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4103550 tx=0x7f9eb8008040 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 msgr2=0x7f9ec4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c9a0 tx=0x7f9eb400cd60 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 shutdown_connections 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9eac0a2ca0 0x7f9eac0a5160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ec41024f0 0x7f9ec4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ec4104090 0x7f9ec419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 --2- 172.21.15.87:0/2487933220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ec41036f0 0x7f9ec4198c70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.167+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 >> 172.21.15.87:0/2487933220 conn(0x7f9ec40fda90 msgr2=0x7f9ec41069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:32.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.168+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 shutdown_connections 2024-06-06T07:47:32.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:32.168+0000 7f9ec8e9d700 1 -- 172.21.15.87:0/2487933220 wait complete. 2024-06-06T07:47:32.170 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:47:32.488 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:32 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2487933220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:47:32.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:32 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2487933220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:47:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:32 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2487933220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:47:32.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-06T07:47:33.097 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:33.475 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:33 smithi087 ceph-mon[32276]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:33 smithi195 ceph-mon[37040]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:33.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:33 smithi181 ceph-mon[36790]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.748+0000 7fdea7d5e700 1 -- 172.21.15.87:0/3912396424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102600 msgr2=0x7fdea0102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.748+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/3912396424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102600 0x7fdea0102a60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0099c0 tx=0x7fde9c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- 172.21.15.87:0/3912396424 shutdown_connections 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/3912396424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102fa0 0x7fdea0109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/3912396424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 0x7fdea0101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/3912396424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102600 0x7fdea0102a60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- 172.21.15.87:0/3912396424 >> 172.21.15.87:0/3912396424 conn(0x7fdea00fc9c0 msgr2=0x7fdea00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- 172.21.15.87:0/3912396424 shutdown_connections 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- 172.21.15.87:0/3912396424 wait complete. 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 Processor -- start 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- start start 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 0x7fdea0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102600 0x7fdea0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdea010cff0 con 0x7fdea0102fa0 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdea010ce70 con 0x7fdea0102600 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea7d5e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdea010d170 con 0x7fdea0101400 2024-06-06T07:47:34.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea62fb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.749+0000 7fdea62fb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40330/0 (socket says 172.21.15.87:40330) 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 -- 172.21.15.87:0/2672328365 learned_addr learned my addr 172.21.15.87:0/2672328365 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea5afa700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 0x7fdea0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea52f9700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102600 0x7fdea0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 -- 172.21.15.87:0/2672328365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 msgr2=0x7fdea0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 0x7fdea0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 -- 172.21.15.87:0/2672328365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102600 msgr2=0x7fdea0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102600 0x7fdea0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 -- 172.21.15.87:0/2672328365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea01a0390 con 0x7fdea0102fa0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea62fb700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fde98007f10 tx=0x7fde9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98004bb0 con 0x7fdea0102fa0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde98004d50 con 0x7fdea0102fa0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea01a0680 con 0x7fdea0102fa0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde980059f0 con 0x7fdea0102fa0 2024-06-06T07:47:34.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.750+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea01a0b70 con 0x7fdea0102fa0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.751+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea0069180 con 0x7fdea0102fa0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.752+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde9801a020 con 0x7fdea0102fa0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.753+0000 7fde96ffd700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 0x7fde8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.753+0000 7fdea5afa700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 0x7fde8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.753+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fde980cf210 con 0x7fdea0102fa0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.753+0000 7fdea5afa700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 0x7fde8c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdea0102460 tx=0x7fde9000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.756+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde98010070 con 0x7fdea0102fa0 2024-06-06T07:47:34.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.952+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fdea01a0f90 con 0x7fdea0102fa0 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.953+0000 7fde96ffd700 1 -- 172.21.15.87:0/2672328365 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5742 (secure 0 0 0) 0x7fde9809bd80 con 0x7fdea0102fa0 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 99 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-06-06T07:35:49.599649+0000 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:modified 2024-06-06T07:45:48.222161+0000 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-06T07:47:34.953 INFO:teuthology.orchestra.run.smithi087.stdout:crush_version 27 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:full_ratio 0.95 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:backfillfull_ratio 0.9 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:nearfull_ratio 0.85 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:require_min_compat_client luminous 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:min_compat_client luminous 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:require_osd_release squid 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:stretch_mode_enabled false 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-06T07:47:34.954 INFO:teuthology.orchestra.run.smithi087.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:max_osd 12 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 up in weight 1 up_from 8 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6802/3682387176,v1:172.21.15.87:6803/3682387176] [v2:172.21.15.87:6804/3682387176,v1:172.21.15.87:6805/3682387176] exists,up 1cb21c7c-fbf5-4e45-9644-1e793e30f06d 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 up in weight 1 up_from 13 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/1976649118,v1:172.21.15.181:6801/1976649118] [v2:172.21.15.181:6802/1976649118,v1:172.21.15.181:6803/1976649118] exists,up 144fe682-6bd8-4452-8e0c-cb40af20028b 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/2910401977,v1:172.21.15.195:6801/2910401977] [v2:172.21.15.195:6802/2910401977,v1:172.21.15.195:6803/2910401977] exists,up cabc61d6-3e07-4e07-96bf-474a45e3c752 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6810/4203203594,v1:172.21.15.87:6811/4203203594] [v2:172.21.15.87:6812/4203203594,v1:172.21.15.87:6813/4203203594] exists,up 1da26cda-ee26-4b26-b7c8-fed45cdac09a 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/3265557015,v1:172.21.15.181:6809/3265557015] [v2:172.21.15.181:6810/3265557015,v1:172.21.15.181:6811/3265557015] exists,up 3da9aeb5-12e1-46c8-8369-e1ee920ca615 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 up in weight 1 up_from 38 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/1086493667,v1:172.21.15.195:6809/1086493667] [v2:172.21.15.195:6810/1086493667,v1:172.21.15.195:6811/1086493667] exists,up 3fd8d26c-d2e3-4542-9624-1d5c7cbeb684 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 up in weight 1 up_from 46 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6818/2277253418,v1:172.21.15.87:6819/2277253418] [v2:172.21.15.87:6820/2277253418,v1:172.21.15.87:6821/2277253418] exists,up 12b3728b-ea86-47f0-8f77-13631e3eded7 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 up in weight 1 up_from 51 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/1036398045,v1:172.21.15.181:6817/1036398045] [v2:172.21.15.181:6818/1036398045,v1:172.21.15.181:6819/1036398045] exists,up 2403cabf-c28a-4c74-b355-6c695506c49f 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.8 up in weight 1 up_from 59 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/2643069924,v1:172.21.15.195:6817/2643069924] [v2:172.21.15.195:6818/2643069924,v1:172.21.15.195:6819/2643069924] exists,up 3720aa4b-ec90-4411-b392-93bdbe855dac 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.9 up in weight 1 up_from 67 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6826/2016619295,v1:172.21.15.87:6827/2016619295] [v2:172.21.15.87:6828/2016619295,v1:172.21.15.87:6829/2016619295] exists,up e7b1b810-b726-42ff-9bea-814b83c404f1 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.10 up in weight 1 up_from 72 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/649796316,v1:172.21.15.181:6825/649796316] [v2:172.21.15.181:6826/649796316,v1:172.21.15.181:6827/649796316] exists,up 1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c 2024-06-06T07:47:34.955 INFO:teuthology.orchestra.run.smithi087.stdout:osd.11 up in weight 1 up_from 80 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/1273097292,v1:172.21.15.195:6825/1273097292] [v2:172.21.15.195:6826/1273097292,v1:172.21.15.195:6827/1273097292] exists,up 53318824-a6a1-4e13-8840-030854cc881d 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.c [2,1] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.e [2,4] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.f [8,11] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.10 [8,11] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.6 [3,7] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.10 [5,9] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.14 [4,6] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.1d [4,8] 2024-06-06T07:47:34.956 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.25 [5,9] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.31 [4,8] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.37 [1,9] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 4.5 [9,10] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 4.6 [9,11] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 4.10 [9,8] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 4.15 [5,10] 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6800/4176793720 expires 2024-06-07T07:36:20.511068+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3853061787 expires 2024-06-07T07:36:46.051849+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3813763622 expires 2024-06-07T07:36:20.511068+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6800/1605115594 expires 2024-06-07T07:36:46.051849+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6801/4176793720 expires 2024-06-07T07:36:20.511068+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/1264886975 expires 2024-06-07T07:36:46.051849+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/2314391466 expires 2024-06-07T07:36:20.511068+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3870018525 expires 2024-06-07T07:36:20.511068+0000 2024-06-06T07:47:34.957 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6801/1605115594 expires 2024-06-07T07:36:46.051849+0000 2024-06-06T07:47:34.958 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/2887993384 expires 2024-06-07T07:36:46.051849+0000 2024-06-06T07:47:34.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 msgr2=0x7fde8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 0x7fde8c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdea0102460 tx=0x7fde9000b410 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 msgr2=0x7fdea019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fde98007f10 tx=0x7fde9800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 shutdown_connections 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fde8c0a2c20 0x7fde8c0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdea0102fa0 0x7fdea019fcf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea0101400 0x7fdea0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 --2- 172.21.15.87:0/2672328365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea0102600 0x7fdea0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 >> 172.21.15.87:0/2672328365 conn(0x7fdea00fc9c0 msgr2=0x7fdea00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 shutdown_connections 2024-06-06T07:47:34.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:34.955+0000 7fdea7d5e700 1 -- 172.21.15.87:0/2672328365 wait complete. 2024-06-06T07:47:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:35 smithi195 ceph-mon[37040]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:35.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:35 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2672328365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:47:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:35 smithi181 ceph-mon[36790]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:35.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:35 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2672328365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:47:35.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:35 smithi087 ceph-mon[32276]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:35.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:35 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2672328365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:47:35.655 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-06T07:47:35.668 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:47:35.906 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:37.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:37 smithi087 ceph-mon[32276]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:37.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:37 smithi195 ceph-mon[37040]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:37 smithi181 ceph-mon[36790]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 -- 172.21.15.87:0/4213533258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806b3c0 msgr2=0x7f8fb810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 --2- 172.21.15.87:0/4213533258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806b3c0 0x7f8fb810f310 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009990 tx=0x7f8fb0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 -- 172.21.15.87:0/4213533258 shutdown_connections 2024-06-06T07:47:37.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 --2- 172.21.15.87:0/4213533258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806b3c0 0x7f8fb810f310 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 --2- 172.21.15.87:0/4213533258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb81084f0 0x7f8fb81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 --2- 172.21.15.87:0/4213533258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb806aa00 0x7f8fb806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 -- 172.21.15.87:0/4213533258 >> 172.21.15.87:0/4213533258 conn(0x7f8fb8078300 msgr2=0x7f8fb8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:37.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 -- 172.21.15.87:0/4213533258 shutdown_connections 2024-06-06T07:47:37.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.665+0000 7f8fc018a700 1 -- 172.21.15.87:0/4213533258 wait complete. 2024-06-06T07:47:37.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 Processor -- start 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 -- start start 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb806b3c0 0x7f8fb819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb81084f0 0x7f8fb819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81127d0 con 0x7f8fb806aa00 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8112650 con 0x7f8fb81084f0 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fc018a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8112950 con 0x7f8fb806b3c0 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fbdf26700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fbdf26700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39734/0 (socket says 172.21.15.87:39734) 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.666+0000 7f8fbdf26700 1 -- 172.21.15.87:0/1457968571 learned_addr learned my addr 172.21.15.87:0/1457968571 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbd725700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb806b3c0 0x7f8fb819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbe727700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb81084f0 0x7f8fb819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 -- 172.21.15.87:0/1457968571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb806b3c0 msgr2=0x7f8fb819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb806b3c0 0x7f8fb819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 -- 172.21.15.87:0/1457968571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb81084f0 msgr2=0x7f8fb819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb81084f0 0x7f8fb819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 -- 172.21.15.87:0/1457968571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819fe40 con 0x7f8fb806aa00 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fbdf26700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ba50 tx=0x7f8fa800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8019070 con 0x7f8fb806aa00 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a0130 con 0x7f8fb806aa00 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa8014440 con 0x7f8fb806aa00 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80075d0 con 0x7f8fb806aa00 2024-06-06T07:47:37.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.667+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a79b0 con 0x7f8fb806aa00 2024-06-06T07:47:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.668+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8069180 con 0x7f8fb806aa00 2024-06-06T07:47:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.670+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fa801c070 con 0x7f8fb806aa00 2024-06-06T07:47:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.670+0000 7f8fb6ffd700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 0x7f8fa00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.670+0000 7f8fbd725700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 0x7f8fa00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:37.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.670+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f8fa80ceda0 con 0x7f8fb806aa00 2024-06-06T07:47:37.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.671+0000 7f8fbd725700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 0x7f8fa00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8fac005fd0 tx=0x7f8fac00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.674+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fa809b910 con 0x7f8fb806aa00 2024-06-06T07:47:37.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.912+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb819c190 con 0x7f8fb806aa00 2024-06-06T07:47:37.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.912+0000 7f8fb6ffd700 1 -- 172.21.15.87:0/1457968571 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f8fb819c190 con 0x7f8fb806aa00 2024-06-06T07:47:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:37.913 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 msgr2=0x7f8fa00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 0x7f8fa00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8fac005fd0 tx=0x7f8fac00a560 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 msgr2=0x7f8fb819ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ba50 tx=0x7f8fa800be10 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 shutdown_connections 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f8fa00a2c20 0x7f8fa00a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fb81084f0 0x7f8fb819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fb806aa00 0x7f8fb819ae30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 --2- 172.21.15.87:0/1457968571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb806b3c0 0x7f8fb819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:37.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 >> 172.21.15.87:0/1457968571 conn(0x7f8fb8078300 msgr2=0x7f8fb810d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:37.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 shutdown_connections 2024-06-06T07:47:37.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:37.915+0000 7f8fc018a700 1 -- 172.21.15.87:0/1457968571 wait complete. 2024-06-06T07:47:37.916 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:47:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:38 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1457968571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:38.558 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:38 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1457968571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:38.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:47:38.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:38 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1457968571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:38.837 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:39 smithi181 ceph-mon[36790]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:39.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:39 smithi087 ceph-mon[32276]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:39.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:39 smithi195 ceph-mon[37040]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/4283745703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 msgr2=0x7f9ac410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/4283745703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac410e230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b0a0 tx=0x7f9abc00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/4283745703 shutdown_connections 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/4283745703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac41073f0 0x7f9ac41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/4283745703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac410e230 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/4283745703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac4101380 0x7f9ac4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/4283745703 >> 172.21.15.87:0/4283745703 conn(0x7f9ac4078300 msgr2=0x7f9ac4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/4283745703 shutdown_connections 2024-06-06T07:47:40.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.647+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/4283745703 wait complete. 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.648+0000 7f9ac8f0a700 1 Processor -- start 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.648+0000 7f9ac8f0a700 1 -- start start 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac4101380 0x7f9ac41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac41073f0 0x7f9ac41a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41116e0 con 0x7f9ac4101d40 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4111560 con 0x7f9ac4101380 2024-06-06T07:47:40.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac8f0a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4111860 con 0x7f9ac41073f0 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39750/0 (socket says 172.21.15.87:39750) 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 -- 172.21.15.87:0/3653777732 learned_addr learned my addr 172.21.15.87:0/3653777732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac259c700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac4101380 0x7f9ac41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac2d9d700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac41073f0 0x7f9ac41a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 -- 172.21.15.87:0/3653777732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac41073f0 msgr2=0x7f9ac41a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac41073f0 0x7f9ac41a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 -- 172.21.15.87:0/3653777732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac4101380 msgr2=0x7f9ac41999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac4101380 0x7f9ac41999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 -- 172.21.15.87:0/3653777732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac41a4fa0 con 0x7f9ac4101d40 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ac1d9b700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b530 tx=0x7f9ab800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.649+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8015020 con 0x7f9ac4101d40 2024-06-06T07:47:40.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.650+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab8011d10 con 0x7f9ac4101d40 2024-06-06T07:47:40.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.650+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac41a51d0 con 0x7f9ac4101d40 2024-06-06T07:47:40.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.650+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800f2e0 con 0x7f9ac4101d40 2024-06-06T07:47:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.650+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac41a5690 con 0x7f9ac4101d40 2024-06-06T07:47:40.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.650+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac4069180 con 0x7f9ac4101d40 2024-06-06T07:47:40.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.652+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ab8018020 con 0x7f9ac4101d40 2024-06-06T07:47:40.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.652+0000 7f9ab37fe700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 0x7f9aac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:40.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.652+0000 7f9ac259c700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 0x7f9aac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:40.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.652+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f9ab80cef00 con 0x7f9ac4101d40 2024-06-06T07:47:40.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.653+0000 7f9ac259c700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 0x7f9aac0a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400ac50 tx=0x7f9ab400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:40.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.656+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ab801f020 con 0x7f9ac4101d40 2024-06-06T07:47:40.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.894+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ac41a5950 con 0x7f9ac4101d40 2024-06-06T07:47:40.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.895+0000 7f9ab37fe700 1 -- 172.21.15.87:0/3653777732 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f9ab809ba70 con 0x7f9ac4101d40 2024-06-06T07:47:40.895 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:40.895 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:45:07.358878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 msgr2=0x7f9aac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 0x7f9aac0a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400ac50 tx=0x7f9ab400a380 comp rx=0 tx=0).stop 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 msgr2=0x7f9ac4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b530 tx=0x7f9ab800b840 comp rx=0 tx=0).stop 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 shutdown_connections 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f9aac0a2c20 0x7f9aac0a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ac4101380 0x7f9ac41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac41073f0 0x7f9ac41a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 --2- 172.21.15.87:0/3653777732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ac4101d40 0x7f9ac4199f20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 >> 172.21.15.87:0/3653777732 conn(0x7f9ac4078300 msgr2=0x7f9ac40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 shutdown_connections 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:40.897+0000 7f9ac8f0a700 1 -- 172.21.15.87:0/3653777732 wait complete. 2024-06-06T07:47:40.898 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-06-06T07:47:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:41 smithi181 ceph-mon[36790]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:41 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3653777732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:41.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:41 smithi087 ceph-mon[32276]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:41.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3653777732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:41.602 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:47:41.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:41 smithi195 ceph-mon[37040]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:41.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:41 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3653777732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:47:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:42.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:42 smithi087 ceph-mon[32276]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:42.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:42 smithi195 ceph-mon[37040]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:42.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:42 smithi181 ceph-mon[36790]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:43.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 -- 172.21.15.87:0/3985244547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4102000 msgr2=0x7f0cd4105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:43.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/3985244547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4102000 0x7f0cd4105fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0099c0 tx=0x7f0cbc009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 -- 172.21.15.87:0/3985244547 shutdown_connections 2024-06-06T07:47:43.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/3985244547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/3985244547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd41066a0 0x7f0cd410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/3985244547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4102000 0x7f0cd4105fc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 -- 172.21.15.87:0/3985244547 >> 172.21.15.87:0/3985244547 conn(0x7f0cd40773e0 msgr2=0x7f0cd40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:43.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 -- 172.21.15.87:0/3985244547 shutdown_connections 2024-06-06T07:47:43.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.576+0000 7f0cd89e7700 1 -- 172.21.15.87:0/3985244547 wait complete. 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 Processor -- start 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 -- start start 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd4102000 0x7f0cd4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4111680 con 0x7f0cd41066a0 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4111500 con 0x7f0cd4101650 2024-06-06T07:47:43.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd89e7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4111800 con 0x7f0cd4102000 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.577+0000 7f0cd2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39758/0 (socket says 172.21.15.87:39758) 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 -- 172.21.15.87:0/4116922997 learned_addr learned my addr 172.21.15.87:0/4116922997 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd259c700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0ccbfff700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd4102000 0x7f0cd4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 -- 172.21.15.87:0/4116922997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd4102000 msgr2=0x7f0cd4199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd4102000 0x7f0cd4199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 -- 172.21.15.87:0/4116922997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 msgr2=0x7f0cd41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 -- 172.21.15.87:0/4116922997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd41a4f80 con 0x7f0cd41066a0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd259c700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd2d9d700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007ee0 tx=0x7f0ccc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00dbd0 con 0x7f0cd41066a0 2024-06-06T07:47:43.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc00dd70 con 0x7f0cd41066a0 2024-06-06T07:47:43.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd41a51b0 con 0x7f0cd41066a0 2024-06-06T07:47:43.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0133f0 con 0x7f0cd41066a0 2024-06-06T07:47:43.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.578+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a56c0 con 0x7f0cd41066a0 2024-06-06T07:47:43.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.579+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd4069180 con 0x7f0cd41066a0 2024-06-06T07:47:43.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.580+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ccc01a020 con 0x7f0cd41066a0 2024-06-06T07:47:43.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.581+0000 7f0ccb7fe700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 0x7f0cc00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:43.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.581+0000 7f0cd259c700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 0x7f0cc00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:43.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.581+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f0ccc0ceb70 con 0x7f0cd41066a0 2024-06-06T07:47:43.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.581+0000 7f0cd259c700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 0x7f0cc00a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005fd0 tx=0x7f0cc4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:43.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.585+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ccc010070 con 0x7f0cd41066a0 2024-06-06T07:47:43.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.783+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cd41a5890 con 0x7f0cd41066a0 2024-06-06T07:47:43.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.785+0000 7f0ccb7fe700 1 -- 172.21.15.87:0/4116922997 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24916 (secure 0 0 0) 0x7f0ccc09b6e0 con 0x7f0cd41066a0 2024-06-06T07:47:43.785 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:47:43.785 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":99,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:45:48.222161+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 msgr2=0x7f0cc00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 0x7f0cc00a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005fd0 tx=0x7f0cc4005e30 comp rx=0 tx=0).stop 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 msgr2=0x7f0cd41a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007ee0 tx=0x7f0ccc00d3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 shutdown_connections 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f0cc00a2c20 0x7f0cc00a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cd4101650 0x7f0cd41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd41066a0 0x7f0cd41a4a40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 --2- 172.21.15.87:0/4116922997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cd4102000 0x7f0cd4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:43.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 >> 172.21.15.87:0/4116922997 conn(0x7f0cd40773e0 msgr2=0x7f0cd40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:43.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 shutdown_connections 2024-06-06T07:47:43.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:43.787+0000 7f0cd89e7700 1 -- 172.21.15.87:0/4116922997 wait complete. 2024-06-06T07:47:44.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:43 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4116922997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:44.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:43 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4116922997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:43 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4116922997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:47:44.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T07:44:02.139989+0000', 'modified': '2024-06-06T07:45:07.358878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24467, 'mds_1': 24448, 'mds_2': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/922595027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 922595027}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 922595027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/1343084853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1343084853}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 1343084853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/494555841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 494555841}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 494555841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.181:6837/3077028235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3077028235}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3077028235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.195:6839/203076359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 203076359}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 203076359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/851837058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 851837058}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 851837058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24467, 'qdb_cluster': [24448, 24446, 24467]} 2024-06-06T07:47:44.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T07:47:44.481 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T07:47:44.721 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:45.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:44 smithi087 ceph-mon[32276]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:45.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:44 smithi195 ceph-mon[37040]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:45.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:44 smithi181 ceph-mon[36790]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:46.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 -- 172.21.15.87:0/3011456291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 msgr2=0x7f7b341077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 --2- 172.21.15.87:0/3011456291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b341077b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009990 tx=0x7f7b28009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 -- 172.21.15.87:0/3011456291 shutdown_connections 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 --2- 172.21.15.87:0/3011456291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 0x7f7b3410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 --2- 172.21.15.87:0/3011456291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b341077b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 --2- 172.21.15.87:0/3011456291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 -- 172.21.15.87:0/3011456291 >> 172.21.15.87:0/3011456291 conn(0x7f7b340782c0 msgr2=0x7f7b340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 -- 172.21.15.87:0/3011456291 shutdown_connections 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.460+0000 7f7b3b097700 1 -- 172.21.15.87:0/3011456291 wait complete. 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 Processor -- start 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 -- start start 2024-06-06T07:47:46.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b3419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 0x7f7b3419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111690 con 0x7f7b341073d0 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111510 con 0x7f7b34101390 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b3b097700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111810 con 0x7f7b34101d30 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b39634700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b39634700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39768/0 (socket says 172.21.15.87:39768) 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b39634700 1 -- 172.21.15.87:0/31983454 learned_addr learned my addr 172.21.15.87:0/31983454 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b38e33700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b3419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.461+0000 7f7b33fff700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 0x7f7b3419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 -- 172.21.15.87:0/31983454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 msgr2=0x7f7b3419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 0x7f7b3419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 -- 172.21.15.87:0/31983454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 msgr2=0x7f7b3419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b3419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 -- 172.21.15.87:0/31983454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b341990d0 con 0x7f7b341073d0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b38e33700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b3419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b39634700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b970 tx=0x7f7b2c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b31ffb700 1 -- 172.21.15.87:0/31983454 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c019070 con 0x7f7b341073d0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b31ffb700 1 -- 172.21.15.87:0/31983454 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2c00ce10 con 0x7f7b341073d0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b341993c0 con 0x7f7b341073d0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b31ffb700 1 -- 172.21.15.87:0/31983454 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c007370 con 0x7f7b341073d0 2024-06-06T07:47:46.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.462+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341a5680 con 0x7f7b341073d0 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.464+0000 7f7b31ffb700 1 -- 172.21.15.87:0/31983454 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b2c01c070 con 0x7f7b341073d0 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.465+0000 7f7b31ffb700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 0x7f7b240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.465+0000 7f7b38e33700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 0x7f7b240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.465+0000 7f7b31ffb700 1 -- 172.21.15.87:0/31983454 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7b2c0ce800 con 0x7f7b341073d0 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.465+0000 7f7b38e33700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 0x7f7b240a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b34199ab0 tx=0x7f7b28005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 Processor -- start 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 -- start start 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 0x5631478b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5631477a1f10 con 0x5631478b5830 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563147800800 con 0x5631478b4f10 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563147768410 con 0x5631478af750 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39784/0 (socket says 172.21.15.87:39784) 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 -- 172.21.15.87:0/853464616 learned_addr learned my addr 172.21.15.87:0/853464616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 -- 172.21.15.87:0/853464616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 msgr2=0x5631478afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b38e33700 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 0x5631478b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 -- 172.21.15.87:0/853464616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 msgr2=0x5631478b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 0x5631478b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b33fff700 1 -- 172.21.15.87:0/853464616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314774f880 con 0x5631478b5830 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.466+0000 7f7b38e33700 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 0x5631478b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b33fff700 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7b20008c30 tx=0x7f7b20008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad03aa86e35dd6c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b1effd700 1 -- 172.21.15.87:0/853464616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20014070 con 0x5631478b5830 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b1effd700 1 -- 172.21.15.87:0/853464616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2000ee20 con 0x5631478b5830 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b1effd700 1 -- 172.21.15.87:0/853464616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20010630 con 0x5631478b5830 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b4591ab80 1 -- 172.21.15.87:0/853464616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 msgr2=0x5631478af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.467+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7b20008c30 tx=0x7f7b20008f40 comp rx=0 tx=0).stop 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- 172.21.15.87:0/853464616 shutdown_connections 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b4f10 0x5631478b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/853464616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b5830 0x5631478af210 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- 172.21.15.87:0/853464616 >> 172.21.15.87:0/853464616 conn(0x5631477dc030 msgr2=0x5631478aa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- 172.21.15.87:0/853464616 shutdown_connections 2024-06-06T07:47:46.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- 172.21.15.87:0/853464616 wait complete. 2024-06-06T07:47:46.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- start start 2024-06-06T07:47:46.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 Processor -- start 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5631477a0f70 con 0x5631478b4f10 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5631477fc7c0 con 0x5631478b5830 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.468+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5631478bc210 con 0x5631478af750 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b4591ab80 10 client.0 _ll_register_callbacks cb 0x563147796560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39800/0 (socket says 172.21.15.87:39800) 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 -- 172.21.15.87:0/3481303733 learned_addr learned my addr 172.21.15.87:0/3481303733 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b39634700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b38e33700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 -- 172.21.15.87:0/3481303733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 msgr2=0x5631478bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478bb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 -- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 msgr2=0x5631478b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 -- 172.21.15.87:0/3481303733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631477c6700 con 0x5631478b4f10 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b33fff700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000f010 tx=0x7f7b20004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fb0329a13138a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b13fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20027060 con 0x5631478b4f10 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7b2000ec20 con 0x5631478b4f10 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2000ee20 con 0x5631478b4f10 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b2002e050 con 0x5631478b4f10 2024-06-06T07:47:46.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b4591ab80 10 client.15177 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.469+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631478c6980 con 0x5631478b4f10 2024-06-06T07:47:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2000ee20 con 0x5631478b4f10 2024-06-06T07:47:46.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7b20018410 con 0x5631478b4f10 2024-06-06T07:47:46.475 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:46 smithi087 ceph-mon[32276]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 10 client.15177 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:47:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 20 client.15177 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5631478c7d80 0x5631478c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 4 client.15177 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56314780dd00 con 0x5631478c7d80 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.470+0000 7f7b38e33700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5631478c7d80 0x5631478c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.471+0000 7f7b38e33700 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5631478c7d80 0x5631478c9f70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4642fc10968eaba server_cookie=d2768f30018e5c3f in_seq=0 out_seq=0 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.471+0000 7f7b13fff700 10 client.15177 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.471+0000 7f7b13fff700 1 -- 172.21.15.87:0/3481303733 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56314780dd00 con 0x5631478c7d80 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.471+0000 7f7b13fff700 10 client.15177 handle_command_reply: tid=0 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.471+0000 7f7b4591ab80 1 client.15177 shutdown 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 msgr2=0x5631478b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000f010 tx=0x7f7b20004ed0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 shutdown_connections 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5631478c7d80 0x5631478c9f70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478af750 0x5631478bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/3481303733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478b8920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 >> 172.21.15.87:0/3481303733 conn(0x5631477dc030 msgr2=0x5631478a96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b13fff700 0 client.15177 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 shutdown_connections 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 1 -- 172.21.15.87:0/3481303733 wait complete. 2024-06-06T07:47:46.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.472+0000 7f7b4591ab80 20 client.15177 trim_cache size 0 max 16384 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 Processor -- start 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 -- start start 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478af750 0x5631478b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b5830 0x5631478aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5631478c82b0 con 0x5631478b4f10 2024-06-06T07:47:46.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5631478c8980 con 0x5631478af750 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5631478c9050 con 0x5631478b5830 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39810/0 (socket says 172.21.15.87:39810) 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 -- 172.21.15.87:0/1214232270 learned_addr learned my addr 172.21.15.87:0/1214232270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 -- 172.21.15.87:0/1214232270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b5830 msgr2=0x5631478aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b5830 0x5631478aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 -- 172.21.15.87:0/1214232270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478af750 msgr2=0x5631478b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b38e33700 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478af750 0x5631478b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478af750 0x5631478b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.494+0000 7f7b33fff700 1 -- 172.21.15.87:0/1214232270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631478b6900 con 0x5631478b4f10 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b33fff700 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000c00 tx=0x7f7b200083b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83a20687f65a1de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b1e7fc700 1 -- 172.21.15.87:0/1214232270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20027070 con 0x5631478b4f10 2024-06-06T07:47:46.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b1e7fc700 1 -- 172.21.15.87:0/1214232270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b200053b0 con 0x5631478b4f10 2024-06-06T07:47:46.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b1e7fc700 1 -- 172.21.15.87:0/1214232270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20014070 con 0x5631478b4f10 2024-06-06T07:47:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 -- 172.21.15.87:0/1214232270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 msgr2=0x5631478a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000c00 tx=0x7f7b200083b0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 -- 172.21.15.87:0/1214232270 shutdown_connections 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478af750 0x5631478b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b5830 0x5631478aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/1214232270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478b4f10 0x5631478a9e30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.495+0000 7f7b4591ab80 1 -- 172.21.15.87:0/1214232270 >> 172.21.15.87:0/1214232270 conn(0x5631477dc030 msgr2=0x5631478b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- 172.21.15.87:0/1214232270 shutdown_connections 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- 172.21.15.87:0/1214232270 wait complete. 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- start start 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 Processor -- start 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b4f10 0x5631478a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5631478c9df0 con 0x5631478af750 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5631478c9720 con 0x5631478b5830 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5631478ca4c0 con 0x5631478b4f10 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b4591ab80 10 client.0 _ll_register_callbacks cb 0x563147796560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.496+0000 7f7b38e33700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39816/0 (socket says 172.21.15.87:39816) 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 -- 172.21.15.87:0/2323031710 learned_addr learned my addr 172.21.15.87:0/2323031710 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b33fff700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b4f10 0x5631478a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b39634700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 -- 172.21.15.87:0/2323031710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b4f10 msgr2=0x5631478a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b4f10 0x5631478a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 -- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 msgr2=0x5631478d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 -- 172.21.15.87:0/2323031710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631478d2a80 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b38e33700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7b280092e0 tx=0x7f7b2801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a686171b409836ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b13fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800fde0 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f7b28032440 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b200053b0 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b28043050 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b4591ab80 10 client.15183 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631478d3dc0 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.497+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b200053b0 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7b28024710 con 0x5631478af750 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 10 client.15183 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 10 client.15183 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 20 client.15183 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:46.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x563147938b10 0x563147937820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 4 client.15183 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "status"}) -- 0x563147808280 con 0x563147938b10 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b33fff700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x563147938b10 0x563147937820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b33fff700 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x563147938b10 0x563147937820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35e74e76f2044dfd server_cookie=aca0ed05bfd1125a in_seq=0 out_seq=0 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.498+0000 7f7b13fff700 10 client.15183 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b13fff700 1 -- 172.21.15.87:0/2323031710 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563147808280 con 0x563147938b10 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b13fff700 10 client.15183 handle_command_reply: tid=0 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 client.15183 shutdown 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 msgr2=0x5631478a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7b280092e0 tx=0x7f7b2801a050 comp rx=0 tx=0).stop 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 shutdown_connections 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5631478b5830 0x5631478d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x563147938b10 0x563147937820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5631478af750 0x5631478a9040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 --2- 172.21.15.87:0/2323031710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631478b4f10 0x5631478a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 >> 172.21.15.87:0/2323031710 conn(0x5631477dc030 msgr2=0x5631478ba150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b13fff700 0 client.15183 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 shutdown_connections 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 1 -- 172.21.15.87:0/2323031710 wait complete. 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.499+0000 7f7b4591ab80 20 client.15183 trim_cache size 0 max 16384 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-06-06T07:47:46.505 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24446, 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 164.276057167, 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 253.692134291 2024-06-06T07:47:46.506 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 msgr2=0x7f7b240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 0x7f7b240a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b34199ab0 tx=0x7f7b28005960 comp rx=0 tx=0).stop 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 msgr2=0x7f7b34198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b970 tx=0x7f7b2c00bc80 comp rx=0 tx=0).stop 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 shutdown_connections 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f7b240a2c20 0x7f7b240a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b34101390 0x7f7b3419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b341073d0 0x7f7b34198970 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 --2- 172.21.15.87:0/31983454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b34101d30 0x7f7b3419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:46.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 >> 172.21.15.87:0/31983454 conn(0x7f7b340782c0 msgr2=0x7f7b341005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:46.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 shutdown_connections 2024-06-06T07:47:46.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:46.500+0000 7f7b3b097700 1 -- 172.21.15.87:0/31983454 wait complete. 2024-06-06T07:47:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:46 smithi195 ceph-mon[37040]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:46 smithi181 ceph-mon[36790]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T07:47:47.282 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T07:47:47.529 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:47.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:47 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:47:47.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:47 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:47:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:47 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:47:48.726 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:48.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:48.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:48.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:48 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:48.727 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:48 smithi087 ceph-mon[32276]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:48 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:48 smithi195 ceph-mon[37040]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:48 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:48 smithi181 ceph-mon[36790]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 -- 172.21.15.87:0/1504345550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 msgr2=0x7f6ff01089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/1504345550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff01089a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00099c0 tx=0x7f6fe0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 -- 172.21.15.87:0/1504345550 shutdown_connections 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/1504345550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102ee0 0x7f6ff010f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/1504345550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff01089a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/1504345550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102520 0x7f6ff01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 -- 172.21.15.87:0/1504345550 >> 172.21.15.87:0/1504345550 conn(0x7f6ff0079680 msgr2=0x7f6ff0079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 -- 172.21.15.87:0/1504345550 shutdown_connections 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.255+0000 7f6ff83bd700 1 -- 172.21.15.87:0/1504345550 wait complete. 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 Processor -- start 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 -- start start 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102520 0x7f6ff019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102ee0 0x7f6ff019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0112810 con 0x7f6ff01085c0 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0112690 con 0x7f6ff0102520 2024-06-06T07:47:49.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff83bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0112990 con 0x7f6ff0102ee0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff695a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff695a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46884/0 (socket says 172.21.15.87:46884) 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff695a700 1 -- 172.21.15.87:0/3198040872 learned_addr learned my addr 172.21.15.87:0/3198040872 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff6159700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102520 0x7f6ff019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.256+0000 7f6ff5958700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102ee0 0x7f6ff019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 -- 172.21.15.87:0/3198040872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102ee0 msgr2=0x7f6ff019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102ee0 0x7f6ff019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 -- 172.21.15.87:0/3198040872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102520 msgr2=0x7f6ff019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102520 0x7f6ff019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 -- 172.21.15.87:0/3198040872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01991f0 con 0x7f6ff01085c0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff695a700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007ee0 tx=0x7f6fe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6fe77fe700 1 -- 172.21.15.87:0/3198040872 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800ddd0 con 0x7f6ff01085c0 2024-06-06T07:47:49.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01994e0 con 0x7f6ff01085c0 2024-06-06T07:47:49.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6fe77fe700 1 -- 172.21.15.87:0/3198040872 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe8004020 con 0x7f6ff01085c0 2024-06-06T07:47:49.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6fe77fe700 1 -- 172.21.15.87:0/3198040872 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8015660 con 0x7f6ff01085c0 2024-06-06T07:47:49.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.257+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01a57a0 con 0x7f6ff01085c0 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.259+0000 7f6fe77fe700 1 -- 172.21.15.87:0/3198040872 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fe80041d0 con 0x7f6ff01085c0 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.260+0000 7f6fe77fe700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 0x7f6fdc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.260+0000 7f6ff6159700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 0x7f6fdc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f6fe77fe700 1 -- 172.21.15.87:0/3198040872 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6fe80cef00 con 0x7f6ff01085c0 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 Processor -- start 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f6ff6159700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 0x7f6fdc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 -- start start 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e0aed0 0x556e69e0b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e69d5c7c0 con 0x556e69e10f10 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e69d046d0 con 0x556e69e105f0 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f7002c40b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e69d58780 con 0x556e69e0aed0 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f6ff5958700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f6ff5958700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46892/0 (socket says 172.21.15.87:46892) 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.261+0000 7f6ff5958700 1 -- 172.21.15.87:0/4026092331 learned_addr learned my addr 172.21.15.87:0/4026092331 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff6159700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff695a700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e0aed0 0x556e69e0b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 -- 172.21.15.87:0/4026092331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e0aed0 msgr2=0x556e69e0b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e0aed0 0x556e69e0b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 -- 172.21.15.87:0/4026092331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 msgr2=0x556e69e109d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e109d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 -- 172.21.15.87:0/4026092331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e69cfaf00 con 0x556e69e10f10 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff6159700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6ff5958700 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00bd00 tx=0x7f6fec00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6610133bba7eb2a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6fe4d70700 1 -- 172.21.15.87:0/4026092331 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec015070 con 0x556e69e10f10 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6fe4d70700 1 -- 172.21.15.87:0/4026092331 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fec00be50 con 0x556e69e10f10 2024-06-06T07:47:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.262+0000 7f6fe4d70700 1 -- 172.21.15.87:0/4026092331 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011440 con 0x556e69e10f10 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 -- 172.21.15.87:0/4026092331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 msgr2=0x556e69e0a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00bd00 tx=0x7f6fec00f640 comp rx=0 tx=0).stop 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 -- 172.21.15.87:0/4026092331 shutdown_connections 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e0aed0 0x556e69e0b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 --2- 172.21.15.87:0/4026092331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e10f10 0x556e69e0a990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 -- 172.21.15.87:0/4026092331 >> 172.21.15.87:0/4026092331 conn(0x556e69d35c70 msgr2=0x556e69e05800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 -- 172.21.15.87:0/4026092331 shutdown_connections 2024-06-06T07:47:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.263+0000 7f7002c40b80 1 -- 172.21.15.87:0/4026092331 wait complete. 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 -- start start 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 Processor -- start 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e0aed0 0x556e69e16c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e14490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e69e17980 con 0x556e69e105f0 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e69cfcba0 con 0x556e69e0aed0 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e69e17b00 con 0x556e69e10f10 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f7002c40b80 10 client.0 _ll_register_callbacks cb 0x556e69cf68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46908/0 (socket says 172.21.15.87:46908) 2024-06-06T07:47:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 -- 172.21.15.87:0/1327418763 learned_addr learned my addr 172.21.15.87:0/1327418763 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e0aed0 0x556e69e16c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff695a700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e14490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 -- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 msgr2=0x556e69e14490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e14490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 -- 172.21.15.87:0/1327418763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e0aed0 msgr2=0x556e69e16c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e0aed0 0x556e69e16c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.264+0000 7f6ff5958700 1 -- 172.21.15.87:0/1327418763 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e69cab880 con 0x556e69e105f0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005450 tx=0x7f6fec0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45fb326fc597412 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec015030 con 0x556e69e105f0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6fec004750 con 0x556e69e105f0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fec00be50 con 0x556e69e105f0 2024-06-06T07:47:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6fec02d050 con 0x556e69e105f0 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 10 client.15195 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e69e22290 con 0x556e69e105f0 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fec00be50 con 0x556e69e105f0 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6fec011bf0 con 0x556e69e105f0 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.265+0000 7f7002c40b80 20 client.15195 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e244d0 0x556e69e25940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f7002c40b80 4 client.15195 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e69d69cc0 con 0x556e69e244d0 2024-06-06T07:47:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e244d0 0x556e69e25940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e244d0 0x556e69e25940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bf2db1e3a55052c server_cookie=e666ac69a3eb9177 in_seq=0 out_seq=0 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.266+0000 7f6fd9ffb700 10 client.15195 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1327418763 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e69d69cc0 con 0x556e69e244d0 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f6fd9ffb700 10 client.15195 handle_command_reply: tid=0 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 client.15195 shutdown 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 msgr2=0x556e69e13f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005450 tx=0x7f6fec0056e0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 shutdown_connections 2024-06-06T07:47:49.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e244d0 0x556e69e25940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e0aed0 0x556e69e16c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e14490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1327418763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e105f0 0x556e69e13f50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 >> 172.21.15.87:0/1327418763 conn(0x556e69d35c70 msgr2=0x556e69e09980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.267+0000 7f6fd9ffb700 0 client.15195 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.268+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 shutdown_connections 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.268+0000 7f7002c40b80 1 -- 172.21.15.87:0/1327418763 wait complete. 2024-06-06T07:47:49.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.268+0000 7f7002c40b80 20 client.15195 trim_cache size 0 max 16384 2024-06-06T07:47:49.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 Processor -- start 2024-06-06T07:47:49.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 -- start start 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e105f0 0x556e69e144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 0x556e69e112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e69e23c80 con 0x556e69e0aed0 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e69e24350 con 0x556e69e10f10 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f7002c40b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e69e24a20 con 0x556e69e105f0 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f6ff6159700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f6ff6159700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46918/0 (socket says 172.21.15.87:46918) 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f6ff6159700 1 -- 172.21.15.87:0/1273751594 learned_addr learned my addr 172.21.15.87:0/1273751594 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.289+0000 7f6ff695a700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 0x556e69e112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e105f0 0x556e69e144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 -- 172.21.15.87:0/1273751594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e105f0 msgr2=0x556e69e144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e105f0 0x556e69e144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 -- 172.21.15.87:0/1273751594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 msgr2=0x556e69e112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 0x556e69e112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 -- 172.21.15.87:0/1273751594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e69e064b0 con 0x556e69e0aed0 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff695a700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 0x556e69e112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00098e0 tx=0x7f6fe0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd89b60e420f1d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6fdbfff700 1 -- 172.21.15.87:0/1273751594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000f720 con 0x556e69e0aed0 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6fdbfff700 1 -- 172.21.15.87:0/1273751594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe000fda0 con 0x556e69e0aed0 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.290+0000 7f6fdbfff700 1 -- 172.21.15.87:0/1273751594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe003cda0 con 0x556e69e0aed0 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- 172.21.15.87:0/1273751594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 msgr2=0x556e69e0a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00098e0 tx=0x7f6fe0023000 comp rx=0 tx=0).stop 2024-06-06T07:47:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- 172.21.15.87:0/1273751594 shutdown_connections 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e10f10 0x556e69e112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e0a990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1273751594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e105f0 0x556e69e144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- 172.21.15.87:0/1273751594 >> 172.21.15.87:0/1273751594 conn(0x556e69d35c70 msgr2=0x556e69e12840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- 172.21.15.87:0/1273751594 shutdown_connections 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- 172.21.15.87:0/1273751594 wait complete. 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- start start 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 Processor -- start 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e13f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.291+0000 7f7002c40b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e69e257c0 con 0x556e69e0aed0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f7002c40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e69e250f0 con 0x556e69e105f0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f7002c40b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e69e25e90 con 0x556e69e10f10 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f7002c40b80 10 client.0 _ll_register_callbacks cb 0x556e69cf68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46924/0 (socket says 172.21.15.87:46924) 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 -- 172.21.15.87:0/1772107444 learned_addr learned my addr 172.21.15.87:0/1772107444 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff695a700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e13f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 -- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 msgr2=0x556e69e13f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e13f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 -- 172.21.15.87:0/1772107444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 msgr2=0x556e69e13890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e13890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 -- 172.21.15.87:0/1772107444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e69e2d150 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6ff6159700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009f70 tx=0x7f6fe003c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e498850a2039d9b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6fd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe003ca20 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe000fda0 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f6fe003cbc0 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.292+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6fe004e050 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e69e942f0 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe000fda0 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6fe003a020 con 0x556e69e0aed0 2024-06-06T07:47:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 20 client.15201 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e988f0 0x556e69e97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 4 client.15201 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "status"}) -- 0x556e69d64240 con 0x556e69e988f0 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.293+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e988f0 0x556e69e97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f6fd9ffb700 10 client.15201 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f6ff5958700 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e988f0 0x556e69e97600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f07a6848a6c30743 server_cookie=9a535d51c4dc598c in_seq=0 out_seq=0 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f6fd9ffb700 1 -- 172.21.15.87:0/1772107444 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556e69d64240 con 0x556e69e988f0 2024-06-06T07:47:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f6fd9ffb700 10 client.15201 handle_command_reply: tid=0 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 client.15201 shutdown 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 msgr2=0x556e69e2f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009f70 tx=0x7f6fe003c440 comp rx=0 tx=0).stop 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 shutdown_connections 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e69e10f10 0x556e69e13f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e69e0aed0 0x556e69e2f5c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x556e69e988f0 0x556e69e97600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 --2- 172.21.15.87:0/1772107444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e69e105f0 0x556e69e13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 >> 172.21.15.87:0/1772107444 conn(0x556e69d35c70 msgr2=0x556e69dfedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f6fd9ffb700 0 client.15201 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 shutdown_connections 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 1 -- 172.21.15.87:0/1772107444 wait complete. 2024-06-06T07:47:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.294+0000 7f7002c40b80 20 client.15201 trim_cache size 0 max 16384 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24448, 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 168.08019466600001, 2024-06-06T07:47:49.303 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:49.304 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:49.304 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:49.304 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 253.62955784600001 2024-06-06T07:47:49.304 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 msgr2=0x7f6fdc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 0x7f6fdc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 msgr2=0x7f6ff0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007ee0 tx=0x7f6fe800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 shutdown_connections 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f6fdc0a2c20 0x7f6fdc0a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff0102520 0x7f6ff019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ff01085c0 0x7f6ff0198a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 --2- 172.21.15.87:0/3198040872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0102ee0 0x7f6ff019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 >> 172.21.15.87:0/3198040872 conn(0x7f6ff0079680 msgr2=0x7f6ff01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 shutdown_connections 2024-06-06T07:47:49.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:49.295+0000 7f6ff83bd700 1 -- 172.21.15.87:0/3198040872 wait complete. 2024-06-06T07:47:50.036 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:47:50.283 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:50 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:50 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:50 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:51.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:51 smithi087 ceph-mon[32276]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:47:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:47:51.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:51 smithi087 ceph-mon[32276]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:51 smithi195 ceph-mon[37040]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:47:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:47:51.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:51 smithi195 ceph-mon[37040]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:51 smithi181 ceph-mon[36790]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T07:47:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:47:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:47:51.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:51 smithi181 ceph-mon[36790]: from='mgr.14150 172.21.15.87:0/1422442698' entity='mgr.x' 2024-06-06T07:47:52.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/2796109730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541063c0 msgr2=0x7f4c541067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/2796109730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541063c0 0x7f4c541067a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000b0a0 tx=0x7f4c5000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/2796109730 shutdown_connections 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/2796109730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541024b0 0x7f4c5410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/2796109730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541063c0 0x7f4c541067a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/2796109730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 0x7f4c54101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/2796109730 >> 172.21.15.87:0/2796109730 conn(0x7f4c54077990 msgr2=0x7f4c540fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/2796109730 shutdown_connections 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.073+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/2796109730 wait complete. 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 Processor -- start 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 -- start start 2024-06-06T07:47:52.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 0x7f4c5419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 0x7f4c541978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54111710 con 0x7f4c541024b0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54111590 con 0x7f4c541063c0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c5b4a5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54111890 con 0x7f4c54101af0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46936/0 (socket says 172.21.15.87:46936) 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 -- 172.21.15.87:0/348969100 learned_addr learned my addr 172.21.15.87:0/348969100 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c59241700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 0x7f4c5419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 -- 172.21.15.87:0/348969100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 msgr2=0x7f4c5419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 0x7f4c5419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c58a40700 1 -- 172.21.15.87:0/348969100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 msgr2=0x7f4c541978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.074+0000 7f4c59a42700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 0x7f4c541978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c58a40700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 0x7f4c541978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c58a40700 1 -- 172.21.15.87:0/348969100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c541980d0 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c59a42700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 0x7f4c541978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c58a40700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b530 tx=0x7f4c4400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c4a7fc700 1 -- 172.21.15.87:0/348969100 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44015020 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c4a7fc700 1 -- 172.21.15.87:0/348969100 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c44011d10 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c541983c0 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c4a7fc700 1 -- 172.21.15.87:0/348969100 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44010e50 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.075+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a4530 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.077+0000 7f4c4a7fc700 1 -- 172.21.15.87:0/348969100 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c44018020 con 0x7f4c541024b0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.078+0000 7f4c4a7fc700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 0x7f4c400a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.078+0000 7f4c59241700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 0x7f4c400a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.078+0000 7f4c4a7fc700 1 -- 172.21.15.87:0/348969100 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4c440ced10 con 0x7f4c541024b0 2024-06-06T07:47:52.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.078+0000 7f4c59241700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 0x7f4c400a5280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000bf50 tx=0x7f4c500060a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 Processor -- start 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 -- start start 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 0x5647b56cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5647b55c1f10 con 0x5647b56cf750 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647b5620800 con 0x5647b56d4f10 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647b5588410 con 0x5647b56d5830 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46946/0 (socket says 172.21.15.87:46946) 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 -- 172.21.15.87:0/2336199465 learned_addr learned my addr 172.21.15.87:0/2336199465 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59241700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c58a40700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 0x5647b56cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 -- 172.21.15.87:0/2336199465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 msgr2=0x5647b56cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 0x5647b56cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 -- 172.21.15.87:0/2336199465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 msgr2=0x5647b56d52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c59a42700 1 -- 172.21.15.87:0/2336199465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b556f880 con 0x5647b56cf750 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.079+0000 7f4c58a40700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 0x5647b56cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.080+0000 7f4c59a42700 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00bbf0 tx=0x7f4c4c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d8916ccf0588fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.080+0000 7f4c3f7fe700 1 -- 172.21.15.87:0/2336199465 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00ddb0 con 0x5647b56cf750 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.080+0000 7f4c3f7fe700 1 -- 172.21.15.87:0/2336199465 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c004500 con 0x5647b56cf750 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.080+0000 7f4c3f7fe700 1 -- 172.21.15.87:0/2336199465 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00f680 con 0x5647b56cf750 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- 172.21.15.87:0/2336199465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 msgr2=0x5647b56cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00bbf0 tx=0x7f4c4c00bf00 comp rx=0 tx=0).stop 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- 172.21.15.87:0/2336199465 shutdown_connections 2024-06-06T07:47:52.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56cf750 0x5647b56cfb70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/2336199465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56d5830 0x5647b56cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- 172.21.15.87:0/2336199465 >> 172.21.15.87:0/2336199465 conn(0x5647b55fc030 msgr2=0x5647b56ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- 172.21.15.87:0/2336199465 shutdown_connections 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- 172.21.15.87:0/2336199465 wait complete. 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- start start 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 Processor -- start 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5647b55c0f70 con 0x5647b56d5830 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647b561c7c0 con 0x5647b56d4f10 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.081+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647b56dc210 con 0x5647b56cf750 2024-06-06T07:47:52.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c65d28b80 10 client.0 _ll_register_callbacks cb 0x5647b55b6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46956/0 (socket says 172.21.15.87:46956) 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59241700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 -- 172.21.15.87:0/1899910443 learned_addr learned my addr 172.21.15.87:0/1899910443 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c58a40700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 -- 172.21.15.87:0/1899910443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 msgr2=0x5647b56db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 -- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 msgr2=0x5647b56d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 -- 172.21.15.87:0/1899910443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b55e6700 con 0x5647b56d5830 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c59a42700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c003da0 tx=0x7f4c4c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f77081761a7dd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:52.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c024020 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4c4c003a50 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c004500 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c02f050 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c65d28b80 10 client.15213 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.082+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647b56e6910 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c004500 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4c4c017af0 con 0x5647b56d5830 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 10 client.15213 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 20 client.15213 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b56e8b50 0x5647b56e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 4 client.15213 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:47:52.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b562dd00 con 0x5647b56e8b50 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.083+0000 7f4c59241700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b56e8b50 0x5647b56e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c3cff9700 10 client.15213 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c59241700 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b56e8b50 0x5647b56e9fc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4296239e13defd server_cookie=d1b8af53a11dd03f in_seq=0 out_seq=0 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c3cff9700 1 -- 172.21.15.87:0/1899910443 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647b562dd00 con 0x5647b56e8b50 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c3cff9700 10 client.15213 handle_command_reply: tid=0 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c65d28b80 1 client.15213 shutdown 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 msgr2=0x5647b56d8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.084+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c003da0 tx=0x7f4c4c005e30 comp rx=0 tx=0).stop 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 shutdown_connections 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b56e8b50 0x5647b56e9fc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56d8e40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/1899910443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 >> 172.21.15.87:0/1899910443 conn(0x5647b55fc030 msgr2=0x5647b56c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c3cff9700 0 client.15213 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 shutdown_connections 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 1 -- 172.21.15.87:0/1899910443 wait complete. 2024-06-06T07:47:52.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.085+0000 7f4c65d28b80 20 client.15213 trim_cache size 0 max 16384 2024-06-06T07:47:52.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.105+0000 7f4c65d28b80 1 Processor -- start 2024-06-06T07:47:52.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 -- start start 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5647b56e8300 con 0x5647b56d5830 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647b56e89d0 con 0x5647b56d4f10 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647b56e90a0 con 0x5647b56cf750 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46964/0 (socket says 172.21.15.87:46964) 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 -- 172.21.15.87:0/4061087825 learned_addr learned my addr 172.21.15.87:0/4061087825 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59241700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c58a40700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 -- 172.21.15.87:0/4061087825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 msgr2=0x5647b56cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 -- 172.21.15.87:0/4061087825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 msgr2=0x5647b56d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 -- 172.21.15.87:0/4061087825 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b56cb7b0 con 0x5647b56d5830 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c58a40700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59241700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c59a42700 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42db72dc1f3712c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.106+0000 7f4c3effd700 1 -- 172.21.15.87:0/4061087825 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c038070 con 0x5647b56d5830 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c3effd700 1 -- 172.21.15.87:0/4061087825 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c00d6d0 con 0x5647b56d5830 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c3effd700 1 -- 172.21.15.87:0/4061087825 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c024070 con 0x5647b56d5830 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 -- 172.21.15.87:0/4061087825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 msgr2=0x5647b56c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c00bf00 comp rx=0 tx=0).stop 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 -- 172.21.15.87:0/4061087825 shutdown_connections 2024-06-06T07:47:52.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d5830 0x5647b56c9e30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/4061087825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d4f10 0x5647b56d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 -- 172.21.15.87:0/4061087825 >> 172.21.15.87:0/4061087825 conn(0x5647b55fc030 msgr2=0x5647b56d5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 -- 172.21.15.87:0/4061087825 shutdown_connections 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.107+0000 7f4c65d28b80 1 -- 172.21.15.87:0/4061087825 wait complete. 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 -- start start 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 Processor -- start 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 0x5647b56f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5647b56e9e40 con 0x5647b56d4f10 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647b56e9770 con 0x5647b56d5830 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647b56ea510 con 0x5647b56cf750 2024-06-06T07:47:52.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c58a40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c65d28b80 10 client.0 _ll_register_callbacks cb 0x5647b55b6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c58a40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46970/0 (socket says 172.21.15.87:46970) 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c58a40700 1 -- 172.21.15.87:0/819837457 learned_addr learned my addr 172.21.15.87:0/819837457 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c59a42700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 0x5647b56f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c58a40700 1 -- 172.21.15.87:0/819837457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 msgr2=0x5647b56c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.108+0000 7f4c58a40700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c58a40700 1 -- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 msgr2=0x5647b56f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c58a40700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 0x5647b56f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c58a40700 1 -- 172.21.15.87:0/819837457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b56f29f0 con 0x5647b56d4f10 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c59a42700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 0x5647b56f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c58a40700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400bcf0 tx=0x7f4c440108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bafee4a87cd6c46 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440b4060 con 0x5647b56d4f10 2024-06-06T07:47:52.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f4c4409f150 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c00d6d0 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c440a3df0 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 10 client.15219 fetch_fsmap learned FSMap version 26 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647b56f3cc0 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c00d6d0 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4c440a39f0 con 0x5647b56d4f10 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 20 client.15219 populate_metadata read hostname 'smithi087' 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b5758ad0 0x5647b57577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 4 client.15219 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.109+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "status"}) -- 0x5647b5628280 con 0x5647b5758ad0 2024-06-06T07:47:52.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c59241700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b5758ad0 0x5647b57577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c3cff9700 10 client.15219 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c59241700 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b5758ad0 0x5647b57577e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be50fcda74cfb711 server_cookie=6eaf4d22b326ad40 in_seq=0 out_seq=0 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c3cff9700 1 -- 172.21.15.87:0/819837457 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5647b5628280 con 0x5647b5758ad0 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c3cff9700 10 client.15219 handle_command_reply: tid=0 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 client.15219 shutdown 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 msgr2=0x5647b56c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400bcf0 tx=0x7f4c440108f0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 shutdown_connections 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5647b5758ad0 0x5647b57577e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647b56d5830 0x5647b56f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.110+0000 7f4c3cff9700 0 client.15219 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647b56cf750 0x5647b56c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 1 --2- 172.21.15.87:0/819837457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5647b56d4f10 0x5647b56c9540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 >> 172.21.15.87:0/819837457 conn(0x5647b55fc030 msgr2=0x5647b56d7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 shutdown_connections 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 1 -- 172.21.15.87:0/819837457 wait complete. 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c65d28b80 20 client.15219 trim_cache size 0 max 16384 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-06-06T07:47:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24467, 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 229.78008400300001, 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 26, 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 99, 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 246.53900420400001 2024-06-06T07:47:52.118 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 msgr2=0x7f4c400a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 0x7f4c400a5280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000bf50 tx=0x7f4c500060a0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 msgr2=0x7f4c5419dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b530 tx=0x7f4c4400b8f0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 shutdown_connections 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f4c400a2dc0 0x7f4c400a5280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c541063c0 0x7f4c541978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c54101af0 0x7f4c5419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 --2- 172.21.15.87:0/348969100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c541024b0 0x7f4c5419dc10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.111+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 >> 172.21.15.87:0/348969100 conn(0x7f4c54077990 msgr2=0x7f4c54104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.112+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 shutdown_connections 2024-06-06T07:47:52.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.112+0000 7f4c5b4a5700 1 -- 172.21.15.87:0/348969100 wait complete. 2024-06-06T07:47:52.809 INFO:teuthology.run_tasks:Running task sequential... 2024-06-06T07:47:52.819 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-06T07:47:52.819 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-06T07:47:52.819 INFO:teuthology.task.print:Enabling mgr modules 2024-06-06T07:47:52.819 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-06T07:47:52.820 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:47:52.820 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:47:52.820 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 -- 172.21.15.87:0/3437563612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab581024b0 msgr2=0x7fab58108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 --2- 172.21.15.87:0/3437563612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab581024b0 0x7fab58108d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fab50009900 tx=0x7fab5002a410 comp rx=0 tx=0).stop 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 -- 172.21.15.87:0/3437563612 shutdown_connections 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 --2- 172.21.15.87:0/3437563612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab581024b0 0x7fab58108d40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 --2- 172.21.15.87:0/3437563612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab58101af0 0x7fab58101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 --2- 172.21.15.87:0/3437563612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab58100900 0x7fab58100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 -- 172.21.15.87:0/3437563612 >> 172.21.15.87:0/3437563612 conn(0x7fab580fbe70 msgr2=0x7fab580fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:52.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 -- 172.21.15.87:0/3437563612 shutdown_connections 2024-06-06T07:47:52.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.923+0000 7fab5c970640 1 -- 172.21.15.87:0/3437563612 wait complete. 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 Processor -- start 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 -- start start 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab58101af0 0x7fab58198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab581024b0 0x7fab5819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab5810d700 con 0x7fab58100900 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab5810d580 con 0x7fab581024b0 2024-06-06T07:47:52.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5c970640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab5810d880 con 0x7fab58101af0 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46988/0 (socket says 172.21.15.87:46988) 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 -- 172.21.15.87:0/2152486000 learned_addr learned my addr 172.21.15.87:0/2152486000 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 -- 172.21.15.87:0/2152486000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab58101af0 msgr2=0x7fab58198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab58101af0 0x7fab58198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 -- 172.21.15.87:0/2152486000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab581024b0 msgr2=0x7fab5819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab581024b0 0x7fab5819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 -- 172.21.15.87:0/2152486000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5819f860 con 0x7fab58100900 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab5659c640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b9c0 tx=0x7fab4800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.924+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48019070 con 0x7fab58100900 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.925+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5819fb50 con 0x7fab58100900 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.925+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4800cba0 con 0x7fab58100900 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.925+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48012620 con 0x7fab58100900 2024-06-06T07:47:52.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.925+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a0170 con 0x7fab58100900 2024-06-06T07:47:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.925+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab58107b50 con 0x7fab58100900 2024-06-06T07:47:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.927+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab4801c070 con 0x7fab58100900 2024-06-06T07:47:52.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.927+0000 7fab4f7fe640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 0x7fab440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:52.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.928+0000 7fab55d9b640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 0x7fab440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:52.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.928+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7fab480ceb20 con 0x7fab58100900 2024-06-06T07:47:52.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.928+0000 7fab55d9b640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 0x7fab440a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a990 tx=0x7fab40005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:52.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:52.932+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab4809b690 con 0x7fab58100900 2024-06-06T07:47:53.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.154+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fab58062640 con 0x7fab58100900 2024-06-06T07:47:53.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.256+0000 7fab4f7fe640 1 -- 172.21.15.87:0/2152486000 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fab4800cd50 con 0x7fab58100900 2024-06-06T07:47:53.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 msgr2=0x7fab440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 0x7fab440a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a990 tx=0x7fab40005c70 comp rx=0 tx=0).stop 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 msgr2=0x7fab58197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b9c0 tx=0x7fab4800be80 comp rx=0 tx=0).stop 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 shutdown_connections 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7fab440a2c30 0x7fab440a50f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab581024b0 0x7fab5819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab58101af0 0x7fab58198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:53.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 --2- 172.21.15.87:0/2152486000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab58100900 0x7fab58197c50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:53.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.259+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 >> 172.21.15.87:0/2152486000 conn(0x7fab580fbe70 msgr2=0x7fab58104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:53.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.261+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 shutdown_connections 2024-06-06T07:47:53.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:53.262+0000 7fab5c970640 1 -- 172.21.15.87:0/2152486000 wait complete. 2024-06-06T07:47:53.276 INFO:teuthology.run_tasks:Running task kclient... 2024-06-06T07:47:53.290 INFO:tasks.kclient:Mounting kernel clients... 2024-06-06T07:47:53.290 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-06T07:47:53.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:53.291 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-06-06T07:47:53.306 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-06T07:47:53.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:53.307 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:47:53.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:53.375 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-06-06T07:47:53.481 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-06T07:47:53.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T07:47:53.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T07:47:53.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T07:47:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-mon[36790]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:53.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:47:53.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: ignoring --setuser ceph since I am not root 2024-06-06T07:47:53.557 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: ignoring --setgroup ceph since I am not root 2024-06-06T07:47:53.557 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-mgr[38210]: -- 172.21.15.181:0/3546957551 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6eb5685a0 con 0x55e6eb53d000 2024-06-06T07:47:53.557 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:53.438+0000 7f5995e97200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:47:53.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-mon[32276]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:53.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:47:53.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setuser ceph since I am not root 2024-06-06T07:47:53.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: ignoring --setgroup ceph since I am not root 2024-06-06T07:47:53.610 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:53.442+0000 7f585db52200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-mon[37040]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: ignoring --setuser ceph since I am not root 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: ignoring --setgroup ceph since I am not root 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-mgr[38819]: -- 172.21.15.195:0/824597729 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bd74ba5a0 con 0x559bd748e800 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:53.440+0000 7ff5a9577200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:47:53.623 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:53.622+0000 7ff5a9577200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:53.729 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:47:53.999 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:53.638+0000 7f585db52200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:53.999 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:53.753+0000 7f585db52200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:47:54.009 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:53.725+0000 7ff5a9577200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:47:54.030 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:53.622+0000 7f5995e97200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:54.030 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:53.724+0000 7f5995e97200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:47:54.030 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:54.030+0000 7f5995e97200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:47:54.270 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:54 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:54.073+0000 7f585db52200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:47:54.270 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:54 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:54.167+0000 7f585db52200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:47:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:54 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:47:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:54 smithi181 ceph-mon[36790]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:47:54.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:54.114+0000 7f5995e97200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:47:54.510 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:54.034+0000 7ff5a9577200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:47:54.510 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:54.118+0000 7ff5a9577200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:47:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:54 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:47:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:54 smithi195 ceph-mon[37040]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:47:54.525 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:54 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2152486000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:47:54.525 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:54 smithi087 ceph-mon[32276]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:47:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 -- 172.21.15.87:0/794602917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344107360 msgr2=0x7f4344107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 --2- 172.21.15.87:0/794602917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344107360 0x7f4344107740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4334009990 tx=0x7f4334009ca0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 -- 172.21.15.87:0/794602917 shutdown_connections 2024-06-06T07:47:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 --2- 172.21.15.87:0/794602917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344101ef0 0x7f434410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 --2- 172.21.15.87:0/794602917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344107360 0x7f4344107740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 --2- 172.21.15.87:0/794602917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101550 0x7f43441019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 -- 172.21.15.87:0/794602917 >> 172.21.15.87:0/794602917 conn(0x7f43440781d0 msgr2=0x7f43440785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:55.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 -- 172.21.15.87:0/794602917 shutdown_connections 2024-06-06T07:47:55.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.452+0000 7f434359e700 1 -- 172.21.15.87:0/794602917 wait complete. 2024-06-06T07:47:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 Processor -- start 2024-06-06T07:47:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 -- start start 2024-06-06T07:47:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 0x7f43441999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 0x7f4344199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344107360 0x7f43441a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.453+0000 7f434359e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43441115c0 con 0x7f4344101550 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111440 con 0x7f4344101ef0 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111740 con 0x7f4344107360 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 0x7f43441999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 0x7f43441999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47016/0 (socket says 172.21.15.87:47016) 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434259c700 1 -- 172.21.15.87:0/3924872091 learned_addr learned my addr 172.21.15.87:0/3924872091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 0x7f4344199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4342d9d700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344107360 0x7f43441a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:55.455 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:55 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:55.392+0000 7f585db52200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344107360 msgr2=0x7f43441a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344107360 0x7f43441a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 msgr2=0x7f43441999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 0x7f43441999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 -- 172.21.15.87:0/3924872091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441a4f80 con 0x7f4344101ef0 2024-06-06T07:47:55.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f4341d9b700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 0x7f4344199f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f433800e9f0 tx=0x7f433800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433800c9c0 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a51b0 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433800cb60 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338018680 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.454+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a5710 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.455+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344069180 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.456+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f4338010430 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.457+0000 7f43337fe700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 0x7f432c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.457+0000 7f434259c700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 msgr2=0x7f432c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.457+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f43380cf4f0 con 0x7f4344101ef0 2024-06-06T07:47:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.457+0000 7f434259c700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 0x7f432c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:47:55.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.462+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f433809c060 con 0x7f4344101ef0 2024-06-06T07:47:55.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:55.282+0000 7f5995e97200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:47:55.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:55.452+0000 7f5995e97200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:55.641 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:55.297+0000 7ff5a9577200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:47:55.641 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:55.470+0000 7ff5a9577200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:55.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.658+0000 7f434259c700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 msgr2=0x7f432c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:55.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.658+0000 7f434259c700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 0x7f432c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:47:55.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.712+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f434402dca0 con 0x7f4344101ef0 2024-06-06T07:47:55.712 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:55 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:55.575+0000 7f585db52200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:55.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.713+0000 7f43337fe700 1 -- 172.21.15.87:0/3924872091 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f433809b960 con 0x7f4344101ef0 2024-06-06T07:47:55.713 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 msgr2=0x7f432c0a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 0x7f432c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 msgr2=0x7f4344199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 0x7f4344199f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f433800e9f0 tx=0x7f433800ed00 comp rx=0 tx=0).stop 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 shutdown_connections 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f432c0a2cf0 0x7f432c0a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4344107360 0x7f43441a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4344101550 0x7f43441999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 --2- 172.21.15.87:0/3924872091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4344101ef0 0x7f4344199f00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 >> 172.21.15.87:0/3924872091 conn(0x7f43440781d0 msgr2=0x7f43441002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 shutdown_connections 2024-06-06T07:47:55.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:55.715+0000 7f434359e700 1 -- 172.21.15.87:0/3924872091 wait complete. 2024-06-06T07:47:55.910 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:55.623+0000 7f5995e97200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:47:55.910 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:55 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3924872091' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:47:55.932 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:55 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3924872091' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:47:55.932 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:55.641+0000 7ff5a9577200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:47:56.038 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:55 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3924872091' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:47:56.038 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:55 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:55.754+0000 7f585db52200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:47:56.203 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:55.910+0000 7f5995e97200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:47:56.203 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:56.001+0000 7f5995e97200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:47:56.203 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:56.202+0000 7f5995e97200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:47:56.226 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:55.932+0000 7ff5a9577200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:47:56.227 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:56.024+0000 7ff5a9577200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:47:56.349 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:56.050+0000 7f585db52200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:47:56.349 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:56.143+0000 7f585db52200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:47:56.349 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:56.348+0000 7f585db52200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T07:47:56.493 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-06T07:47:56.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:56.493 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-06-06T07:47:56.509 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:47:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42030sec preferred_lft 42030sec 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-06-06T07:47:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:47:56.511 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T07:47:56.511 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:47:56.511 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:47:56.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-06-06T07:47:56.512 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-06-06T07:47:56.512 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-06-06T07:47:56.512 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T07:47:56.512 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:47:56.512 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:56.227+0000 7ff5a9577200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:47:56.512 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:56.420+0000 7ff5a9577200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:56.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:56.395+0000 7f5995e97200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:56.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:56.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:56 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:56.550+0000 7f585db52200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:47:56.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:56.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:56.658 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T07:47:56.683 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-06-06T07:47:56.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:56.685 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-06-06T07:47:56.741 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-06-06T07:47:56.742 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T07:47:56.742 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-06T07:47:56.742 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-06T07:47:56.742 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T07:47:56.743 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:47:56.743 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:47:56.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-06T07:47:56.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-06T07:47:56.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-06T07:47:56.744 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:47:56.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:56.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:56.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:56.871 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-06-06T07:47:56.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:56.886 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-06-06T07:47:56.942 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:47:56.942 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:47:56.942 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:47:56.942 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-06T07:47:56.942 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:47:57.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:57.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.064 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-06T07:47:57.064 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-06T07:47:57.065 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-06T07:47:57.065 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:47:57.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:57.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.182 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:47:57.182 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-06-06T07:47:57.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-06-06T07:47:57.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T07:47:57.183 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-06-06T07:47:57.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:57.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:47:57.262 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-06T07:47:57.277 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-06T07:47:57.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:47:57.277 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-06T07:47:57.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:56.994+0000 7f5995e97200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:47:57.333 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-06T07:47:57.333 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-06T07:47:57.333 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 2024-06-06T07:47:57.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:57.333 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-06T07:47:57.396 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:57.202+0000 7f585db52200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:47:57.411 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:57.018+0000 7ff5a9577200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:47:57.418 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-06T07:47:57.418 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-06T07:47:57.419 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:invalid new device string format 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.181:3300,172.21.15.195:3300" 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.87:3300,172.21.15.181:3300,172.21.15.195:3300:/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 2024-06-06T07:47:57.561 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=3f55109c-23d7-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-06T07:47:57.562 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-06T07:47:57.562 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-06-06T07:47:57.562 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-06T07:47:57.562 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-06-06T07:47:57.562 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-06T07:47:57.563 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-06T07:47:57.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:47:57.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-06T07:47:57.629 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T07:47:57.710 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:57.411+0000 7f5995e97200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:47:57.711 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:57.508+0000 7f5995e97200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:47:57.711 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:57.710+0000 7f5995e97200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:47:57.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T07:47:57.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.728 INFO:teuthology.orchestra.run.smithi087.stdout:client24812 2024-06-06T07:47:57.729 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:47:57.729 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:47:57.739 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:57.441+0000 7ff5a9577200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:47:57.740 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:57.539+0000 7ff5a9577200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:47:57.746 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T07:47:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T07:47:57.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.843 INFO:teuthology.orchestra.run.smithi087.stdout:client24812 2024-06-06T07:47:57.843 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T07:47:57.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T07:47:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.900 INFO:teuthology.orchestra.run.smithi087.stdout:client24812 2024-06-06T07:47:57.900 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968.client24812/status') 2024-06-06T07:47:57.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:47:57.957 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:57.676+0000 7f585db52200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:47:57.957 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:57.780+0000 7f585db52200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:47:57.959 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.24812 (3)172.21.15.87:0/1932774303 2024-06-06T07:47:57.959 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-06-06T07:47:57.960 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-06-06T07:47:57.960 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-06-06T07:47:57.960 INFO:teuthology.orchestra.run.smithi087.stderr:71 bytes copied, 2.5212e-05 s, 2.8 MB/s 2024-06-06T07:47:57.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:47:57.968 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T07:47:57.977 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:47:57.977 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:47:57.977 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-06T07:47:58.009 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:57.740+0000 7ff5a9577200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 -- 172.21.15.87:0/740119136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7384104db0 msgr2=0x7f73841071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 --2- 172.21.15.87:0/740119136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7384104db0 0x7f73841071a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7374009900 tx=0x7f737402a410 comp rx=0 tx=0).stop 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 -- 172.21.15.87:0/740119136 shutdown_connections 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 --2- 172.21.15.87:0/740119136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73841076e0 0x7f7384109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 --2- 172.21.15.87:0/740119136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7384104db0 0x7f73841071a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 --2- 172.21.15.87:0/740119136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 0x7f7384104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 -- 172.21.15.87:0/740119136 >> 172.21.15.87:0/740119136 conn(0x7f73840fbe70 msgr2=0x7f73840fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 -- 172.21.15.87:0/740119136 shutdown_connections 2024-06-06T07:47:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 -- 172.21.15.87:0/740119136 wait complete. 2024-06-06T07:47:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.082+0000 7f738c14f640 1 Processor -- start 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 -- start start 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 0x7f7384197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7384104db0 0x7f73841981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f738410d680 con 0x7f73841076e0 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f738410d500 con 0x7f7384102480 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738c14f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f738410d800 con 0x7f7384104db0 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33248/0 (socket says 172.21.15.87:33248) 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 -- 172.21.15.87:0/3048442633 learned_addr learned my addr 172.21.15.87:0/3048442633 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f73896ea640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7384104db0 0x7f73841981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f7389eeb640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 0x7f7384197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7384104db0 msgr2=0x7f73841981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7384104db0 0x7f73841981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 msgr2=0x7f7384197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 0x7f7384197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 -- 172.21.15.87:0/3048442633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738419f8d0 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f738a6ec640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f737c00eb10 tx=0x7f737c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.083+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c004b20 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.084+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738419fbc0 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.084+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f737c004cc0 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.084+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c010760 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.084+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73841a0130 con 0x7f73841076e0 2024-06-06T07:47:58.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.085+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73840fd960 con 0x7f73841076e0 2024-06-06T07:47:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.086+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f737c005020 con 0x7f73841076e0 2024-06-06T07:47:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.086+0000 7f737affd640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 0x7f73700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.086+0000 7f7389eeb640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 msgr2=0x7f73700a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.087+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f737c0cfb30 con 0x7f73841076e0 2024-06-06T07:47:58.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.087+0000 7f7389eeb640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 0x7f73700a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:47:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.091+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f737c09c6a0 con 0x7f73841076e0 2024-06-06T07:47:58.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.288+0000 7f7389eeb640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 msgr2=0x7f73700a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:58.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.288+0000 7f7389eeb640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 0x7f73700a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:47:58.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.306+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f73840625a0 con 0x7f73841076e0 2024-06-06T07:47:58.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.358+0000 7f737affd640 1 -- 172.21.15.87:0/3048442633 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f737c09bfa0 con 0x7f73841076e0 2024-06-06T07:47:58.359 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:57 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:57.997+0000 7f585db52200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:47:58.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 msgr2=0x7f73700a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:47:58.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 0x7f73700a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 msgr2=0x7f738419f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f737c00eb10 tx=0x7f737c00c2d0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 shutdown_connections 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f73700a2c80 0x7f73700a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73841076e0 0x7f738419f230 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7384104db0 0x7f73841981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 --2- 172.21.15.87:0/3048442633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7384102480 0x7f7384197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 >> 172.21.15.87:0/3048442633 conn(0x7f73840fbe70 msgr2=0x7f7384102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 shutdown_connections 2024-06-06T07:47:58.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.360+0000 7f738c14f640 1 -- 172.21.15.87:0/3048442633 wait complete. 2024-06-06T07:47:58.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 -- 172.21.15.87:0/385610325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10701024b0 msgr2=0x7f1070108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 --2- 172.21.15.87:0/385610325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10701024b0 0x7f1070108d40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ca0 tx=0x7f106802a8c0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 -- 172.21.15.87:0/385610325 shutdown_connections 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 --2- 172.21.15.87:0/385610325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10701024b0 0x7f1070108d40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 --2- 172.21.15.87:0/385610325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 0x7f1070101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 --2- 172.21.15.87:0/385610325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1070100900 0x7f1070100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 -- 172.21.15.87:0/385610325 >> 172.21.15.87:0/385610325 conn(0x7f10700fbe70 msgr2=0x7f10700fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:47:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 -- 172.21.15.87:0/385610325 shutdown_connections 2024-06-06T07:47:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.476+0000 7f1074d5a640 1 -- 172.21.15.87:0/385610325 wait complete. 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 Processor -- start 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 -- start start 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 0x7f1070069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 0x7f107019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10701024b0 0x7f107019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f107010c560 con 0x7f1070100900 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f107010c3e0 con 0x7f1070101af0 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f1074d5a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f107010c6e0 con 0x7f10701024b0 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 0x7f1070069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 0x7f1070069bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33262/0 (socket says 172.21.15.87:33262) 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 -- 172.21.15.87:0/2330728152 learned_addr learned my addr 172.21.15.87:0/2330728152 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 0x7f107019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106ed9d640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10701024b0 0x7f107019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10701024b0 msgr2=0x7f107019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10701024b0 0x7f107019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 msgr2=0x7f107019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 0x7f107019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106e59c640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107019f890 con 0x7f1070100900 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.477+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1070101af0 0x7f107019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:47:58.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f106e59c640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 0x7f1070069bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f106400d770 tx=0x7f106400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064017070 con 0x7f1070100900 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1064009c30 con 0x7f1070100900 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064003dd0 con 0x7f1070100900 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107019fb20 con 0x7f1070100900 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.478+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701a0020 con 0x7f1070100900 2024-06-06T07:47:58.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.479+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1070107bc0 con 0x7f1070100900 2024-06-06T07:47:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.480+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1064003f70 con 0x7f1070100900 2024-06-06T07:47:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.481+0000 7f105f7fe640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:47:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.481+0000 7f106dd9b640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 msgr2=0x7f10580a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.481+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7f10640cf1a0 con 0x7f1070100900 2024-06-06T07:47:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.481+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:47:58.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.484+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1064009de0 con 0x7f1070100900 2024-06-06T07:47:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T07:47:58.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.234+0000 7f5995e97200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:47:58.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.326+0000 7f5995e97200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:47:58.609 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T07:47:58.609 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.262+0000 7ff5a9577200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:47:58.609 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.356+0000 7ff5a9577200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:47:58.640 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:58 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T07:47:58.640 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:58.542+0000 7f585db52200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:47:58.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.682+0000 7f106dd9b640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 msgr2=0x7f10580a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:58.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.682+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:47:58.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:58.694+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1070061f40 con 0x7f10580a2c80 2024-06-06T07:47:58.916 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.572+0000 7f5995e97200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:47:58.916 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.704+0000 7f5995e97200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:47:58.916 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.800+0000 7f5995e97200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:47:58.916 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:58.916+0000 7f5995e97200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:58.972 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.609+0000 7ff5a9577200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:47:58.972 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.749+0000 7ff5a9577200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:47:58.973 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.852+0000 7ff5a9577200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:47:58.973 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:58.972+0000 7ff5a9577200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:59.025 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:58.640+0000 7f585db52200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:47:59.025 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:58 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:58.889+0000 7f585db52200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:47:59.025 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:59.024+0000 7f585db52200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:47:59.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:59.083+0000 7f106dd9b640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 msgr2=0x7f10580a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:59.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:59.083+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T07:47:59.359 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:59.123+0000 7f585db52200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:47:59.360 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:59.241+0000 7f585db52200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:47:59.556 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:59.220+0000 7f5995e97200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:47:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:59 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T07:47:59.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:47:59 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:47:59.640 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:59.281+0000 7ff5a9577200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:47:59.641 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:59 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T07:47:59.641 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:47:59 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:47:59.859 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:47:59.544+0000 7f585db52200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:47:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3048442633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T07:47:59.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:47:59 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:47:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:59.884+0000 7f106dd9b640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 msgr2=0x7f10580a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2460003046 2024-06-06T07:47:59.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:47:59.884+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T07:48:00.010 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:47:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:47:59.888+0000 7ff5a9577200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:48:00.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:47:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:47:59.824+0000 7f5995e97200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:48:00.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y[38206]: 2024-06-06T07:48:00.042+0000 7f5995e97200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:48:00.365 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z[38775]: 2024-06-06T07:48:00.105+0000 7ff5a9577200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:48:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:00.379+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f1064004260 con 0x7f1070100900 2024-06-06T07:48:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:00.397+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f1064097d30 con 0x7f1070100900 2024-06-06T07:48:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:00.398+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 msgr2=0x7f10580a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:48:00.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:00.398+0000 7f105f7fe640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/2460003046,v1:172.21.15.87:6801/2460003046] conn(0x7f10580a2c80 0x7f10580a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:00.533 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.151+0000 7f585db52200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.382+0000 7f585db52200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: Standby manager daemon y restarted 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: Standby manager daemon y started 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: Standby manager daemon z restarted 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: Standby manager daemon z started 2024-06-06T07:48:00.534 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:48:00.535 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.535 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:48:00.535 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-mon[32276]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: Standby manager daemon y restarted 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: Standby manager daemon y started 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: Standby manager daemon z restarted 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: Standby manager daemon z started 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:48:00.620 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:00 smithi181 ceph-mon[36790]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: Standby manager daemon y restarted 2024-06-06T07:48:00.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: Standby manager daemon y started 2024-06-06T07:48:00.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:48:00.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.181:0/864073154' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: Standby manager daemon z restarted 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: Standby manager daemon z started 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:48:00.622 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:00 smithi195 ceph-mon[37040]: from='mgr.? 172.21.15.195:0/2557925876' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:48:00.859 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.627+0000 7f57c0452700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.859 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.627+0000 7f57c0452700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.627+0000 7f57c0452700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.627+0000 7f57c0452700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.627+0000 7f57c0452700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.632+0000 7f57bd44c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.632+0000 7f57bd44c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.632+0000 7f57bd44c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.632+0000 7f57bd44c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:00.860 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:00.632+0000 7f57bd44c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.402+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mon.0 v2:172.21.15.87:3300/0 9 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f10640cc900 con 0x7f1070100900 2024-06-06T07:48:01.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.403+0000 7f105f7fe640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1058035160 0x7f1058035540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.403+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f10640c43d0 con 0x7f1058035160 2024-06-06T07:48:01.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.405+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1058035160 0x7f1058035540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.407+0000 7f106dd9b640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1058035160 0x7f1058035540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1070069930 tx=0x7f1060005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:01.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.427+0000 7f105f7fe640 1 -- 172.21.15.87:0/2330728152 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f10640c43d0 con 0x7f1058035160 2024-06-06T07:48:01.427 INFO:teuthology.orchestra.run.smithi087.stdout:{} 2024-06-06T07:48:01.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1058035160 msgr2=0x7f1058035540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1058035160 0x7f1058035540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1070069930 tx=0x7f1060005f90 comp rx=0 tx=0).stop 2024-06-06T07:48:01.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 msgr2=0x7f1070069bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 --2- 172.21.15.87:0/2330728152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1070100900 0x7f1070069bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f106400d770 tx=0x7f106400dc30 comp rx=0 tx=0).stop 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f106ed9d640 1 -- 172.21.15.87:0/2330728152 reap_dead start 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 shutdown_connections 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 >> 172.21.15.87:0/2330728152 conn(0x7f10700fbe70 msgr2=0x7f1070104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 shutdown_connections 2024-06-06T07:48:01.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.429+0000 7f1074d5a640 1 -- 172.21.15.87:0/2330728152 wait complete. 2024-06-06T07:48:01.442 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-06T07:48:01.455 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-06T07:48:01.465 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T07:48:01.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82ddd9b640 1 -- 172.21.15.87:0/3264017144 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00319b0 con 0x7f82e01024e0 2024-06-06T07:48:01.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 -- 172.21.15.87:0/3264017144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e01024e0 msgr2=0x7f82e01048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 --2- 172.21.15.87:0/3264017144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e01024e0 0x7f82e01048d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009900 tx=0x7f82d002a410 comp rx=0 tx=0).stop 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 -- 172.21.15.87:0/3264017144 shutdown_connections 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 --2- 172.21.15.87:0/3264017144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0107740 0x7f82e0109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 --2- 172.21.15.87:0/3264017144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e0104e10 0x7f82e0107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 --2- 172.21.15.87:0/3264017144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e01024e0 0x7f82e01048d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 -- 172.21.15.87:0/3264017144 >> 172.21.15.87:0/3264017144 conn(0x7f82e00fbef0 msgr2=0x7f82e00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 -- 172.21.15.87:0/3264017144 shutdown_connections 2024-06-06T07:48:01.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.568+0000 7f82e4c93640 1 -- 172.21.15.87:0/3264017144 wait complete. 2024-06-06T07:48:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 Processor -- start 2024-06-06T07:48:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 -- start start 2024-06-06T07:48:01.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 0x7f82e0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 0x7f82e0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82e010d6f0 con 0x7f82e0107740 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82e010d570 con 0x7f82e0104e10 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.569+0000 7f82e4c93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82e010d870 con 0x7f82e01024e0 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82d5d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 0x7f82e0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33420/0 (socket says 172.21.15.87:33420) 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 -- 172.21.15.87:0/1497632750 learned_addr learned my addr 172.21.15.87:0/1497632750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82de59c640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 0x7f82e0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 -- 172.21.15.87:0/1497632750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 msgr2=0x7f82e0197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 0x7f82e0197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 -- 172.21.15.87:0/1497632750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 msgr2=0x7f82e0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 0x7f82e0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 -- 172.21.15.87:0/1497632750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e019f900 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82de59c640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 0x7f82e0197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82d5d9b640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 0x7f82e0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82ded9d640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82d800cce0 tx=0x7f82d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d800f450 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d800fad0 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8018c10 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e019fbf0 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.570+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e01a00f0 con 0x7f82e0107740 2024-06-06T07:48:01.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.571+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e00fd9e0 con 0x7f82e0107740 2024-06-06T07:48:01.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.572+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f82d800f5f0 con 0x7f82e0107740 2024-06-06T07:48:01.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.573+0000 7f82d7fff640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 0x7f82cc0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.573+0000 7f82de59c640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 0x7f82cc0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.573+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f82d80cf8b0 con 0x7f82e0107740 2024-06-06T07:48:01.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.574+0000 7f82de59c640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 0x7f82cc0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82d0006010 tx=0x7f82d00095f0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:01.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.578+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82d8014030 con 0x7f82e0107740 2024-06-06T07:48:01.631 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-06-06T07:48:01.631 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: Active manager daemon x restarted 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: Activating manager daemon x 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: mgrmap e21: x(active, starting, since 0.014126s), standbys: y, z 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:48:01.632 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:48:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: Manager daemon x is now available 2024-06-06T07:48:01.634 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:01.635 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:48:01.635 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:01 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:48:01.664 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-06-06T07:48:01.664 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: Active manager daemon x restarted 2024-06-06T07:48:01.664 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: Activating manager daemon x 2024-06-06T07:48:01.664 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: mgrmap e21: x(active, starting, since 0.014126s), standbys: y, z 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:48:01.665 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:48:01.666 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:48:01.667 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:48:01.668 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: Manager daemon x is now available 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:48:01.669 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:01 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:48:01.671 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:01.413+0000 7f57ba446700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.672 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:01.413+0000 7f57ba446700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.672 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:01.413+0000 7f57ba446700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.672 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:01.413+0000 7f57ba446700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.672 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:48:01.413+0000 7f57ba446700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:48:01.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: Active manager daemon x restarted 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: Activating manager daemon x 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: mgrmap e21: x(active, starting, since 0.014126s), standbys: y, z 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:48:01.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:48:01.674 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:48:01.675 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: Manager daemon x is now available 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:48:01.676 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:01 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:48:01.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.762+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82e00625a0 con 0x7f82e0107740 2024-06-06T07:48:01.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.764+0000 7f82d7fff640 1 -- 172.21.15.87:0/1497632750 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25294 (secure 0 0 0) 0x7f82d809c610 con 0x7f82e0107740 2024-06-06T07:48:01.764 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:01.764 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:48:00.385938+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/3344462507":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/207935511":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/4065374776":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/2153987725":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:48:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 msgr2=0x7f82cc0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 0x7f82cc0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82d0006010 tx=0x7f82d00095f0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 msgr2=0x7f82e019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82d800cce0 tx=0x7f82d800a620 comp rx=0 tx=0).stop 2024-06-06T07:48:01.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 shutdown_connections 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82cc0a2990 0x7f82cc0a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e0107740 0x7f82e019f260 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e0104e10 0x7f82e0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 --2- 172.21.15.87:0/1497632750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e01024e0 0x7f82e0197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 >> 172.21.15.87:0/1497632750 conn(0x7f82e00fbef0 msgr2=0x7f82e0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:01.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 shutdown_connections 2024-06-06T07:48:01.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.766+0000 7f82e4c93640 1 -- 172.21.15.87:0/1497632750 wait complete. 2024-06-06T07:48:01.778 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T07:39:10.366934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T07:43:56.169718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T07:43:59.265590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T07:44:09.602679+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T07:48:01.779 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T07:48:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 -- 172.21.15.87:0/3436412440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 msgr2=0x7f7298101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 --2- 172.21.15.87:0/3436412440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298101f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f728c009900 tx=0x7f728c02a410 comp rx=0 tx=0).stop 2024-06-06T07:48:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 -- 172.21.15.87:0/3436412440 shutdown_connections 2024-06-06T07:48:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 --2- 172.21.15.87:0/3436412440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7298102490 0x7f7298108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 --2- 172.21.15.87:0/3436412440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298101f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 --2- 172.21.15.87:0/3436412440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72981008e0 0x7f7298100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 -- 172.21.15.87:0/3436412440 >> 172.21.15.87:0/3436412440 conn(0x7f72980fbe50 msgr2=0x7f72980fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:01.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 -- 172.21.15.87:0/3436412440 shutdown_connections 2024-06-06T07:48:01.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.884+0000 7f729cd12640 1 -- 172.21.15.87:0/3436412440 wait complete. 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 Processor -- start 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 -- start start 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 0x7f7298197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 0x7f729819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f729810d680 con 0x7f7298101ad0 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f729810d500 con 0x7f7298102490 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729cd12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f729810d800 con 0x7f72981008e0 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33448/0 (socket says 172.21.15.87:33448) 2024-06-06T07:48:01.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 -- 172.21.15.87:0/3408358187 learned_addr learned my addr 172.21.15.87:0/3408358187 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729659c640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 0x7f7298197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7296d9d640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 0x7f729819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 -- 172.21.15.87:0/3408358187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 msgr2=0x7f7298197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 0x7f7298197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 -- 172.21.15.87:0/3408358187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 msgr2=0x7f729819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 0x7f729819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7295d9b640 1 -- 172.21.15.87:0/3408358187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729819f910 con 0x7f7298101ad0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f7296d9d640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 0x7f729819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.885+0000 7f729659c640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 0x7f7298197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f7295d9b640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f728c009230 tx=0x7f728c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c03f030 con 0x7f7298101ad0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729819fba0 con 0x7f7298101ad0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728c003ce0 con 0x7f7298101ad0 2024-06-06T07:48:01.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c035070 con 0x7f7298101ad0 2024-06-06T07:48:01.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.886+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981a0090 con 0x7f7298101ad0 2024-06-06T07:48:01.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.887+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298107c80 con 0x7f7298101ad0 2024-06-06T07:48:01.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.888+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f728c042030 con 0x7f7298101ad0 2024-06-06T07:48:01.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.889+0000 7f72877fe640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 0x7f72800a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:01.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.889+0000 7f729659c640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 0x7f72800a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:01.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.889+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f728c0efc90 con 0x7f7298101ad0 2024-06-06T07:48:01.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.889+0000 7f729659c640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 0x7f72800a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7288005fd0 tx=0x7f7288005ee0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:01.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:01.893+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f728c0bc9f0 con 0x7f7298101ad0 2024-06-06T07:48:02.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.072+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72980625d0 con 0x7f7298101ad0 2024-06-06T07:48:02.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.073+0000 7f72877fe640 1 -- 172.21.15.87:0/3408358187 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f728c0bc2f0 con 0x7f7298101ad0 2024-06-06T07:48:02.074 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.074+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 msgr2=0x7f72800a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 0x7f72800a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7288005fd0 tx=0x7f7288005ee0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 msgr2=0x7f7298198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f728c009230 tx=0x7f728c004360 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 shutdown_connections 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72800a2990 0x7f72800a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7298102490 0x7f729819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7298101ad0 0x7f7298198240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 --2- 172.21.15.87:0/3408358187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72981008e0 0x7f7298197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 >> 172.21.15.87:0/3408358187 conn(0x7f72980fbe50 msgr2=0x7f7298104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 shutdown_connections 2024-06-06T07:48:02.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.075+0000 7f729cd12640 1 -- 172.21.15.87:0/3408358187 wait complete. 2024-06-06T07:48:02.087 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T07:48:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 -- 172.21.15.87:0/4099945076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 msgr2=0x7fdd081071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4099945076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd081071e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009900 tx=0x7fdcf002a400 comp rx=0 tx=0).stop 2024-06-06T07:48:02.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 -- 172.21.15.87:0/4099945076 shutdown_connections 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4099945076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd08107720 0x7fdd08109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4099945076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd081071e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4099945076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd081024c0 0x7fdd081048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 -- 172.21.15.87:0/4099945076 >> 172.21.15.87:0/4099945076 conn(0x7fdd080fbe90 msgr2=0x7fdd080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 -- 172.21.15.87:0/4099945076 shutdown_connections 2024-06-06T07:48:02.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.199+0000 7fdd0cded640 1 -- 172.21.15.87:0/4099945076 wait complete. 2024-06-06T07:48:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 Processor -- start 2024-06-06T07:48:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 -- start start 2024-06-06T07:48:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd081024c0 0x7fdd08197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107720 0x7fdd0819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810d630 con 0x7fdd08104df0 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810d4b0 con 0x7fdd08107720 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0cded640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810d7b0 con 0x7fdd081024c0 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd05d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd05d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33462/0 (socket says 172.21.15.87:33462) 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd05d9b640 1 -- 172.21.15.87:0/4007427893 learned_addr learned my addr 172.21.15.87:0/4007427893 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd0659c640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd081024c0 0x7fdd08197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.200+0000 7fdd06d9d640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107720 0x7fdd0819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 -- 172.21.15.87:0/4007427893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd081024c0 msgr2=0x7fdd08197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd081024c0 0x7fdd08197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 -- 172.21.15.87:0/4007427893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107720 msgr2=0x7fdd0819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107720 0x7fdd0819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 -- 172.21.15.87:0/4007427893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0819f960 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd05d9b640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0031610 tx=0x7fdcf00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf003f030 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0819fbf0 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf0003ce0 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0035070 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.201+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd081a0130 con 0x7fdd08104df0 2024-06-06T07:48:02.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.202+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd080fd980 con 0x7fdd08104df0 2024-06-06T07:48:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.203+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fdcf0042030 con 0x7fdd08104df0 2024-06-06T07:48:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.204+0000 7fdcff7fe640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 0x7fdcf40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.204+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fdcf00ef6d0 con 0x7fdd08104df0 2024-06-06T07:48:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.204+0000 7fdd0659c640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 0x7fdcf40a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.204+0000 7fdd0659c640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 0x7fdcf40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd08199220 tx=0x7fdd00009680 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.212+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcf00bbde0 con 0x7fdd08104df0 2024-06-06T07:48:02.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.412+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7fdcf00b83c0 con 0x7fdd08104df0 2024-06-06T07:48:02.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.422+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd080625a0 con 0x7fdd08104df0 2024-06-06T07:48:02.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.423+0000 7fdcff7fe640 1 -- 172.21.15.87:0/4007427893 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7fdcf00717c0 con 0x7fdd08104df0 2024-06-06T07:48:02.424 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 msgr2=0x7fdcf40a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 0x7fdcf40a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd08199220 tx=0x7fdd00009680 comp rx=0 tx=0).stop 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 msgr2=0x7fdd08198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0031610 tx=0x7fdcf00043b0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 shutdown_connections 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdcf40a2990 0x7fdcf40a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107720 0x7fdd0819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd08104df0 0x7fdd08198290 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 --2- 172.21.15.87:0/4007427893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd081024c0 0x7fdd08197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 >> 172.21.15.87:0/4007427893 conn(0x7fdd080fbe90 msgr2=0x7fdd08102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 shutdown_connections 2024-06-06T07:48:02.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.425+0000 7fdd0cded640 1 -- 172.21.15.87:0/4007427893 wait complete. 2024-06-06T07:48:02.438 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 -- 172.21.15.87:0/501017057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 msgr2=0x7f6880100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 --2- 172.21.15.87:0/501017057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880100d00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6870009900 tx=0x7f687002a410 comp rx=0 tx=0).stop 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 -- 172.21.15.87:0/501017057 shutdown_connections 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 --2- 172.21.15.87:0/501017057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f6880108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 --2- 172.21.15.87:0/501017057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f6880101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 --2- 172.21.15.87:0/501017057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880100d00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 -- 172.21.15.87:0/501017057 >> 172.21.15.87:0/501017057 conn(0x7f68800fbe50 msgr2=0x7f68800fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 -- 172.21.15.87:0/501017057 shutdown_connections 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.554+0000 7f6886309640 1 -- 172.21.15.87:0/501017057 wait complete. 2024-06-06T07:48:02.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 Processor -- start 2024-06-06T07:48:02.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 -- start start 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f688010c560 con 0x7f68801008e0 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f688010c3e0 con 0x7f6880101ad0 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.555+0000 7f6886309640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f688010c6e0 con 0x7f6880102490 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:58404/0 (socket says 172.21.15.87:58404) 2024-06-06T07:48:02.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:57414/0 (socket says 172.21.15.87:57414) 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33490/0 (socket says 172.21.15.87:33490) 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687f7fe640 1 -- 172.21.15.87:0/2199210107 learned_addr learned my addr 172.21.15.87:0/2199210107 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 -- 172.21.15.87:0/2199210107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 msgr2=0x7f68801981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 -- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 msgr2=0x7f6880197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 -- 172.21.15.87:0/2199210107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688019f8f0 con 0x7f6880102490 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687f7fe640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f68848a6640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6878009fd0 tx=0x7f687800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687800ce20 con 0x7f6880102490 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6878005020 con 0x7f6880102490 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688019fbe0 con 0x7f6880102490 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878010720 con 0x7f6880102490 2024-06-06T07:48:02.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.556+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a0100 con 0x7f6880102490 2024-06-06T07:48:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.557+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68801a0330 con 0x7f6880102490 2024-06-06T07:48:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.559+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f6878005540 con 0x7f6880102490 2024-06-06T07:48:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.560+0000 7f687d7fa640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 0x7f68680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.560+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f68780cff90 con 0x7f6880102490 2024-06-06T07:48:02.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.560+0000 7f687ffff640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 0x7f68680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.560+0000 7f687ffff640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 0x7f68680a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6870005ab0 tx=0x7f6870000bc0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.563+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f687809c670 con 0x7f6880102490 2024-06-06T07:48:02.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.748+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f68801a0500 con 0x7f6880102490 2024-06-06T07:48:02.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.749+0000 7f687d7fa640 1 -- 172.21.15.87:0/2199210107 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f687809bf70 con 0x7f6880102490 2024-06-06T07:48:02.750 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-06-06T07:48:02.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 msgr2=0x7f68680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 0x7f68680a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6870005ab0 tx=0x7f6870000bc0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 msgr2=0x7f688019f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6878009fd0 tx=0x7f687800ef10 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 shutdown_connections 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f68680a2d90 0x7f68680a5250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6880102490 0x7f688019f250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6880101ad0 0x7f68801981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 --2- 172.21.15.87:0/2199210107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68801008e0 0x7f6880197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 >> 172.21.15.87:0/2199210107 conn(0x7f68800fbe50 msgr2=0x7f6880068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 shutdown_connections 2024-06-06T07:48:02.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.751+0000 7f6886309640 1 -- 172.21.15.87:0/2199210107 wait complete. 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: [06/Jun/2024:07:48:01] ENGINE Bus STARTING 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: mgrmap e22: x(active, since 1.02039s), standbys: y, z 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: [06/Jun/2024:07:48:01] ENGINE Serving on http://172.21.15.87:8765 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: [06/Jun/2024:07:48:01] ENGINE Serving on https://172.21.15.87:7150 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: [06/Jun/2024:07:48:01] ENGINE Bus STARTED 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1497632750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:02 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3408358187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:48:02.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: [06/Jun/2024:07:48:01] ENGINE Bus STARTING 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: mgrmap e22: x(active, since 1.02039s), standbys: y, z 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: [06/Jun/2024:07:48:01] ENGINE Serving on http://172.21.15.87:8765 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: [06/Jun/2024:07:48:01] ENGINE Serving on https://172.21.15.87:7150 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: [06/Jun/2024:07:48:01] ENGINE Bus STARTED 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1497632750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:02.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:02 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3408358187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:48:02.764 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T07:48:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: [06/Jun/2024:07:48:01] ENGINE Bus STARTING 2024-06-06T07:48:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: mgrmap e22: x(active, since 1.02039s), standbys: y, z 2024-06-06T07:48:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: [06/Jun/2024:07:48:01] ENGINE Serving on http://172.21.15.87:8765 2024-06-06T07:48:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: [06/Jun/2024:07:48:01] ENGINE Serving on https://172.21.15.87:7150 2024-06-06T07:48:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: [06/Jun/2024:07:48:01] ENGINE Bus STARTED 2024-06-06T07:48:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1497632750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:02.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:02 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3408358187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 -- 172.21.15.87:0/2292002657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e001076c0 msgr2=0x7f4e00109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/2292002657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e001076c0 0x7f4e00109b10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009ca0 tx=0x7f4df802a8c0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 -- 172.21.15.87:0/2292002657 shutdown_connections 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/2292002657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e001076c0 0x7f4e00109b10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/2292002657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e00104d90 0x7f4e00107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/2292002657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102460 0x7f4e00104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 -- 172.21.15.87:0/2292002657 >> 172.21.15.87:0/2292002657 conn(0x7f4e000fbe50 msgr2=0x7f4e000fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:02.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 -- 172.21.15.87:0/2292002657 shutdown_connections 2024-06-06T07:48:02.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.869+0000 7f4e04a2a640 1 -- 172.21.15.87:0/2292002657 wait complete. 2024-06-06T07:48:02.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 Processor -- start 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 -- start start 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00104d90 0x7f4e00100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e001076c0 0x7f4e00100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010d680 con 0x7f4e00102460 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010d500 con 0x7f4e001076c0 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4e04a2a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010d800 con 0x7f4e00104d90 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33506/0 (socket says 172.21.15.87:33506) 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 -- 172.21.15.87:0/3871265957 learned_addr learned my addr 172.21.15.87:0/3871265957 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfdd9b640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00104d90 0x7f4e00100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfed9d640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e001076c0 0x7f4e00100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 -- 172.21.15.87:0/3871265957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00104d90 msgr2=0x7f4e00100410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00104d90 0x7f4e00100410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 -- 172.21.15.87:0/3871265957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e001076c0 msgr2=0x7f4e00100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e001076c0 0x7f4e00100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 -- 172.21.15.87:0/3871265957 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e001011e0 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4dfe59c640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e9f0 tx=0x7f4df000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.870+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000cd80 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.871+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e001a1e10 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.871+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0005500 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.871+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0010690 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.871+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e001a2220 con 0x7f4e00102460 2024-06-06T07:48:02.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.872+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e000fd940 con 0x7f4e00102460 2024-06-06T07:48:02.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.874+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f4df00046c0 con 0x7f4e00102460 2024-06-06T07:48:02.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.874+0000 7f4def7fe640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 0x7f4de80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:02.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.875+0000 7f4dfdd9b640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 0x7f4de80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:02.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.875+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4df00cfe40 con 0x7f4e00102460 2024-06-06T07:48:02.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.875+0000 7f4dfdd9b640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 0x7f4de80a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e00101b40 tx=0x7f4df4005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:02.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:02.879+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4df009c520 con 0x7f4e00102460 2024-06-06T07:48:03.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.064+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4e0002d2e0 con 0x7f4e00102460 2024-06-06T07:48:03.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.064+0000 7f4def7fe640 1 -- 172.21.15.87:0/3871265957 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f4df009be20 con 0x7f4e00102460 2024-06-06T07:48:03.065 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-06-06T07:48:03.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 msgr2=0x7f4de80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 0x7f4de80a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e00101b40 tx=0x7f4df4005f90 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 msgr2=0x7f4e00101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e9f0 tx=0x7f4df000edf0 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 shutdown_connections 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4de80a2d90 0x7f4de80a5250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e001076c0 0x7f4e00100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00104d90 0x7f4e00100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 --2- 172.21.15.87:0/3871265957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e00102460 0x7f4e00101dd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 >> 172.21.15.87:0/3871265957 conn(0x7f4e000fbe50 msgr2=0x7f4e00102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:03.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 shutdown_connections 2024-06-06T07:48:03.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:03.066+0000 7f4e04a2a640 1 -- 172.21.15.87:0/3871265957 wait complete. 2024-06-06T07:48:03.079 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-06T07:48:03.079 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:03.324 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4007427893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.716 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.717 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.717 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.717 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2199210107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.717 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3871265957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4007427893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2199210107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3871265957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-06T07:48:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4007427893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2199210107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:48:03.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3871265957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:48:04.759 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.760 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:48:04.761 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:04.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:04 smithi087 ceph-mon[32276]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:05.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:48:05.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:48:05.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:48:05.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.013 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:04 smithi195 ceph-mon[37040]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:05.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:48:05.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi087:/etc/ceph/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi181:/etc/ceph/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi195:/etc/ceph/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:48:05.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:04 smithi181 ceph-mon[36790]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- 172.21.15.87:0/1477290357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e8101af0 msgr2=0x7f35e8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 --2- 172.21.15.87:0/1477290357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e8101af0 0x7f35e8101f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009990 tx=0x7f35d0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- 172.21.15.87:0/1477290357 shutdown_connections 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 --2- 172.21.15.87:0/1477290357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81024b0 0x7f35e810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 --2- 172.21.15.87:0/1477290357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e81063c0 0x7f35e81067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 --2- 172.21.15.87:0/1477290357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e8101af0 0x7f35e8101f70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- 172.21.15.87:0/1477290357 >> 172.21.15.87:0/1477290357 conn(0x7f35e8077990 msgr2=0x7f35e80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- 172.21.15.87:0/1477290357 shutdown_connections 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- 172.21.15.87:0/1477290357 wait complete. 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 Processor -- start 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.191+0000 7f35ee884700 1 -- start start 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 0x7f35e8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:05.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81063c0 0x7f35e8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111690 con 0x7f35e81024b0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111510 con 0x7f35e81063c0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111810 con 0x7f35e8101af0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33518/0 (socket says 172.21.15.87:33518) 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 -- 172.21.15.87:0/2955265555 learned_addr learned my addr 172.21.15.87:0/2955265555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35e7fff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 0x7f35e8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ece21700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81063c0 0x7f35e8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 -- 172.21.15.87:0/2955265555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 msgr2=0x7f35e8197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 0x7f35e8197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 -- 172.21.15.87:0/2955265555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81063c0 msgr2=0x7f35e8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81063c0 0x7f35e8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 -- 172.21.15.87:0/2955265555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e81a7130 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35e7fff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 0x7f35e8197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35dffff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35d000bae0 tx=0x7f35d000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0026070 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d0003d70 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e81a73c0 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.192+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0021410 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.193+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a78f0 con 0x7f35e81024b0 2024-06-06T07:48:05.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.193+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8069180 con 0x7f35e81024b0 2024-06-06T07:48:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.195+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f35d0029070 con 0x7f35e81024b0 2024-06-06T07:48:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.195+0000 7f35e5ffb700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 0x7f35d40a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.195+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f35d00d0d40 con 0x7f35e81024b0 2024-06-06T07:48:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.195+0000 7f35e7fff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 0x7f35d40a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:05.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.196+0000 7f35e7fff700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 0x7f35d40a53f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009fd0 tx=0x7f35d8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:05.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.199+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35d0030080 con 0x7f35e81024b0 2024-06-06T07:48:05.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.442+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e802dca0 con 0x7f35e81024b0 2024-06-06T07:48:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.443+0000 7f35e5ffb700 1 -- 172.21.15.87:0/2955265555 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f35d0013070 con 0x7f35e81024b0 2024-06-06T07:48:05.443 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:05.444 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 msgr2=0x7f35d40a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 0x7f35d40a53f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009fd0 tx=0x7f35d8009260 comp rx=0 tx=0).stop 2024-06-06T07:48:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 msgr2=0x7f35e8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35d000bae0 tx=0x7f35d000bb10 comp rx=0 tx=0).stop 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 shutdown_connections 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.445+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e81063c0 0x7f35e8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f35d40a2f30 0x7f35d40a53f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35e8101af0 0x7f35e8197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 --2- 172.21.15.87:0/2955265555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e81024b0 0x7f35e8197dc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 >> 172.21.15.87:0/2955265555 conn(0x7f35e8077990 msgr2=0x7f35e8104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 shutdown_connections 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:05.446+0000 7f35ee884700 1 -- 172.21.15.87:0/2955265555 wait complete. 2024-06-06T07:48:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:06.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:06.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:05 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2955265555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-06T07:48:06.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:05 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2955265555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.170 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-06T07:48:06.170 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-06T07:48:06.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: Updating smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: Updating smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: Updating smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:48:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:06.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:05 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2955265555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 -- 172.21.15.87:0/3718969630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141012b0 msgr2=0x7f431410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 --2- 172.21.15.87:0/3718969630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141012b0 0x7f431410d770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f430c009cd0 tx=0x7f430c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 -- 172.21.15.87:0/3718969630 shutdown_connections 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 --2- 172.21.15.87:0/3718969630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141012b0 0x7f431410d770 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 --2- 172.21.15.87:0/3718969630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 0x7f4314100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 --2- 172.21.15.87:0/3718969630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141068b0 0x7f4314106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 -- 172.21.15.87:0/3718969630 >> 172.21.15.87:0/3718969630 conn(0x7f4314077a00 msgr2=0x7f4314077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 -- 172.21.15.87:0/3718969630 shutdown_connections 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.276+0000 7f431ab90640 1 -- 172.21.15.87:0/3718969630 wait complete. 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 Processor -- start 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 -- start start 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 0x7f43141983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 0x7f43141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4314110d40 con 0x7f43141068b0 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4314110bc0 con 0x7f43141008f0 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431ab90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4314110ec0 con 0x7f43141012b0 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33536/0 (socket says 172.21.15.87:33536) 2024-06-06T07:48:06.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 -- 172.21.15.87:0/3572957912 learned_addr learned my addr 172.21.15.87:0/3572957912 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f4313fff640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 0x7f43141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 -- 172.21.15.87:0/3572957912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 msgr2=0x7f43141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 0x7f43141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 -- 172.21.15.87:0/3572957912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 msgr2=0x7f43141983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431892c640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 0x7f43141983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 0x7f43141983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 -- 172.21.15.87:0/3572957912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419d3a0 con 0x7f43141068b0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f4313fff640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 0x7f43141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.277+0000 7f431912d640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.278+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c004040 con 0x7f43141068b0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.278+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c0046c0 con 0x7f43141068b0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.278+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c046480 con 0x7f43141068b0 2024-06-06T07:48:06.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.278+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431419d630 con 0x7f43141068b0 2024-06-06T07:48:06.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.278+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a4f20 con 0x7f43141068b0 2024-06-06T07:48:06.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.279+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4314068c10 con 0x7f43141068b0 2024-06-06T07:48:06.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.280+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f430c030710 con 0x7f43141068b0 2024-06-06T07:48:06.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.280+0000 7f4311ffb640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 0x7f43040a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.280+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f430c02e030 con 0x7f43141068b0 2024-06-06T07:48:06.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.280+0000 7f431892c640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 0x7f43040a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.281+0000 7f431892c640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 0x7f43040a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f430800a810 tx=0x7f4308008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.285+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f430c04b060 con 0x7f43141068b0 2024-06-06T07:48:06.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.460+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4314061de0 con 0x7f43040a2e20 2024-06-06T07:48:06.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.470+0000 7f4311ffb640 1 -- 172.21.15.87:0/3572957912 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339521 (secure 0 0 0) 0x7f4314061de0 con 0x7f43040a2e20 2024-06-06T07:48:06.477 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:06.480 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 msgr2=0x7f43040a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 0x7f43040a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f430800a810 tx=0x7f4308008040 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 msgr2=0x7f431419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0039f0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 shutdown_connections 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43040a2e20 0x7f43040a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43141068b0 0x7f431419cc70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43141012b0 0x7f43141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 --2- 172.21.15.87:0/3572957912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43141008f0 0x7f43141983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 >> 172.21.15.87:0/3572957912 conn(0x7f4314077a00 msgr2=0x7f431406a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:06.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 shutdown_connections 2024-06-06T07:48:06.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.473+0000 7f431ab90640 1 -- 172.21.15.87:0/3572957912 wait complete. 2024-06-06T07:48:06.484 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T07:48:06.408144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189568,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1326,"num_read_kb":821,"num_write":486,"num_write_kb":4389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391320,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550504,"statfs":{"total":1151940427776,"available":1151539716096,"internally_reserved":0,"allocated":17588224,"data_stored":10493547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.993037"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.167728+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:46:46.167728+0000","last_peered":"2024-06-06T07:46:46.167728+0000","last_clean":"2024-06-06T07:46:46.167728+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:46:46.167728+0000","last_undegraded":"2024-06-06T07:46:46.167728+0000","last_fullsized":"2024-06-06T07:46:46.167728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469779+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469779+0000","last_peered":"2024-06-06T07:48:01.469779+0000","last_clean":"2024-06-06T07:48:01.469779+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469779+0000","last_undegraded":"2024-06-06T07:48:01.469779+0000","last_fullsized":"2024-06-06T07:48:01.469779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469577+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469577+0000","last_peered":"2024-06-06T07:48:01.469577+0000","last_clean":"2024-06-06T07:48:01.469577+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469577+0000","last_undegraded":"2024-06-06T07:48:01.469577+0000","last_fullsized":"2024-06-06T07:48:01.469577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168125+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:46:46.168125+0000","last_peered":"2024-06-06T07:46:46.168125+0000","last_clean":"2024-06-06T07:46:46.168125+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:46:46.168125+0000","last_undegraded":"2024-06-06T07:46:46.168125+0000","last_fullsized":"2024-06-06T07:46:46.168125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489224+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:00.489224+0000","last_peered":"2024-06-06T07:48:00.489224+0000","last_clean":"2024-06-06T07:48:00.489224+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:00.489224+0000","last_undegraded":"2024-06-06T07:48:00.489224+0000","last_fullsized":"2024-06-06T07:48:00.489224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469788+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469788+0000","last_peered":"2024-06-06T07:48:01.469788+0000","last_clean":"2024-06-06T07:48:01.469788+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469788+0000","last_undegraded":"2024-06-06T07:48:01.469788+0000","last_fullsized":"2024-06-06T07:48:01.469788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.503221+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:00.503221+0000","last_peered":"2024-06-06T07:48:00.503221+0000","last_clean":"2024-06-06T07:48:00.503221+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:00.503221+0000","last_undegraded":"2024-06-06T07:48:00.503221+0000","last_fullsized":"2024-06-06T07:48:00.503221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228208+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:45:48.228208+0000","last_peered":"2024-06-06T07:45:48.228208+0000","last_clean":"2024-06-06T07:45:48.228208+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:45:48.228208+0000","last_undegraded":"2024-06-06T07:45:48.228208+0000","last_fullsized":"2024-06-06T07:45:48.228208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469713+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469713+0000","last_peered":"2024-06-06T07:48:01.469713+0000","last_clean":"2024-06-06T07:48:01.469713+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469713+0000","last_undegraded":"2024-06-06T07:48:01.469713+0000","last_fullsized":"2024-06-06T07:48:01.469713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.502853+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:00.502853+0000","last_peered":"2024-06-06T07:48:00.502853+0000","last_clean":"2024-06-06T07:48:00.502853+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:00.502853+0000","last_undegraded":"2024-06-06T07:48:00.502853+0000","last_fullsized":"2024-06-06T07:48:00.502853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":452,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.637707+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:00.637707+0000","last_peered":"2024-06-06T07:48:00.637707+0000","last_clean":"2024-06-06T07:48:00.637707+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:00.637707+0000","last_undegraded":"2024-06-06T07:48:00.637707+0000","last_fullsized":"2024-06-06T07:48:00.637707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":365,"num_read_kb":244,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501562+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501562+0000","last_peered":"2024-06-06T07:48:00.501562+0000","last_clean":"2024-06-06T07:48:00.501562+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501562+0000","last_undegraded":"2024-06-06T07:48:00.501562+0000","last_fullsized":"2024-06-06T07:48:00.501562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227501+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:45:48.227501+0000","last_peered":"2024-06-06T07:45:48.227501+0000","last_clean":"2024-06-06T07:45:48.227501+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:45:48.227501+0000","last_undegraded":"2024-06-06T07:45:48.227501+0000","last_fullsized":"2024-06-06T07:45:48.227501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'6","reported_seq":234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554773+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:00.554773+0000","last_peered":"2024-06-06T07:48:00.554773+0000","last_clean":"2024-06-06T07:48:00.554773+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:00.554773+0000","last_undegraded":"2024-06-06T07:48:00.554773+0000","last_fullsized":"2024-06-06T07:48:00.554773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.248883+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:45:48.248883+0000","last_peered":"2024-06-06T07:45:48.248883+0000","last_clean":"2024-06-06T07:45:48.248883+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:45:48.248883+0000","last_undegraded":"2024-06-06T07:45:48.248883+0000","last_fullsized":"2024-06-06T07:45:48.248883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.252519+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:45:48.252519+0000","last_peered":"2024-06-06T07:45:48.252519+0000","last_clean":"2024-06-06T07:45:48.252519+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:45:48.252519+0000","last_undegraded":"2024-06-06T07:45:48.252519+0000","last_fullsized":"2024-06-06T07:45:48.252519+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171950+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:46:46.171950+0000","last_peered":"2024-06-06T07:46:46.171950+0000","last_clean":"2024-06-06T07:46:46.171950+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:46:46.171950+0000","last_undegraded":"2024-06-06T07:46:46.171950+0000","last_fullsized":"2024-06-06T07:46:46.171950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501696+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501696+0000","last_peered":"2024-06-06T07:48:00.501696+0000","last_clean":"2024-06-06T07:48:00.501696+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501696+0000","last_undegraded":"2024-06-06T07:48:00.501696+0000","last_fullsized":"2024-06-06T07:48:00.501696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395576+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395576+0000","last_peered":"2024-06-06T07:48:00.395576+0000","last_clean":"2024-06-06T07:48:00.395576+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395576+0000","last_undegraded":"2024-06-06T07:48:00.395576+0000","last_fullsized":"2024-06-06T07:48:00.395576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501494+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501494+0000","last_peered":"2024-06-06T07:48:00.501494+0000","last_clean":"2024-06-06T07:48:00.501494+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501494+0000","last_undegraded":"2024-06-06T07:48:00.501494+0000","last_fullsized":"2024-06-06T07:48:00.501494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498708+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498708+0000","last_peered":"2024-06-06T07:48:00.498708+0000","last_clean":"2024-06-06T07:48:00.498708+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498708+0000","last_undegraded":"2024-06-06T07:48:00.498708+0000","last_fullsized":"2024-06-06T07:48:00.498708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:05.561191+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:05.561191+0000","last_peered":"2024-06-06T07:48:05.561191+0000","last_clean":"2024-06-06T07:48:05.561191+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:05.561191+0000","last_undegraded":"2024-06-06T07:48:05.561191+0000","last_fullsized":"2024-06-06T07:48:05.561191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491258+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491258+0000","last_peered":"2024-06-06T07:48:00.491258+0000","last_clean":"2024-06-06T07:48:00.491258+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491258+0000","last_undegraded":"2024-06-06T07:48:00.491258+0000","last_fullsized":"2024-06-06T07:48:00.491258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554746+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554746+0000","last_peered":"2024-06-06T07:48:00.554746+0000","last_clean":"2024-06-06T07:48:00.554746+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554746+0000","last_undegraded":"2024-06-06T07:48:00.554746+0000","last_fullsized":"2024-06-06T07:48:00.554746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":382,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395397+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:00.395397+0000","last_peered":"2024-06-06T07:48:00.395397+0000","last_clean":"2024-06-06T07:48:00.395397+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:00.395397+0000","last_undegraded":"2024-06-06T07:48:00.395397+0000","last_fullsized":"2024-06-06T07:48:00.395397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227366+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:45:48.227366+0000","last_peered":"2024-06-06T07:45:48.227366+0000","last_clean":"2024-06-06T07:45:48.227366+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:45:48.227366+0000","last_undegraded":"2024-06-06T07:45:48.227366+0000","last_fullsized":"2024-06-06T07:45:48.227366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491349+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491349+0000","last_peered":"2024-06-06T07:48:00.491349+0000","last_clean":"2024-06-06T07:48:00.491349+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491349+0000","last_undegraded":"2024-06-06T07:48:00.491349+0000","last_fullsized":"2024-06-06T07:48:00.491349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393936+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393936+0000","last_peered":"2024-06-06T07:48:00.393936+0000","last_clean":"2024-06-06T07:48:00.393936+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393936+0000","last_undegraded":"2024-06-06T07:48:00.393936+0000","last_fullsized":"2024-06-06T07:48:00.393936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490908+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.490908+0000","last_peered":"2024-06-06T07:48:00.490908+0000","last_clean":"2024-06-06T07:48:00.490908+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.490908+0000","last_undegraded":"2024-06-06T07:48:00.490908+0000","last_fullsized":"2024-06-06T07:48:00.490908+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.369652+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:01.369652+0000","last_peered":"2024-06-06T07:48:01.369652+0000","last_clean":"2024-06-06T07:48:01.369652+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:01.369652+0000","last_undegraded":"2024-06-06T07:48:01.369652+0000","last_fullsized":"2024-06-06T07:48:01.369652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491172+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491172+0000","last_peered":"2024-06-06T07:48:00.491172+0000","last_clean":"2024-06-06T07:48:00.491172+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491172+0000","last_undegraded":"2024-06-06T07:48:00.491172+0000","last_fullsized":"2024-06-06T07:48:00.491172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555078+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555078+0000","last_peered":"2024-06-06T07:48:00.555078+0000","last_clean":"2024-06-06T07:48:00.555078+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555078+0000","last_undegraded":"2024-06-06T07:48:00.555078+0000","last_fullsized":"2024-06-06T07:48:00.555078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469845+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469845+0000","last_peered":"2024-06-06T07:48:01.469845+0000","last_clean":"2024-06-06T07:48:01.469845+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469845+0000","last_undegraded":"2024-06-06T07:48:01.469845+0000","last_fullsized":"2024-06-06T07:48:01.469845+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581656+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581656+0000","last_peered":"2024-06-06T07:48:00.581656+0000","last_clean":"2024-06-06T07:48:00.581656+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581656+0000","last_undegraded":"2024-06-06T07:48:00.581656+0000","last_fullsized":"2024-06-06T07:48:00.581656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490919+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490919+0000","last_peered":"2024-06-06T07:48:00.490919+0000","last_clean":"2024-06-06T07:48:00.490919+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490919+0000","last_undegraded":"2024-06-06T07:48:00.490919+0000","last_fullsized":"2024-06-06T07:48:00.490919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491031+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491031+0000","last_peered":"2024-06-06T07:48:00.491031+0000","last_clean":"2024-06-06T07:48:00.491031+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491031+0000","last_undegraded":"2024-06-06T07:48:00.491031+0000","last_fullsized":"2024-06-06T07:48:00.491031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171863+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:46:46.171863+0000","last_peered":"2024-06-06T07:46:46.171863+0000","last_clean":"2024-06-06T07:46:46.171863+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:46:46.171863+0000","last_undegraded":"2024-06-06T07:46:46.171863+0000","last_fullsized":"2024-06-06T07:46:46.171863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392587+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:00.392587+0000","last_peered":"2024-06-06T07:48:00.392587+0000","last_clean":"2024-06-06T07:48:00.392587+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:00.392587+0000","last_undegraded":"2024-06-06T07:48:00.392587+0000","last_fullsized":"2024-06-06T07:48:00.392587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581868+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581868+0000","last_peered":"2024-06-06T07:48:00.581868+0000","last_clean":"2024-06-06T07:48:00.581868+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581868+0000","last_undegraded":"2024-06-06T07:48:00.581868+0000","last_fullsized":"2024-06-06T07:48:00.581868+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.597759+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:00.597759+0000","last_peered":"2024-06-06T07:48:00.597759+0000","last_clean":"2024-06-06T07:48:00.597759+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:00.597759+0000","last_undegraded":"2024-06-06T07:48:00.597759+0000","last_fullsized":"2024-06-06T07:48:00.597759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172843+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:46:46.172843+0000","last_peered":"2024-06-06T07:46:46.172843+0000","last_clean":"2024-06-06T07:46:46.172843+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:46:46.172843+0000","last_undegraded":"2024-06-06T07:46:46.172843+0000","last_fullsized":"2024-06-06T07:46:46.172843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392753+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392753+0000","last_peered":"2024-06-06T07:48:00.392753+0000","last_clean":"2024-06-06T07:48:00.392753+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392753+0000","last_undegraded":"2024-06-06T07:48:00.392753+0000","last_fullsized":"2024-06-06T07:48:00.392753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392493+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392493+0000","last_peered":"2024-06-06T07:48:00.392493+0000","last_clean":"2024-06-06T07:48:00.392493+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392493+0000","last_undegraded":"2024-06-06T07:48:00.392493+0000","last_fullsized":"2024-06-06T07:48:00.392493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469791+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469791+0000","last_peered":"2024-06-06T07:48:01.469791+0000","last_clean":"2024-06-06T07:48:01.469791+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469791+0000","last_undegraded":"2024-06-06T07:48:01.469791+0000","last_fullsized":"2024-06-06T07:48:01.469791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393766+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393766+0000","last_peered":"2024-06-06T07:48:00.393766+0000","last_clean":"2024-06-06T07:48:00.393766+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393766+0000","last_undegraded":"2024-06-06T07:48:00.393766+0000","last_fullsized":"2024-06-06T07:48:00.393766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.170081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:46:46.170081+0000","last_peered":"2024-06-06T07:46:46.170081+0000","last_clean":"2024-06-06T07:46:46.170081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:46:46.170081+0000","last_undegraded":"2024-06-06T07:46:46.170081+0000","last_fullsized":"2024-06-06T07:46:46.170081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393093+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393093+0000","last_peered":"2024-06-06T07:48:00.393093+0000","last_clean":"2024-06-06T07:48:00.393093+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393093+0000","last_undegraded":"2024-06-06T07:48:00.393093+0000","last_fullsized":"2024-06-06T07:48:00.393093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491388+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491388+0000","last_peered":"2024-06-06T07:48:00.491388+0000","last_clean":"2024-06-06T07:48:00.491388+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491388+0000","last_undegraded":"2024-06-06T07:48:00.491388+0000","last_fullsized":"2024-06-06T07:48:00.491388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169888+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:46:46.169888+0000","last_peered":"2024-06-06T07:46:46.169888+0000","last_clean":"2024-06-06T07:46:46.169888+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:46:46.169888+0000","last_undegraded":"2024-06-06T07:46:46.169888+0000","last_fullsized":"2024-06-06T07:46:46.169888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598966,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1190,"num_read_kb":702,"num_write":303,"num_write_kb":1449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1796898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383744,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":570184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645381,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28960,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716192,"statfs":{"total":95995035648,"available":95965380608,"internally_reserved":0,"allocated":1990656,"data_stored":1372586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1177437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1228800,"data_stored":644687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1245184,"data_stored":644624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1691648,"data_stored":1100425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":580904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37200,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707952,"statfs":{"total":95995035648,"available":95956942848,"internally_reserved":0,"allocated":1908736,"data_stored":1303786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:06.490 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- 172.21.15.87:0/16023071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280100f10 msgr2=0x7f0280106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 --2- 172.21.15.87:0/16023071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280100f10 0x7f0280106640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f027c009900 tx=0x7f027c02a410 comp rx=0 tx=0).stop 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- 172.21.15.87:0/16023071 shutdown_connections 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 --2- 172.21.15.87:0/16023071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280106d20 0x7f028010e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 --2- 172.21.15.87:0/16023071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280100f10 0x7f0280106640 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 --2- 172.21.15.87:0/16023071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0280100560 0x7f0280100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- 172.21.15.87:0/16023071 >> 172.21.15.87:0/16023071 conn(0x7f0280075b80 msgr2=0x7f0280075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- 172.21.15.87:0/16023071 shutdown_connections 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- 172.21.15.87:0/16023071 wait complete. 2024-06-06T07:48:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 Processor -- start 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.626+0000 7f0287930640 1 -- start start 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280100560 0x7f0280198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 0x7f028019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0280111dc0 con 0x7f0280106d20 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0280111c40 con 0x7f0280100560 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0280111f40 con 0x7f028019cbe0 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33552/0 (socket says 172.21.15.87:33552) 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 -- 172.21.15.87:0/4230335708 learned_addr learned my addr 172.21.15.87:0/4230335708 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f02856cc640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280100560 0x7f0280198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0285ecd640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 0x7f028019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 -- 172.21.15.87:0/4230335708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 msgr2=0x7f028019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 0x7f028019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 -- 172.21.15.87:0/4230335708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280100560 msgr2=0x7f0280198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280100560 0x7f0280198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 -- 172.21.15.87:0/4230335708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028019d6d0 con 0x7f0280106d20 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0285ecd640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 0x7f028019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0284ecb640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f027c005940 tx=0x7f027c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c03f030 con 0x7f0280106d20 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c003ce0 con 0x7f0280106d20 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c035070 con 0x7f0280106d20 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028019d960 con 0x7f0280106d20 2024-06-06T07:48:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.627+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02801a5180 con 0x7f0280106d20 2024-06-06T07:48:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.628+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0280068c10 con 0x7f0280106d20 2024-06-06T07:48:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.629+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f027c042030 con 0x7f0280106d20 2024-06-06T07:48:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.630+0000 7f02727fc640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 0x7f026c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.630+0000 7f02856cc640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 0x7f026c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.630+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f027c0f01c0 con 0x7f0280106d20 2024-06-06T07:48:06.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.631+0000 7f02856cc640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 0x7f026c0a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0274005fd0 tx=0x7f027400b590 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.634+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f027c0bc890 con 0x7f0280106d20 2024-06-06T07:48:06.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.808+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0280100f10 con 0x7f026c0a2ea0 2024-06-06T07:48:06.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.817+0000 7f02727fc640 1 -- 172.21.15.87:0/4230335708 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339521 (secure 0 0 0) 0x7f0280100f10 con 0x7f026c0a2ea0 2024-06-06T07:48:06.821 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:06.822 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.820+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 msgr2=0x7f026c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.820+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 0x7f026c0a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0274005fd0 tx=0x7f027400b590 comp rx=0 tx=0).stop 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.820+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 msgr2=0x7f0280198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.820+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f027c005940 tx=0x7f027c0042f0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 shutdown_connections 2024-06-06T07:48:06.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f026c0a2ea0 0x7f026c0a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f028019cbe0 0x7f028019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0280106d20 0x7f0280198860 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 --2- 172.21.15.87:0/4230335708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0280100560 0x7f0280198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 >> 172.21.15.87:0/4230335708 conn(0x7f0280075b80 msgr2=0x7f02800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 shutdown_connections 2024-06-06T07:48:06.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.821+0000 7f0287930640 1 -- 172.21.15.87:0/4230335708 wait complete. 2024-06-06T07:48:06.832 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T07:48:06.408144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189568,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1326,"num_read_kb":821,"num_write":486,"num_write_kb":4389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391320,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550504,"statfs":{"total":1151940427776,"available":1151539716096,"internally_reserved":0,"allocated":17588224,"data_stored":10493547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.993037"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.167728+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:46:46.167728+0000","last_peered":"2024-06-06T07:46:46.167728+0000","last_clean":"2024-06-06T07:46:46.167728+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:46:46.167728+0000","last_undegraded":"2024-06-06T07:46:46.167728+0000","last_fullsized":"2024-06-06T07:46:46.167728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469779+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469779+0000","last_peered":"2024-06-06T07:48:01.469779+0000","last_clean":"2024-06-06T07:48:01.469779+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469779+0000","last_undegraded":"2024-06-06T07:48:01.469779+0000","last_fullsized":"2024-06-06T07:48:01.469779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469577+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469577+0000","last_peered":"2024-06-06T07:48:01.469577+0000","last_clean":"2024-06-06T07:48:01.469577+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469577+0000","last_undegraded":"2024-06-06T07:48:01.469577+0000","last_fullsized":"2024-06-06T07:48:01.469577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168125+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:46:46.168125+0000","last_peered":"2024-06-06T07:46:46.168125+0000","last_clean":"2024-06-06T07:46:46.168125+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:46:46.168125+0000","last_undegraded":"2024-06-06T07:46:46.168125+0000","last_fullsized":"2024-06-06T07:46:46.168125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489224+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:00.489224+0000","last_peered":"2024-06-06T07:48:00.489224+0000","last_clean":"2024-06-06T07:48:00.489224+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:00.489224+0000","last_undegraded":"2024-06-06T07:48:00.489224+0000","last_fullsized":"2024-06-06T07:48:00.489224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469788+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469788+0000","last_peered":"2024-06-06T07:48:01.469788+0000","last_clean":"2024-06-06T07:48:01.469788+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469788+0000","last_undegraded":"2024-06-06T07:48:01.469788+0000","last_fullsized":"2024-06-06T07:48:01.469788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.503221+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:00.503221+0000","last_peered":"2024-06-06T07:48:00.503221+0000","last_clean":"2024-06-06T07:48:00.503221+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:00.503221+0000","last_undegraded":"2024-06-06T07:48:00.503221+0000","last_fullsized":"2024-06-06T07:48:00.503221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228208+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:45:48.228208+0000","last_peered":"2024-06-06T07:45:48.228208+0000","last_clean":"2024-06-06T07:45:48.228208+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:45:48.228208+0000","last_undegraded":"2024-06-06T07:45:48.228208+0000","last_fullsized":"2024-06-06T07:45:48.228208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469713+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469713+0000","last_peered":"2024-06-06T07:48:01.469713+0000","last_clean":"2024-06-06T07:48:01.469713+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469713+0000","last_undegraded":"2024-06-06T07:48:01.469713+0000","last_fullsized":"2024-06-06T07:48:01.469713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.502853+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:00.502853+0000","last_peered":"2024-06-06T07:48:00.502853+0000","last_clean":"2024-06-06T07:48:00.502853+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:00.502853+0000","last_undegraded":"2024-06-06T07:48:00.502853+0000","last_fullsized":"2024-06-06T07:48:00.502853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":452,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.637707+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:00.637707+0000","last_peered":"2024-06-06T07:48:00.637707+0000","last_clean":"2024-06-06T07:48:00.637707+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:00.637707+0000","last_undegraded":"2024-06-06T07:48:00.637707+0000","last_fullsized":"2024-06-06T07:48:00.637707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":365,"num_read_kb":244,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501562+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501562+0000","last_peered":"2024-06-06T07:48:00.501562+0000","last_clean":"2024-06-06T07:48:00.501562+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501562+0000","last_undegraded":"2024-06-06T07:48:00.501562+0000","last_fullsized":"2024-06-06T07:48:00.501562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227501+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:45:48.227501+0000","last_peered":"2024-06-06T07:45:48.227501+0000","last_clean":"2024-06-06T07:45:48.227501+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:45:48.227501+0000","last_undegraded":"2024-06-06T07:45:48.227501+0000","last_fullsized":"2024-06-06T07:45:48.227501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'6","reported_seq":234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554773+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:00.554773+0000","last_peered":"2024-06-06T07:48:00.554773+0000","last_clean":"2024-06-06T07:48:00.554773+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:00.554773+0000","last_undegraded":"2024-06-06T07:48:00.554773+0000","last_fullsized":"2024-06-06T07:48:00.554773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.248883+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:45:48.248883+0000","last_peered":"2024-06-06T07:45:48.248883+0000","last_clean":"2024-06-06T07:45:48.248883+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:45:48.248883+0000","last_undegraded":"2024-06-06T07:45:48.248883+0000","last_fullsized":"2024-06-06T07:45:48.248883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.252519+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:45:48.252519+0000","last_peered":"2024-06-06T07:45:48.252519+0000","last_clean":"2024-06-06T07:45:48.252519+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:45:48.252519+0000","last_undegraded":"2024-06-06T07:45:48.252519+0000","last_fullsized":"2024-06-06T07:45:48.252519+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171950+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:46:46.171950+0000","last_peered":"2024-06-06T07:46:46.171950+0000","last_clean":"2024-06-06T07:46:46.171950+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:46:46.171950+0000","last_undegraded":"2024-06-06T07:46:46.171950+0000","last_fullsized":"2024-06-06T07:46:46.171950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501696+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501696+0000","last_peered":"2024-06-06T07:48:00.501696+0000","last_clean":"2024-06-06T07:48:00.501696+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501696+0000","last_undegraded":"2024-06-06T07:48:00.501696+0000","last_fullsized":"2024-06-06T07:48:00.501696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395576+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395576+0000","last_peered":"2024-06-06T07:48:00.395576+0000","last_clean":"2024-06-06T07:48:00.395576+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395576+0000","last_undegraded":"2024-06-06T07:48:00.395576+0000","last_fullsized":"2024-06-06T07:48:00.395576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501494+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501494+0000","last_peered":"2024-06-06T07:48:00.501494+0000","last_clean":"2024-06-06T07:48:00.501494+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501494+0000","last_undegraded":"2024-06-06T07:48:00.501494+0000","last_fullsized":"2024-06-06T07:48:00.501494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498708+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498708+0000","last_peered":"2024-06-06T07:48:00.498708+0000","last_clean":"2024-06-06T07:48:00.498708+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498708+0000","last_undegraded":"2024-06-06T07:48:00.498708+0000","last_fullsized":"2024-06-06T07:48:00.498708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:05.561191+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:05.561191+0000","last_peered":"2024-06-06T07:48:05.561191+0000","last_clean":"2024-06-06T07:48:05.561191+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:05.561191+0000","last_undegraded":"2024-06-06T07:48:05.561191+0000","last_fullsized":"2024-06-06T07:48:05.561191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491258+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491258+0000","last_peered":"2024-06-06T07:48:00.491258+0000","last_clean":"2024-06-06T07:48:00.491258+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491258+0000","last_undegraded":"2024-06-06T07:48:00.491258+0000","last_fullsized":"2024-06-06T07:48:00.491258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554746+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554746+0000","last_peered":"2024-06-06T07:48:00.554746+0000","last_clean":"2024-06-06T07:48:00.554746+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554746+0000","last_undegraded":"2024-06-06T07:48:00.554746+0000","last_fullsized":"2024-06-06T07:48:00.554746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":382,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395397+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:00.395397+0000","last_peered":"2024-06-06T07:48:00.395397+0000","last_clean":"2024-06-06T07:48:00.395397+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:00.395397+0000","last_undegraded":"2024-06-06T07:48:00.395397+0000","last_fullsized":"2024-06-06T07:48:00.395397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227366+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:45:48.227366+0000","last_peered":"2024-06-06T07:45:48.227366+0000","last_clean":"2024-06-06T07:45:48.227366+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:45:48.227366+0000","last_undegraded":"2024-06-06T07:45:48.227366+0000","last_fullsized":"2024-06-06T07:45:48.227366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491349+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491349+0000","last_peered":"2024-06-06T07:48:00.491349+0000","last_clean":"2024-06-06T07:48:00.491349+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491349+0000","last_undegraded":"2024-06-06T07:48:00.491349+0000","last_fullsized":"2024-06-06T07:48:00.491349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393936+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393936+0000","last_peered":"2024-06-06T07:48:00.393936+0000","last_clean":"2024-06-06T07:48:00.393936+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393936+0000","last_undegraded":"2024-06-06T07:48:00.393936+0000","last_fullsized":"2024-06-06T07:48:00.393936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490908+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.490908+0000","last_peered":"2024-06-06T07:48:00.490908+0000","last_clean":"2024-06-06T07:48:00.490908+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.490908+0000","last_undegraded":"2024-06-06T07:48:00.490908+0000","last_fullsized":"2024-06-06T07:48:00.490908+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.369652+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:01.369652+0000","last_peered":"2024-06-06T07:48:01.369652+0000","last_clean":"2024-06-06T07:48:01.369652+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:01.369652+0000","last_undegraded":"2024-06-06T07:48:01.369652+0000","last_fullsized":"2024-06-06T07:48:01.369652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491172+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491172+0000","last_peered":"2024-06-06T07:48:00.491172+0000","last_clean":"2024-06-06T07:48:00.491172+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491172+0000","last_undegraded":"2024-06-06T07:48:00.491172+0000","last_fullsized":"2024-06-06T07:48:00.491172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555078+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555078+0000","last_peered":"2024-06-06T07:48:00.555078+0000","last_clean":"2024-06-06T07:48:00.555078+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555078+0000","last_undegraded":"2024-06-06T07:48:00.555078+0000","last_fullsized":"2024-06-06T07:48:00.555078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469845+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469845+0000","last_peered":"2024-06-06T07:48:01.469845+0000","last_clean":"2024-06-06T07:48:01.469845+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469845+0000","last_undegraded":"2024-06-06T07:48:01.469845+0000","last_fullsized":"2024-06-06T07:48:01.469845+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581656+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581656+0000","last_peered":"2024-06-06T07:48:00.581656+0000","last_clean":"2024-06-06T07:48:00.581656+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581656+0000","last_undegraded":"2024-06-06T07:48:00.581656+0000","last_fullsized":"2024-06-06T07:48:00.581656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490919+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490919+0000","last_peered":"2024-06-06T07:48:00.490919+0000","last_clean":"2024-06-06T07:48:00.490919+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490919+0000","last_undegraded":"2024-06-06T07:48:00.490919+0000","last_fullsized":"2024-06-06T07:48:00.490919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491031+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491031+0000","last_peered":"2024-06-06T07:48:00.491031+0000","last_clean":"2024-06-06T07:48:00.491031+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491031+0000","last_undegraded":"2024-06-06T07:48:00.491031+0000","last_fullsized":"2024-06-06T07:48:00.491031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171863+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:46:46.171863+0000","last_peered":"2024-06-06T07:46:46.171863+0000","last_clean":"2024-06-06T07:46:46.171863+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:46:46.171863+0000","last_undegraded":"2024-06-06T07:46:46.171863+0000","last_fullsized":"2024-06-06T07:46:46.171863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392587+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:00.392587+0000","last_peered":"2024-06-06T07:48:00.392587+0000","last_clean":"2024-06-06T07:48:00.392587+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:00.392587+0000","last_undegraded":"2024-06-06T07:48:00.392587+0000","last_fullsized":"2024-06-06T07:48:00.392587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581868+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581868+0000","last_peered":"2024-06-06T07:48:00.581868+0000","last_clean":"2024-06-06T07:48:00.581868+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581868+0000","last_undegraded":"2024-06-06T07:48:00.581868+0000","last_fullsized":"2024-06-06T07:48:00.581868+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.597759+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:00.597759+0000","last_peered":"2024-06-06T07:48:00.597759+0000","last_clean":"2024-06-06T07:48:00.597759+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:00.597759+0000","last_undegraded":"2024-06-06T07:48:00.597759+0000","last_fullsized":"2024-06-06T07:48:00.597759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172843+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:46:46.172843+0000","last_peered":"2024-06-06T07:46:46.172843+0000","last_clean":"2024-06-06T07:46:46.172843+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:46:46.172843+0000","last_undegraded":"2024-06-06T07:46:46.172843+0000","last_fullsized":"2024-06-06T07:46:46.172843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392753+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392753+0000","last_peered":"2024-06-06T07:48:00.392753+0000","last_clean":"2024-06-06T07:48:00.392753+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392753+0000","last_undegraded":"2024-06-06T07:48:00.392753+0000","last_fullsized":"2024-06-06T07:48:00.392753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392493+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392493+0000","last_peered":"2024-06-06T07:48:00.392493+0000","last_clean":"2024-06-06T07:48:00.392493+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392493+0000","last_undegraded":"2024-06-06T07:48:00.392493+0000","last_fullsized":"2024-06-06T07:48:00.392493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469791+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469791+0000","last_peered":"2024-06-06T07:48:01.469791+0000","last_clean":"2024-06-06T07:48:01.469791+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469791+0000","last_undegraded":"2024-06-06T07:48:01.469791+0000","last_fullsized":"2024-06-06T07:48:01.469791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393766+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393766+0000","last_peered":"2024-06-06T07:48:00.393766+0000","last_clean":"2024-06-06T07:48:00.393766+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393766+0000","last_undegraded":"2024-06-06T07:48:00.393766+0000","last_fullsized":"2024-06-06T07:48:00.393766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.170081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:46:46.170081+0000","last_peered":"2024-06-06T07:46:46.170081+0000","last_clean":"2024-06-06T07:46:46.170081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:46:46.170081+0000","last_undegraded":"2024-06-06T07:46:46.170081+0000","last_fullsized":"2024-06-06T07:46:46.170081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393093+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393093+0000","last_peered":"2024-06-06T07:48:00.393093+0000","last_clean":"2024-06-06T07:48:00.393093+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393093+0000","last_undegraded":"2024-06-06T07:48:00.393093+0000","last_fullsized":"2024-06-06T07:48:00.393093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491388+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491388+0000","last_peered":"2024-06-06T07:48:00.491388+0000","last_clean":"2024-06-06T07:48:00.491388+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491388+0000","last_undegraded":"2024-06-06T07:48:00.491388+0000","last_fullsized":"2024-06-06T07:48:00.491388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169888+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:46:46.169888+0000","last_peered":"2024-06-06T07:46:46.169888+0000","last_clean":"2024-06-06T07:46:46.169888+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:46:46.169888+0000","last_undegraded":"2024-06-06T07:46:46.169888+0000","last_fullsized":"2024-06-06T07:46:46.169888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598966,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1190,"num_read_kb":702,"num_write":303,"num_write_kb":1449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1796898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383744,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":570184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645381,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28960,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716192,"statfs":{"total":95995035648,"available":95965380608,"internally_reserved":0,"allocated":1990656,"data_stored":1372586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1177437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1228800,"data_stored":644687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1245184,"data_stored":644624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1691648,"data_stored":1100425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":580904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37200,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707952,"statfs":{"total":95995035648,"available":95956942848,"internally_reserved":0,"allocated":1908736,"data_stored":1303786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:06.838 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-06T07:48:06.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:48:06.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.973+0000 7f6156b6a640 1 -- 172.21.15.87:0/162513588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150106880 msgr2=0x7f6150106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.973+0000 7f6156b6a640 1 --2- 172.21.15.87:0/162513588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150106880 0x7f6150106c60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6144009930 tx=0x7f614402a420 comp rx=0 tx=0).stop 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- 172.21.15.87:0/162513588 shutdown_connections 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- 172.21.15.87:0/162513588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150101230 0x7f615010d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- 172.21.15.87:0/162513588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150100870 0x7f6150100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- 172.21.15.87:0/162513588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150106880 0x7f6150106c60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- 172.21.15.87:0/162513588 >> 172.21.15.87:0/162513588 conn(0x7f6150076c20 msgr2=0x7f6150077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- 172.21.15.87:0/162513588 shutdown_connections 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- 172.21.15.87:0/162513588 wait complete. 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 Processor -- start 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- start start 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150101230 0x7f6150198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150106880 0x7f615019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6150111d90 con 0x7f6150100870 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6150111c10 con 0x7f6150106880 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6156b6a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6150111f10 con 0x7f6150101230 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6154906640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.974+0000 7f6154906640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49246/0 (socket says 172.21.15.87:49246) 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 -- 172.21.15.87:0/1048356324 learned_addr learned my addr 172.21.15.87:0/1048356324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f614ffff640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150101230 0x7f6150198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6155107640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150106880 0x7f615019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 -- 172.21.15.87:0/1048356324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150101230 msgr2=0x7f6150198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150101230 0x7f6150198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 -- 172.21.15.87:0/1048356324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150106880 msgr2=0x7f615019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150106880 0x7f615019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 -- 172.21.15.87:0/1048356324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615019d220 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6154906640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6144009230 tx=0x7f6144004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614403f030 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6144003ad0 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615019d4b0 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035070 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.975+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a4da0 con 0x7f6150100870 2024-06-06T07:48:06.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.976+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150068c10 con 0x7f6150100870 2024-06-06T07:48:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.977+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f6144042030 con 0x7f6150100870 2024-06-06T07:48:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.978+0000 7f614dffb640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 0x7f61400a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.978+0000 7f614ffff640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 0x7f61400a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.978+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f61440eff90 con 0x7f6150100870 2024-06-06T07:48:06.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.978+0000 7f614ffff640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 0x7f61400a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61501000d0 tx=0x7f613c005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:06.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:06.982+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f61440bc010 con 0x7f6150100870 2024-06-06T07:48:07.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.154+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6150061de0 con 0x7f61400a2e20 2024-06-06T07:48:07.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.163+0000 7f614dffb640 1 -- 172.21.15.87:0/1048356324 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339521 (secure 0 0 0) 0x7f6150061de0 con 0x7f61400a2e20 2024-06-06T07:48:07.167 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:07.169 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 msgr2=0x7f61400a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 0x7f61400a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61501000d0 tx=0x7f613c005f90 comp rx=0 tx=0).stop 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 msgr2=0x7f6150198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6144009230 tx=0x7f6144004270 comp rx=0 tx=0).stop 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 shutdown_connections 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61400a2e20 0x7f61400a52e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6150106880 0x7f615019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:07.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6150101230 0x7f6150198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:07.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 --2- 172.21.15.87:0/1048356324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6150100870 0x7f6150198230 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:07.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 >> 172.21.15.87:0/1048356324 conn(0x7f6150076c20 msgr2=0x7f61500fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:07.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 shutdown_connections 2024-06-06T07:48:07.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:07.167+0000 7f6156b6a640 1 -- 172.21.15.87:0/1048356324 wait complete. 2024-06-06T07:48:07.178 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T07:48:06.408144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189568,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1326,"num_read_kb":821,"num_write":486,"num_write_kb":4389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391320,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550504,"statfs":{"total":1151940427776,"available":1151539716096,"internally_reserved":0,"allocated":17588224,"data_stored":10493547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.993037"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.167728+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:46:46.167728+0000","last_peered":"2024-06-06T07:46:46.167728+0000","last_clean":"2024-06-06T07:46:46.167728+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:46:46.167728+0000","last_undegraded":"2024-06-06T07:46:46.167728+0000","last_fullsized":"2024-06-06T07:46:46.167728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469779+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469779+0000","last_peered":"2024-06-06T07:48:01.469779+0000","last_clean":"2024-06-06T07:48:01.469779+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469779+0000","last_undegraded":"2024-06-06T07:48:01.469779+0000","last_fullsized":"2024-06-06T07:48:01.469779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469577+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469577+0000","last_peered":"2024-06-06T07:48:01.469577+0000","last_clean":"2024-06-06T07:48:01.469577+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469577+0000","last_undegraded":"2024-06-06T07:48:01.469577+0000","last_fullsized":"2024-06-06T07:48:01.469577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.168125+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:46:46.168125+0000","last_peered":"2024-06-06T07:46:46.168125+0000","last_clean":"2024-06-06T07:46:46.168125+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:46:46.168125+0000","last_undegraded":"2024-06-06T07:46:46.168125+0000","last_fullsized":"2024-06-06T07:46:46.168125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489224+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:00.489224+0000","last_peered":"2024-06-06T07:48:00.489224+0000","last_clean":"2024-06-06T07:48:00.489224+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:00.489224+0000","last_undegraded":"2024-06-06T07:48:00.489224+0000","last_fullsized":"2024-06-06T07:48:00.489224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469788+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469788+0000","last_peered":"2024-06-06T07:48:01.469788+0000","last_clean":"2024-06-06T07:48:01.469788+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469788+0000","last_undegraded":"2024-06-06T07:48:01.469788+0000","last_fullsized":"2024-06-06T07:48:01.469788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.503221+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:00.503221+0000","last_peered":"2024-06-06T07:48:00.503221+0000","last_clean":"2024-06-06T07:48:00.503221+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:00.503221+0000","last_undegraded":"2024-06-06T07:48:00.503221+0000","last_fullsized":"2024-06-06T07:48:00.503221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.228208+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:45:48.228208+0000","last_peered":"2024-06-06T07:45:48.228208+0000","last_clean":"2024-06-06T07:45:48.228208+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:45:48.228208+0000","last_undegraded":"2024-06-06T07:45:48.228208+0000","last_fullsized":"2024-06-06T07:45:48.228208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469713+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469713+0000","last_peered":"2024-06-06T07:48:01.469713+0000","last_clean":"2024-06-06T07:48:01.469713+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469713+0000","last_undegraded":"2024-06-06T07:48:01.469713+0000","last_fullsized":"2024-06-06T07:48:01.469713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.502853+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:00.502853+0000","last_peered":"2024-06-06T07:48:00.502853+0000","last_clean":"2024-06-06T07:48:00.502853+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:00.502853+0000","last_undegraded":"2024-06-06T07:48:00.502853+0000","last_fullsized":"2024-06-06T07:48:00.502853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":452,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.637707+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:00.637707+0000","last_peered":"2024-06-06T07:48:00.637707+0000","last_clean":"2024-06-06T07:48:00.637707+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:00.637707+0000","last_undegraded":"2024-06-06T07:48:00.637707+0000","last_fullsized":"2024-06-06T07:48:00.637707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":365,"num_read_kb":244,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501562+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501562+0000","last_peered":"2024-06-06T07:48:00.501562+0000","last_clean":"2024-06-06T07:48:00.501562+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501562+0000","last_undegraded":"2024-06-06T07:48:00.501562+0000","last_fullsized":"2024-06-06T07:48:00.501562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227501+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:45:48.227501+0000","last_peered":"2024-06-06T07:45:48.227501+0000","last_clean":"2024-06-06T07:45:48.227501+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:45:48.227501+0000","last_undegraded":"2024-06-06T07:45:48.227501+0000","last_fullsized":"2024-06-06T07:45:48.227501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'6","reported_seq":234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554773+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:00.554773+0000","last_peered":"2024-06-06T07:48:00.554773+0000","last_clean":"2024-06-06T07:48:00.554773+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:00.554773+0000","last_undegraded":"2024-06-06T07:48:00.554773+0000","last_fullsized":"2024-06-06T07:48:00.554773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.248883+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:45:48.248883+0000","last_peered":"2024-06-06T07:45:48.248883+0000","last_clean":"2024-06-06T07:45:48.248883+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:45:48.248883+0000","last_undegraded":"2024-06-06T07:45:48.248883+0000","last_fullsized":"2024-06-06T07:45:48.248883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.252519+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:45:48.252519+0000","last_peered":"2024-06-06T07:45:48.252519+0000","last_clean":"2024-06-06T07:45:48.252519+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:45:48.252519+0000","last_undegraded":"2024-06-06T07:45:48.252519+0000","last_fullsized":"2024-06-06T07:45:48.252519+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171950+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:46:46.171950+0000","last_peered":"2024-06-06T07:46:46.171950+0000","last_clean":"2024-06-06T07:46:46.171950+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:46:46.171950+0000","last_undegraded":"2024-06-06T07:46:46.171950+0000","last_fullsized":"2024-06-06T07:46:46.171950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501696+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501696+0000","last_peered":"2024-06-06T07:48:00.501696+0000","last_clean":"2024-06-06T07:48:00.501696+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501696+0000","last_undegraded":"2024-06-06T07:48:00.501696+0000","last_fullsized":"2024-06-06T07:48:00.501696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395576+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395576+0000","last_peered":"2024-06-06T07:48:00.395576+0000","last_clean":"2024-06-06T07:48:00.395576+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395576+0000","last_undegraded":"2024-06-06T07:48:00.395576+0000","last_fullsized":"2024-06-06T07:48:00.395576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501494+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501494+0000","last_peered":"2024-06-06T07:48:00.501494+0000","last_clean":"2024-06-06T07:48:00.501494+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501494+0000","last_undegraded":"2024-06-06T07:48:00.501494+0000","last_fullsized":"2024-06-06T07:48:00.501494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498708+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498708+0000","last_peered":"2024-06-06T07:48:00.498708+0000","last_clean":"2024-06-06T07:48:00.498708+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498708+0000","last_undegraded":"2024-06-06T07:48:00.498708+0000","last_fullsized":"2024-06-06T07:48:00.498708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:05.561191+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:05.561191+0000","last_peered":"2024-06-06T07:48:05.561191+0000","last_clean":"2024-06-06T07:48:05.561191+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:05.561191+0000","last_undegraded":"2024-06-06T07:48:05.561191+0000","last_fullsized":"2024-06-06T07:48:05.561191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491258+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491258+0000","last_peered":"2024-06-06T07:48:00.491258+0000","last_clean":"2024-06-06T07:48:00.491258+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491258+0000","last_undegraded":"2024-06-06T07:48:00.491258+0000","last_fullsized":"2024-06-06T07:48:00.491258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554746+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554746+0000","last_peered":"2024-06-06T07:48:00.554746+0000","last_clean":"2024-06-06T07:48:00.554746+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554746+0000","last_undegraded":"2024-06-06T07:48:00.554746+0000","last_fullsized":"2024-06-06T07:48:00.554746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":382,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395397+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:00.395397+0000","last_peered":"2024-06-06T07:48:00.395397+0000","last_clean":"2024-06-06T07:48:00.395397+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:00.395397+0000","last_undegraded":"2024-06-06T07:48:00.395397+0000","last_fullsized":"2024-06-06T07:48:00.395397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:45:48.227366+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:45:48.227366+0000","last_peered":"2024-06-06T07:45:48.227366+0000","last_clean":"2024-06-06T07:45:48.227366+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:45:48.227366+0000","last_undegraded":"2024-06-06T07:45:48.227366+0000","last_fullsized":"2024-06-06T07:45:48.227366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491349+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491349+0000","last_peered":"2024-06-06T07:48:00.491349+0000","last_clean":"2024-06-06T07:48:00.491349+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491349+0000","last_undegraded":"2024-06-06T07:48:00.491349+0000","last_fullsized":"2024-06-06T07:48:00.491349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393936+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393936+0000","last_peered":"2024-06-06T07:48:00.393936+0000","last_clean":"2024-06-06T07:48:00.393936+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393936+0000","last_undegraded":"2024-06-06T07:48:00.393936+0000","last_fullsized":"2024-06-06T07:48:00.393936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490908+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.490908+0000","last_peered":"2024-06-06T07:48:00.490908+0000","last_clean":"2024-06-06T07:48:00.490908+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.490908+0000","last_undegraded":"2024-06-06T07:48:00.490908+0000","last_fullsized":"2024-06-06T07:48:00.490908+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.369652+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:01.369652+0000","last_peered":"2024-06-06T07:48:01.369652+0000","last_clean":"2024-06-06T07:48:01.369652+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:01.369652+0000","last_undegraded":"2024-06-06T07:48:01.369652+0000","last_fullsized":"2024-06-06T07:48:01.369652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491172+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491172+0000","last_peered":"2024-06-06T07:48:00.491172+0000","last_clean":"2024-06-06T07:48:00.491172+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491172+0000","last_undegraded":"2024-06-06T07:48:00.491172+0000","last_fullsized":"2024-06-06T07:48:00.491172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555078+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555078+0000","last_peered":"2024-06-06T07:48:00.555078+0000","last_clean":"2024-06-06T07:48:00.555078+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555078+0000","last_undegraded":"2024-06-06T07:48:00.555078+0000","last_fullsized":"2024-06-06T07:48:00.555078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469845+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469845+0000","last_peered":"2024-06-06T07:48:01.469845+0000","last_clean":"2024-06-06T07:48:01.469845+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469845+0000","last_undegraded":"2024-06-06T07:48:01.469845+0000","last_fullsized":"2024-06-06T07:48:01.469845+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581656+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581656+0000","last_peered":"2024-06-06T07:48:00.581656+0000","last_clean":"2024-06-06T07:48:00.581656+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581656+0000","last_undegraded":"2024-06-06T07:48:00.581656+0000","last_fullsized":"2024-06-06T07:48:00.581656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490919+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490919+0000","last_peered":"2024-06-06T07:48:00.490919+0000","last_clean":"2024-06-06T07:48:00.490919+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490919+0000","last_undegraded":"2024-06-06T07:48:00.490919+0000","last_fullsized":"2024-06-06T07:48:00.490919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491031+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491031+0000","last_peered":"2024-06-06T07:48:00.491031+0000","last_clean":"2024-06-06T07:48:00.491031+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491031+0000","last_undegraded":"2024-06-06T07:48:00.491031+0000","last_fullsized":"2024-06-06T07:48:00.491031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.171863+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:46:46.171863+0000","last_peered":"2024-06-06T07:46:46.171863+0000","last_clean":"2024-06-06T07:46:46.171863+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:46:46.171863+0000","last_undegraded":"2024-06-06T07:46:46.171863+0000","last_fullsized":"2024-06-06T07:46:46.171863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392587+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:00.392587+0000","last_peered":"2024-06-06T07:48:00.392587+0000","last_clean":"2024-06-06T07:48:00.392587+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:00.392587+0000","last_undegraded":"2024-06-06T07:48:00.392587+0000","last_fullsized":"2024-06-06T07:48:00.392587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581868+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581868+0000","last_peered":"2024-06-06T07:48:00.581868+0000","last_clean":"2024-06-06T07:48:00.581868+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581868+0000","last_undegraded":"2024-06-06T07:48:00.581868+0000","last_fullsized":"2024-06-06T07:48:00.581868+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.597759+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:00.597759+0000","last_peered":"2024-06-06T07:48:00.597759+0000","last_clean":"2024-06-06T07:48:00.597759+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:00.597759+0000","last_undegraded":"2024-06-06T07:48:00.597759+0000","last_fullsized":"2024-06-06T07:48:00.597759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.172843+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:46:46.172843+0000","last_peered":"2024-06-06T07:46:46.172843+0000","last_clean":"2024-06-06T07:46:46.172843+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:46:46.172843+0000","last_undegraded":"2024-06-06T07:46:46.172843+0000","last_fullsized":"2024-06-06T07:46:46.172843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392753+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392753+0000","last_peered":"2024-06-06T07:48:00.392753+0000","last_clean":"2024-06-06T07:48:00.392753+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392753+0000","last_undegraded":"2024-06-06T07:48:00.392753+0000","last_fullsized":"2024-06-06T07:48:00.392753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392493+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392493+0000","last_peered":"2024-06-06T07:48:00.392493+0000","last_clean":"2024-06-06T07:48:00.392493+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392493+0000","last_undegraded":"2024-06-06T07:48:00.392493+0000","last_fullsized":"2024-06-06T07:48:00.392493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469791+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469791+0000","last_peered":"2024-06-06T07:48:01.469791+0000","last_clean":"2024-06-06T07:48:01.469791+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469791+0000","last_undegraded":"2024-06-06T07:48:01.469791+0000","last_fullsized":"2024-06-06T07:48:01.469791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393766+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393766+0000","last_peered":"2024-06-06T07:48:00.393766+0000","last_clean":"2024-06-06T07:48:00.393766+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393766+0000","last_undegraded":"2024-06-06T07:48:00.393766+0000","last_fullsized":"2024-06-06T07:48:00.393766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.170081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:46:46.170081+0000","last_peered":"2024-06-06T07:46:46.170081+0000","last_clean":"2024-06-06T07:46:46.170081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:46:46.170081+0000","last_undegraded":"2024-06-06T07:46:46.170081+0000","last_fullsized":"2024-06-06T07:46:46.170081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393093+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393093+0000","last_peered":"2024-06-06T07:48:00.393093+0000","last_clean":"2024-06-06T07:48:00.393093+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393093+0000","last_undegraded":"2024-06-06T07:48:00.393093+0000","last_fullsized":"2024-06-06T07:48:00.393093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491388+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491388+0000","last_peered":"2024-06-06T07:48:00.491388+0000","last_clean":"2024-06-06T07:48:00.491388+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491388+0000","last_undegraded":"2024-06-06T07:48:00.491388+0000","last_fullsized":"2024-06-06T07:48:00.491388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T07:46:46.169888+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:46:46.169888+0000","last_peered":"2024-06-06T07:46:46.169888+0000","last_clean":"2024-06-06T07:46:46.169888+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:46:46.169888+0000","last_undegraded":"2024-06-06T07:46:46.169888+0000","last_fullsized":"2024-06-06T07:46:46.169888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598966,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1190,"num_read_kb":702,"num_write":303,"num_write_kb":1449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1796898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383744,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":570184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645381,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28960,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716192,"statfs":{"total":95995035648,"available":95965380608,"internally_reserved":0,"allocated":1990656,"data_stored":1372586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1177437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1228800,"data_stored":644687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1245184,"data_stored":644624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1691648,"data_stored":1100425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":580904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757341,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37200,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707952,"statfs":{"total":95995035648,"available":95956942848,"internally_reserved":0,"allocated":1908736,"data_stored":1303786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:07.183 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:07.455 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:07.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:07 smithi087 ceph-mon[32276]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-06-06T07:48:07.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:07 smithi087 ceph-mon[32276]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:07.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:07 smithi087 ceph-mon[32276]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:07.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:07 smithi087 ceph-mon[32276]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:07 smithi181 ceph-mon[36790]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-06-06T07:48:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:07 smithi181 ceph-mon[36790]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:07 smithi181 ceph-mon[36790]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:07 smithi181 ceph-mon[36790]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:07 smithi195 ceph-mon[37040]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-06-06T07:48:08.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:07 smithi195 ceph-mon[37040]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:07 smithi195 ceph-mon[37040]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:08.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:07 smithi195 ceph-mon[37040]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:09.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 -- 172.21.15.87:0/2561386355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d4101d30 msgr2=0x7f27d410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2561386355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d4101d30 0x7f27d410e210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b0a0 tx=0x7f27cc00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 -- 172.21.15.87:0/2561386355 shutdown_connections 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2561386355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d4101d30 0x7f27d410e210 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2561386355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d41073d0 0x7f27d41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2561386355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101390 0x7f27d41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 -- 172.21.15.87:0/2561386355 >> 172.21.15.87:0/2561386355 conn(0x7f27d40782c0 msgr2=0x7f27d40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 -- 172.21.15.87:0/2561386355 shutdown_connections 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.309+0000 7f27dc67d700 1 -- 172.21.15.87:0/2561386355 wait complete. 2024-06-06T07:48:09.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 Processor -- start 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 -- start start 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 0x7f27d419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 0x7f27d419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27d4111650 con 0x7f27d41073d0 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27d41114d0 con 0x7f27d4101d30 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dc67d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27d41117d0 con 0x7f27d4101390 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dac1a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dac1a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49268/0 (socket says 172.21.15.87:49268) 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27dac1a700 1 -- 172.21.15.87:0/2806905433 learned_addr learned my addr 172.21.15.87:0/2806905433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27da419700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 0x7f27d419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:09.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.310+0000 7f27d9c18700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 0x7f27d419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 -- 172.21.15.87:0/2806905433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 msgr2=0x7f27d419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 0x7f27d419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 -- 172.21.15.87:0/2806905433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 msgr2=0x7f27d419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 0x7f27d419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 -- 172.21.15.87:0/2806905433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d4199130 con 0x7f27d41073d0 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27da419700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 0x7f27d419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27d9c18700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 0x7f27d419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dac1a700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b780 tx=0x7f27cc004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc021070 con 0x7f27d41073d0 2024-06-06T07:48:09.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc007c80 con 0x7f27d41073d0 2024-06-06T07:48:09.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d41993c0 con 0x7f27d41073d0 2024-06-06T07:48:09.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc005400 con 0x7f27d41073d0 2024-06-06T07:48:09.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.311+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d4102f70 con 0x7f27d41073d0 2024-06-06T07:48:09.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.312+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d4069180 con 0x7f27d41073d0 2024-06-06T07:48:09.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.313+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f27cc024070 con 0x7f27d41073d0 2024-06-06T07:48:09.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.314+0000 7f27cb7fe700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 0x7f27c00a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:09.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.314+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f27cc00f020 con 0x7f27d41073d0 2024-06-06T07:48:09.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.314+0000 7f27da419700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 0x7f27c00a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:09.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.314+0000 7f27da419700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 0x7f27c00a53f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27d4199b10 tx=0x7f27d0009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:09.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.318+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27cc02b020 con 0x7f27d41073d0 2024-06-06T07:48:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.567+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d40630a0 con 0x7f27d41073d0 2024-06-06T07:48:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.568+0000 7f27cb7fe700 1 -- 172.21.15.87:0/2806905433 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f27cc09d720 con 0x7f27d41073d0 2024-06-06T07:48:09.568 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:09.568 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:09.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 msgr2=0x7f27c00a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 0x7f27c00a53f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27d4199b10 tx=0x7f27d0009260 comp rx=0 tx=0).stop 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 msgr2=0x7f27d41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b780 tx=0x7f27cc004db0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 shutdown_connections 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27c00a2f30 0x7f27c00a53f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27d41073d0 0x7f27d41989d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27d4101390 0x7f27d419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 --2- 172.21.15.87:0/2806905433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d4101d30 0x7f27d419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:09.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 >> 172.21.15.87:0/2806905433 conn(0x7f27d40782c0 msgr2=0x7f27d4100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:09.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 shutdown_connections 2024-06-06T07:48:09.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:09.570+0000 7f27dc67d700 1 -- 172.21.15.87:0/2806905433 wait complete. 2024-06-06T07:48:09.573 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:10.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:09 smithi195 ceph-mon[37040]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-06-06T07:48:10.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:09 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2806905433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:10.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:09 smithi181 ceph-mon[36790]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-06-06T07:48:10.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:09 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2806905433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:09 smithi087 ceph-mon[32276]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-06-06T07:48:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:09 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2806905433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:10.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:48:10.531 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:12.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:11 smithi181 ceph-mon[36790]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T07:48:12.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:11 smithi087 ceph-mon[32276]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T07:48:12.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:11 smithi195 ceph-mon[37040]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T07:48:12.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 -- 172.21.15.87:0/1791228849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 msgr2=0x7f308010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:12.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 --2- 172.21.15.87:0/1791228849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308010e230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f307800b0a0 tx=0x7f307800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 -- 172.21.15.87:0/1791228849 shutdown_connections 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 --2- 172.21.15.87:0/1791228849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 0x7f30801077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 --2- 172.21.15.87:0/1791228849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308010e230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 --2- 172.21.15.87:0/1791228849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 0x7f3080101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 -- 172.21.15.87:0/1791228849 >> 172.21.15.87:0/1791228849 conn(0x7f3080078300 msgr2=0x7f3080078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 -- 172.21.15.87:0/1791228849 shutdown_connections 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.328+0000 7f30882ba700 1 -- 172.21.15.87:0/1791228849 wait complete. 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 Processor -- start 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 -- start start 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 0x7f308019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:12.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 0x7f3080198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3080111660 con 0x7f3080101d40 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30801114e0 con 0x7f30801073f0 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.329+0000 7f30882ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30801117e0 con 0x7f3080101380 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49280/0 (socket says 172.21.15.87:49280) 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 -- 172.21.15.87:0/610843714 learned_addr learned my addr 172.21.15.87:0/610843714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3086857700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 0x7f3080198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3086056700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 0x7f308019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 -- 172.21.15.87:0/610843714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 msgr2=0x7f308019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 0x7f308019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 -- 172.21.15.87:0/610843714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 msgr2=0x7f3080198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:12.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 0x7f3080198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 -- 172.21.15.87:0/610843714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30801990c0 con 0x7f3080101d40 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f3085855700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f307000c9b0 tx=0x7f307000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070007b00 con 0x7f3080101d40 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30801993b0 con 0x7f3080101d40 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3070007ca0 con 0x7f3080101d40 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070018730 con 0x7f3080101d40 2024-06-06T07:48:12.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.330+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30801a5670 con 0x7f3080101d40 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.331+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3080069180 con 0x7f3080101d40 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.333+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f30700188d0 con 0x7f3080101d40 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.334+0000 7f30777fe700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 0x7f306c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.334+0000 7f3086056700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 0x7f306c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.334+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f30700cfd30 con 0x7f3080101d40 2024-06-06T07:48:12.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.334+0000 7f3086056700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 0x7f306c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f307c005fd0 tx=0x7f307c005db0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:12.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.337+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3070014030 con 0x7f3080101d40 2024-06-06T07:48:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.534+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3080199e90 con 0x7f3080101d40 2024-06-06T07:48:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.535+0000 7f30777fe700 1 -- 172.21.15.87:0/610843714 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25294 (secure 0 0 0) 0x7f3080199e90 con 0x7f3080101d40 2024-06-06T07:48:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:12.536 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:48:00.385938+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/3344462507":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/207935511":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/4065374776":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/2153987725":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 msgr2=0x7f306c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 0x7f306c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f307c005fd0 tx=0x7f307c005db0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 msgr2=0x7f308019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f307000c9b0 tx=0x7f307000ccc0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 shutdown_connections 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30801073f0 0x7f3080198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f306c0a2eb0 0x7f306c0a5370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3080101380 0x7f308019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 --2- 172.21.15.87:0/610843714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3080101d40 0x7f308019ece0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 >> 172.21.15.87:0/610843714 conn(0x7f3080078300 msgr2=0x7f308006ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 shutdown_connections 2024-06-06T07:48:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:12.538+0000 7f30882ba700 1 -- 172.21.15.87:0/610843714 wait complete. 2024-06-06T07:48:13.059 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:12 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/610843714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:13.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/610843714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:13.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:12 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/610843714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:13.361 DEBUG:tasks.fwd_scrub:Yielding 2024-06-06T07:48:13.362 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T07:48:13.371 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:48:13.371 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:48:13.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-06T07:48:13.372 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-06T07:48:13.372 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:48:13.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 -- 172.21.15.87:0/2337292522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 msgr2=0x7fc4d010d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/2337292522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d010d750 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009cd0 tx=0x7fc4c802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 -- 172.21.15.87:0/2337292522 shutdown_connections 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/2337292522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d010d750 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/2337292522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d01008d0 0x7fc4d0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/2337292522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d0106890 0x7fc4d0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 -- 172.21.15.87:0/2337292522 >> 172.21.15.87:0/2337292522 conn(0x7fc4d0077a20 msgr2=0x7fc4d0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:13.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 -- 172.21.15.87:0/2337292522 shutdown_connections 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.480+0000 7fc4d80ae640 1 -- 172.21.15.87:0/2337292522 wait complete. 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 Processor -- start 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 -- start start 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d01008d0 0x7fc4d0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d0106890 0x7fc4d019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0110d10 con 0x7fc4d0101290 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0110b90 con 0x7fc4d0106890 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d80ae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0110e90 con 0x7fc4d01008d0 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49288/0 (socket says 172.21.15.87:49288) 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 -- 172.21.15.87:0/3518955066 learned_addr learned my addr 172.21.15.87:0/3518955066 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:13.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5e4a640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d01008d0 0x7fc4d0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d664b640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d0106890 0x7fc4d019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 -- 172.21.15.87:0/3518955066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d01008d0 msgr2=0x7fc4d0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d01008d0 0x7fc4d0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 -- 172.21.15.87:0/3518955066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d0106890 msgr2=0x7fc4d019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d0106890 0x7fc4d019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.481+0000 7fc4d5649640 1 -- 172.21.15.87:0/3518955066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d019d1f0 con 0x7fc4d0101290 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4d5649640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b4a0 tx=0x7fc4c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0014070 con 0x7fc4d0101290 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0010af0 con 0x7fc4d0101290 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d019d4e0 con 0x7fc4d0101290 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0017d10 con 0x7fc4d0101290 2024-06-06T07:48:13.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.482+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d01a4d70 con 0x7fc4d0101290 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.483+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d004f760 con 0x7fc4d0101290 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.484+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc4c0017070 con 0x7fc4d0101290 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.485+0000 7fc4c6ffd640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 0x7fc4bc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.485+0000 7fc4d5e4a640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 0x7fc4bc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.485+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc4c00cf1d0 con 0x7fc4d0101290 2024-06-06T07:48:13.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.485+0000 7fc4d5e4a640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 0x7fc4bc0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005850 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:13.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.492+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4c009b8e0 con 0x7fc4d0101290 2024-06-06T07:48:13.625 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:13.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.686+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fc4d0199550 con 0x7fc4d0101290 2024-06-06T07:48:13.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.690+0000 7fc4c6ffd640 1 -- 172.21.15.87:0/3518955066 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fc4d0199550 con 0x7fc4d0101290 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 msgr2=0x7fc4bc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 0x7fc4bc0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005850 comp rx=0 tx=0).stop 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 msgr2=0x7fc4d0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b4a0 tx=0x7fc4c000b960 comp rx=0 tx=0).stop 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 shutdown_connections 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4bc0a2ec0 0x7fc4bc0a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d0106890 0x7fc4d019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d0101290 0x7fc4d0198740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 --2- 172.21.15.87:0/3518955066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4d01008d0 0x7fc4d0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 >> 172.21.15.87:0/3518955066 conn(0x7fc4d0077a20 msgr2=0x7fc4d0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:13.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 shutdown_connections 2024-06-06T07:48:13.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.693+0000 7fc4d80ae640 1 -- 172.21.15.87:0/3518955066 wait complete. 2024-06-06T07:48:13.705 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-06T07:48:13.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 -- 172.21.15.87:0/1015642021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 msgr2=0x7fcb5010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/1015642021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb5010e860 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009cd0 tx=0x7fcb4802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 -- 172.21.15.87:0/1015642021 shutdown_connections 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/1015642021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb5010e860 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/1015642021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 0x7fcb5006a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/1015642021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 0x7fcb50107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 -- 172.21.15.87:0/1015642021 >> 172.21.15.87:0/1015642021 conn(0x7fcb50076be0 msgr2=0x7fcb50076ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 -- 172.21.15.87:0/1015642021 shutdown_connections 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.811+0000 7fcb55cbd640 1 -- 172.21.15.87:0/1015642021 wait complete. 2024-06-06T07:48:13.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 Processor -- start 2024-06-06T07:48:13.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 -- start start 2024-06-06T07:48:13.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 0x7fcb501981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 0x7fcb5019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111df0 con 0x7fcb5006acd0 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111c70 con 0x7fcb5006a310 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb55cbd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111f70 con 0x7fcb50107a50 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49316/0 (socket says 172.21.15.87:49316) 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 -- 172.21.15.87:0/278272329 learned_addr learned my addr 172.21.15.87:0/278272329 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4f7fe640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 0x7fcb501981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4ffff640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 0x7fcb5019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 -- 172.21.15.87:0/278272329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 msgr2=0x7fcb5019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 0x7fcb5019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 -- 172.21.15.87:0/278272329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 msgr2=0x7fcb501981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 0x7fcb501981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:13.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.812+0000 7fcb4effd640 1 -- 172.21.15.87:0/278272329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5019d170 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb4effd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb44009c30 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5019d460 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f6b0 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.813+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501a4cf0 con 0x7fcb5006acd0 2024-06-06T07:48:13.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.814+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5004f760 con 0x7fcb5006acd0 2024-06-06T07:48:13.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.815+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fcb440109d0 con 0x7fcb5006acd0 2024-06-06T07:48:13.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.815+0000 7fcb4cff9640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 0x7fcb380a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:13.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.816+0000 7fcb4f7fe640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 0x7fcb380a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:13.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.816+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fcb440cf6d0 con 0x7fcb5006acd0 2024-06-06T07:48:13.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.816+0000 7fcb4f7fe640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 0x7fcb380a5380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb400058c0 tx=0x7fcb4000a250 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:13.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:13.820+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcb44009de0 con 0x7fcb5006acd0 2024-06-06T07:48:14.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.005+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fcb501a4f90 con 0x7fcb5006acd0 2024-06-06T07:48:14.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.009+0000 7fcb4cff9640 1 -- 172.21.15.87:0/278272329 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fcb440bb020 con 0x7fcb5006acd0 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 msgr2=0x7fcb380a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 0x7fcb380a5380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb400058c0 tx=0x7fcb4000a250 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 msgr2=0x7fcb50198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 shutdown_connections 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb380a2ec0 0x7fcb380a5380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb50107a50 0x7fcb5019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb5006acd0 0x7fcb50198710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 --2- 172.21.15.87:0/278272329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb5006a310 0x7fcb501981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.011+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 >> 172.21.15.87:0/278272329 conn(0x7fcb50076be0 msgr2=0x7fcb500fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.012+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 shutdown_connections 2024-06-06T07:48:14.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.012+0000 7fcb55cbd640 1 -- 172.21.15.87:0/278272329 wait complete. 2024-06-06T07:48:14.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T07:48:14.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3518955066' entity='client.admin' 2024-06-06T07:48:14.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:14.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:14.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:14.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:13 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:14.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-06T07:48:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T07:48:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3518955066' entity='client.admin' 2024-06-06T07:48:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:14.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:14.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:13 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:14.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 -- 172.21.15.87:0/2081348857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481079d0 msgr2=0x7fec48107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/2081348857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481079d0 0x7fec48107db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec38009930 tx=0x7fec3802a420 comp rx=0 tx=0).stop 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 -- 172.21.15.87:0/2081348857 shutdown_connections 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/2081348857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec4810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/2081348857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481019c0 0x7fec48101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/2081348857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481079d0 0x7fec48107db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 -- 172.21.15.87:0/2081348857 >> 172.21.15.87:0/2081348857 conn(0x7fec480fd450 msgr2=0x7fec480ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 -- 172.21.15.87:0/2081348857 shutdown_connections 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.131+0000 7fec4d0f7640 1 -- 172.21.15.87:0/2081348857 wait complete. 2024-06-06T07:48:14.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 Processor -- start 2024-06-06T07:48:14.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 -- start start 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec48198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481079d0 0x7fec4819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec48111de0 con 0x7fec481019c0 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fec48111c60 con 0x7fec48102380 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec4d0f7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec48111f60 con 0x7fec481079d0 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49342/0 (socket says 172.21.15.87:49342) 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.132+0000 7fec46d9d640 1 -- 172.21.15.87:0/1940802676 learned_addr learned my addr 172.21.15.87:0/1940802676 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec4659c640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec48198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec4759e640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481079d0 0x7fec4819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 -- 172.21.15.87:0/1940802676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481079d0 msgr2=0x7fec4819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481079d0 0x7fec4819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 -- 172.21.15.87:0/1940802676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 msgr2=0x7fec48198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec48198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 -- 172.21.15.87:0/1940802676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4819d210 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec4659c640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec48198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec46d9d640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec38009a50 tx=0x7fec38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3803f030 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec38003ce0 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4819d4a0 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38035070 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.133+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481a4d90 con 0x7fec481019c0 2024-06-06T07:48:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.134+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4804f760 con 0x7fec481019c0 2024-06-06T07:48:14.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.135+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fec38042030 con 0x7fec481019c0 2024-06-06T07:48:14.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.136+0000 7fec2ffff640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 0x7fec300a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.136+0000 7fec4659c640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 0x7fec300a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.136+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fec380f0030 con 0x7fec481019c0 2024-06-06T07:48:14.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.136+0000 7fec4659c640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 0x7fec300a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c005ea0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.140+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec380bc700 con 0x7fec481019c0 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3518955066' entity='client.admin' 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:13 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:14.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.319+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fec481a4f60 con 0x7fec481019c0 2024-06-06T07:48:14.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.323+0000 7fec2ffff640 1 -- 172.21.15.87:0/1940802676 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fec380bc000 con 0x7fec481019c0 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 msgr2=0x7fec300a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 0x7fec300a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c005ea0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 msgr2=0x7fec48198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec38009a50 tx=0x7fec38004270 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 shutdown_connections 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fec300a2ec0 0x7fec300a5380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec481079d0 0x7fec4819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec48102380 0x7fec48198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.325+0000 7fec4d0f7640 1 --2- 172.21.15.87:0/1940802676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec481019c0 0x7fec48198220 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.326+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 >> 172.21.15.87:0/1940802676 conn(0x7fec480fd450 msgr2=0x7fec480fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.326+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 shutdown_connections 2024-06-06T07:48:14.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.326+0000 7fec4d0f7640 1 -- 172.21.15.87:0/1940802676 wait complete. 2024-06-06T07:48:14.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 -- 172.21.15.87:0/2083866925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 msgr2=0x7f0718100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 --2- 172.21.15.87:0/2083866925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f0718100d50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0708009930 tx=0x7f070802a420 comp rx=0 tx=0).stop 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 -- 172.21.15.87:0/2083866925 shutdown_connections 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 --2- 172.21.15.87:0/2083866925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0718101290 0x7f071810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 --2- 172.21.15.87:0/2083866925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f0718100d50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 --2- 172.21.15.87:0/2083866925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718106890 0x7f0718106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 -- 172.21.15.87:0/2083866925 >> 172.21.15.87:0/2083866925 conn(0x7f0718077a20 msgr2=0x7f0718077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 -- 172.21.15.87:0/2083866925 shutdown_connections 2024-06-06T07:48:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 -- 172.21.15.87:0/2083866925 wait complete. 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.444+0000 7f07206e1640 1 Processor -- start 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 -- start start 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718101290 0x7f0718197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0718106890 0x7f071819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0718110ca0 con 0x7f07181008d0 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0718110b20 con 0x7f0718106890 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f07206e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0718110e20 con 0x7f0718101290 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49366/0 (socket says 172.21.15.87:49366) 2024-06-06T07:48:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 -- 172.21.15.87:0/1642798151 learned_addr learned my addr 172.21.15.87:0/1642798151 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071dc7c640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718101290 0x7f0718197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 -- 172.21.15.87:0/1642798151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718101290 msgr2=0x7f0718197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718101290 0x7f0718197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 -- 172.21.15.87:0/1642798151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0718106890 msgr2=0x7f071819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0718106890 0x7f071819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 -- 172.21.15.87:0/1642798151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071819c1b0 con 0x7f07181008d0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f071e47d640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f071400d770 tx=0x7f071400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.445+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714017070 con 0x7f07181008d0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.446+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714005500 con 0x7f07181008d0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.446+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071819c4a0 con 0x7f07181008d0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.446+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714004e00 con 0x7f07181008d0 2024-06-06T07:48:14.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.446+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181a3d30 con 0x7f07181008d0 2024-06-06T07:48:14.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.447+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071804f760 con 0x7f07181008d0 2024-06-06T07:48:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.448+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f0714004fa0 con 0x7f07181008d0 2024-06-06T07:48:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.448+0000 7f070f7fe640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 0x7f07040a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.448+0000 7f071dc7c640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 0x7f07040a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.448+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f07140cf4e0 con 0x7f07181008d0 2024-06-06T07:48:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.449+0000 7f071dc7c640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 0x7f07040a5400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07080096c0 tx=0x7f0708000bc0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.452+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f071409bae0 con 0x7f07181008d0 2024-06-06T07:48:14.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.655+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0718061de0 con 0x7f07040a2f40 2024-06-06T07:48:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.719+0000 7f070f7fe640 1 -- 172.21.15.87:0/1642798151 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0718061de0 con 0x7f07040a2f40 2024-06-06T07:48:14.719 INFO:teuthology.orchestra.run.smithi087.stdout:Schedule set for path / 2024-06-06T07:48:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 msgr2=0x7f07040a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 0x7f07040a5400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07080096c0 tx=0x7f0708000bc0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 msgr2=0x7f07181971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f071400d770 tx=0x7f071400dc30 comp rx=0 tx=0).stop 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 shutdown_connections 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f07040a2f40 0x7f07040a5400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0718106890 0x7f071819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0718101290 0x7f0718197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 --2- 172.21.15.87:0/1642798151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07181008d0 0x7f07181971c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 >> 172.21.15.87:0/1642798151 conn(0x7f0718077a20 msgr2=0x7f0718103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 shutdown_connections 2024-06-06T07:48:14.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.721+0000 7f07206e1640 1 -- 172.21.15.87:0/1642798151 wait complete. 2024-06-06T07:48:14.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-06T07:48:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 -- 172.21.15.87:0/77738076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841079c0 msgr2=0x7f2584107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 --2- 172.21.15.87:0/77738076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841079c0 0x7f2584107da0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2574009930 tx=0x7f257402a420 comp rx=0 tx=0).stop 2024-06-06T07:48:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 -- 172.21.15.87:0/77738076 shutdown_connections 2024-06-06T07:48:14.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 --2- 172.21.15.87:0/77738076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 0x7f258410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 --2- 172.21.15.87:0/77738076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841019b0 0x7f2584101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 --2- 172.21.15.87:0/77738076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841079c0 0x7f2584107da0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 -- 172.21.15.87:0/77738076 >> 172.21.15.87:0/77738076 conn(0x7f2584078e30 msgr2=0x7f2584079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 -- 172.21.15.87:0/77738076 shutdown_connections 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.837+0000 7f2589c38640 1 -- 172.21.15.87:0/77738076 wait complete. 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 Processor -- start 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 -- start start 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 0x7f2584198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841079c0 0x7f258419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2584111e20 con 0x7f25841019b0 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2584111ca0 con 0x7f2584102370 2024-06-06T07:48:14.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2589c38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2584111fa0 con 0x7f25841079c0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f25837fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f25837fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49378/0 (socket says 172.21.15.87:49378) 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f25837fe640 1 -- 172.21.15.87:0/2592435084 learned_addr learned my addr 172.21.15.87:0/2592435084 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.838+0000 7f2582ffd640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 0x7f2584198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2583fff640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841079c0 0x7f258419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 -- 172.21.15.87:0/2592435084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841079c0 msgr2=0x7f258419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841079c0 0x7f258419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 -- 172.21.15.87:0/2592435084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 msgr2=0x7f2584198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 0x7f2584198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 -- 172.21.15.87:0/2592435084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258419d1d0 con 0x7f25841019b0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f25837fe640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f257402a940 tx=0x7f2574003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257403f070 con 0x7f25841019b0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2574031c90 con 0x7f25841019b0 2024-06-06T07:48:14.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258419d460 con 0x7f25841019b0 2024-06-06T07:48:14.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574035030 con 0x7f25841019b0 2024-06-06T07:48:14.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.839+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a4d50 con 0x7f25841019b0 2024-06-06T07:48:14.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.840+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258404f760 con 0x7f25841019b0 2024-06-06T07:48:14.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.841+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f2574042070 con 0x7f25841019b0 2024-06-06T07:48:14.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.842+0000 7f2580ff9640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 0x7f256c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:14.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.842+0000 7f2582ffd640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 0x7f256c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:14.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.842+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f25740f01d0 con 0x7f25841019b0 2024-06-06T07:48:14.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.843+0000 7f2582ffd640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 0x7f256c0a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2584101130 tx=0x7f257800a2d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:14.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:14.847+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f25740bc840 con 0x7f25841019b0 2024-06-06T07:48:15.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.054+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f25840ff920 con 0x7f256c0a2ec0 2024-06-06T07:48:15.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.116+0000 7f2580ff9640 1 -- 172.21.15.87:0/2592435084 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f25840ff920 con 0x7f256c0a2ec0 2024-06-06T07:48:15.117 INFO:teuthology.orchestra.run.smithi087.stdout:Retention added to path / 2024-06-06T07:48:15.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 msgr2=0x7f256c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 0x7f256c0a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2584101130 tx=0x7f257800a2d0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 msgr2=0x7f25841981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f257402a940 tx=0x7f2574003b90 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 shutdown_connections 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f256c0a2ec0 0x7f256c0a5380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25841079c0 0x7f258419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584102370 0x7f2584198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 --2- 172.21.15.87:0/2592435084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25841019b0 0x7f25841981e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 >> 172.21.15.87:0/2592435084 conn(0x7f2584078e30 msgr2=0x7f25840ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 shutdown_connections 2024-06-06T07:48:15.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.118+0000 7f2589c38640 1 -- 172.21.15.87:0/2592435084 wait complete. 2024-06-06T07:48:15.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T07:48:15.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3815358601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 msgr2=0x7f5cf8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3815358601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf8106c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009930 tx=0x7f5ce802a440 comp rx=0 tx=0).stop 2024-06-06T07:48:15.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3815358601 shutdown_connections 2024-06-06T07:48:15.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3815358601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cf8101290 0x7f5cf810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3815358601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3815358601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf8106c70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3815358601 >> 172.21.15.87:0/3815358601 conn(0x7f5cf8077a20 msgr2=0x7f5cf8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3815358601 shutdown_connections 2024-06-06T07:48:15.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.235+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3815358601 wait complete. 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 Processor -- start 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 -- start start 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf80fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cf8101290 0x7f5cf80fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110d40 con 0x7f5cf8106890 2024-06-06T07:48:15.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110bc0 con 0x7f5cf8101290 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cfca5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110ec0 con 0x7f5cf81008d0 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49400/0 (socket says 172.21.15.87:49400) 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cf6d9d640 1 -- 172.21.15.87:0/3096978518 learned_addr learned my addr 172.21.15.87:0/3096978518 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.236+0000 7f5cf659c640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf80fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 -- 172.21.15.87:0/3096978518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 msgr2=0x7f5cf80fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf80fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 -- 172.21.15.87:0/3096978518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cf8101290 msgr2=0x7f5cf80fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cf8101290 0x7f5cf80fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 -- 172.21.15.87:0/3096978518 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a68d0 con 0x7f5cf8106890 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf659c640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf80fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cf6d9d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000b990 tx=0x7f5cf000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0019070 con 0x7f5cf8106890 2024-06-06T07:48:15.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a6bc0 con 0x7f5cf8106890 2024-06-06T07:48:15.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf000cba0 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0007650 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.237+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a70c0 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.238+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf804f760 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.239+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f5cf001c070 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.240+0000 7f5ce77fe640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 0x7f5ce00a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.240+0000 7f5cf659c640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 0x7f5ce00a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.240+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f5cf00cf000 con 0x7f5cf8106890 2024-06-06T07:48:15.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.240+0000 7f5cf659c640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 0x7f5ce00a53b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8005a20 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.244+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5cf009b710 con 0x7f5cf8106890 2024-06-06T07:48:15.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/278272329' entity='client.admin' 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1940802676' entity='client.admin' 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/278272329' entity='client.admin' 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1940802676' entity='client.admin' 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.308 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.352 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/278272329' entity='client.admin' 2024-06-06T07:48:15.352 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.352 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1940802676' entity='client.admin' 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:48:15.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:48:15.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.450+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5cf8061de0 con 0x7f5ce00a2ef0 2024-06-06T07:48:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.458+0000 7f5ce77fe640 1 -- 172.21.15.87:0/3096978518 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f5cf8061de0 con 0x7f5ce00a2ef0 2024-06-06T07:48:15.458 INFO:teuthology.orchestra.run.smithi087.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T07:48:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.460+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 msgr2=0x7f5ce00a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.460+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 0x7f5ce00a53b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8005a20 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.460+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 msgr2=0x7f5cf80fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.460+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000b990 tx=0x7f5cf000be50 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 shutdown_connections 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ce00a2ef0 0x7f5ce00a53b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf8106890 0x7f5cf80fe2e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cf8101290 0x7f5cf80fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 --2- 172.21.15.87:0/3096978518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf81008d0 0x7f5cf80fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 >> 172.21.15.87:0/3096978518 conn(0x7f5cf8077a20 msgr2=0x7f5cf8069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 shutdown_connections 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.461+0000 7f5cfca5d640 1 -- 172.21.15.87:0/3096978518 wait complete. 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 -- 172.21.15.87:0/370830011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1073f0 msgr2=0x7f725c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 --2- 172.21.15.87:0/370830011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1073f0 0x7f725c1077d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7258009990 tx=0x7f7258009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 -- 172.21.15.87:0/370830011 shutdown_connections 2024-06-06T07:48:15.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 --2- 172.21.15.87:0/370830011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 0x7f725c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 --2- 172.21.15.87:0/370830011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c1073f0 0x7f725c1077d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 --2- 172.21.15.87:0/370830011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c101380 0x7f725c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 -- 172.21.15.87:0/370830011 >> 172.21.15.87:0/370830011 conn(0x7f725c078300 msgr2=0x7f725c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 -- 172.21.15.87:0/370830011 shutdown_connections 2024-06-06T07:48:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.462+0000 7f7263528700 1 -- 172.21.15.87:0/370830011 wait complete. 2024-06-06T07:48:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 Processor -- start 2024-06-06T07:48:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 -- start start 2024-06-06T07:48:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 0x7f725c1038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 0x7f725c103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f725c111690 con 0x7f725c101380 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f725c111510 con 0x7f725c1073f0 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7263528700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f725c111810 con 0x7f725c101d40 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f72612c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f72612c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49422/0 (socket says 172.21.15.87:49422) 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f72612c4700 1 -- 172.21.15.87:0/875337500 learned_addr learned my addr 172.21.15.87:0/875337500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.463+0000 7f7261ac5700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 0x7f725c103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f7260ac3700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 0x7f725c1038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 -- 172.21.15.87:0/875337500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 msgr2=0x7f725c1038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 0x7f725c1038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 -- 172.21.15.87:0/875337500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 msgr2=0x7f725c103de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 0x7f725c103de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 -- 172.21.15.87:0/875337500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c1a7100 con 0x7f725c101380 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f7261ac5700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 0x7f725c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72612c4700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f7258003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72527fc700 1 -- 172.21.15.87:0/875337500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258027070 con 0x7f725c101380 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72527fc700 1 -- 172.21.15.87:0/875337500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258004190 con 0x7f725c101380 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1a7390 con 0x7f725c101380 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f72527fc700 1 -- 172.21.15.87:0/875337500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800f460 con 0x7f725c101380 2024-06-06T07:48:15.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.464+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a77e0 con 0x7f725c101380 2024-06-06T07:48:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.466+0000 7f72527fc700 1 -- 172.21.15.87:0/875337500 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f725802a070 con 0x7f725c101380 2024-06-06T07:48:15.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.467+0000 7f72527fc700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 0x7f72480a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.468+0000 7f7260ac3700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 0x7f72480a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.468+0000 7f72527fc700 1 -- 172.21.15.87:0/875337500 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f72580d0970 con 0x7f725c101380 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.468+0000 7f7260ac3700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 0x7f72480a5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f724c005ea0 tx=0x7f724c005e30 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.468+0000 7f726ddabb80 1 Processor -- start 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.468+0000 7f726ddabb80 1 -- start start 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc85122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55dcc8404fe0 con 0x55dcc8512820 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dcc84638d0 con 0x55dcc8517fe0 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f726ddabb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dcc83cb4e0 con 0x55dcc8518900 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49436/0 (socket says 172.21.15.87:49436) 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 -- 172.21.15.87:0/1850216762 learned_addr learned my addr 172.21.15.87:0/1850216762 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f72612c4700 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7260ac3700 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc85122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 -- 172.21.15.87:0/1850216762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 msgr2=0x55dcc85122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc85122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 -- 172.21.15.87:0/1850216762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 msgr2=0x55dcc85183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85183c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 -- 172.21.15.87:0/1850216762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc83b2950 con 0x55dcc8512820 2024-06-06T07:48:15.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.469+0000 7f7261ac5700 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f72540085a0 tx=0x7f72540088b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8bd91b86883f8b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.470+0000 7f72477fe700 1 -- 172.21.15.87:0/1850216762 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254003c00 con 0x55dcc8512820 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.470+0000 7f72477fe700 1 -- 172.21.15.87:0/1850216762 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7254003da0 con 0x55dcc8512820 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.470+0000 7f72477fe700 1 -- 172.21.15.87:0/1850216762 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254012af0 con 0x55dcc8512820 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- 172.21.15.87:0/1850216762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 msgr2=0x55dcc8512c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f72540085a0 tx=0x7f72540088b0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- 172.21.15.87:0/1850216762 shutdown_connections 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8512c40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 --2- 172.21.15.87:0/1850216762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc85122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- 172.21.15.87:0/1850216762 >> 172.21.15.87:0/1850216762 conn(0x55dcc843f100 msgr2=0x55dcc850d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- 172.21.15.87:0/1850216762 shutdown_connections 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- 172.21.15.87:0/1850216762 wait complete. 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 -- start start 2024-06-06T07:48:15.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.471+0000 7f726ddabb80 1 Processor -- start 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc851e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8517fe0 0x55dcc851b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55dcc8404040 con 0x55dcc8518900 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dcc845f890 con 0x55dcc8512820 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dcc851f2e0 con 0x55dcc8517fe0 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f726ddabb80 10 client.0 _ll_register_callbacks cb 0x55dcc83f9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49438/0 (socket says 172.21.15.87:49438) 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 -- 172.21.15.87:0/103141886 learned_addr learned my addr 172.21.15.87:0/103141886 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7260ac3700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8517fe0 0x55dcc851b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f72612c4700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc851e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 -- 172.21.15.87:0/103141886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8517fe0 msgr2=0x55dcc851b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8517fe0 0x55dcc851b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 -- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 msgr2=0x55dcc851e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc851e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 -- 172.21.15.87:0/103141886 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc84297d0 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.472+0000 7f7261ac5700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7254008570 tx=0x7f72540077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eef21ebaf99e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254012d70 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7254004ad0 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7254003da0 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f725402b050 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f726ddabb80 10 client.15423 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcc8529af0 con 0x55dcc8518900 2024-06-06T07:48:15.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7254003da0 con 0x55dcc8518900 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7254013070 con 0x55dcc8518900 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f726ddabb80 10 client.15423 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.473+0000 7f726ddabb80 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f726ddabb80 20 client.15423 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc852c0d0 0x55dcc852e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f726ddabb80 4 client.15423 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc8470dd0 con 0x55dcc852c0d0 2024-06-06T07:48:15.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f72612c4700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc852c0d0 0x55dcc852e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f7244ff9700 10 client.15423 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:15.481 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T07:48:15.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.474+0000 7f72612c4700 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc852c0d0 0x55dcc852e4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59313855f840e98 server_cookie=abbe6ddc1746130 in_seq=0 out_seq=0 2024-06-06T07:48:15.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f7244ff9700 1 -- 172.21.15.87:0/103141886 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dcc8470dd0 con 0x55dcc852c0d0 2024-06-06T07:48:15.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f7244ff9700 10 client.15423 handle_command_reply: tid=0 2024-06-06T07:48:15.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 client.15423 shutdown 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 msgr2=0x55dcc851be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7254008570 tx=0x7f72540077e0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 shutdown_connections 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc852c0d0 0x55dcc852e4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc851e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8518900 0x55dcc851be90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 --2- 172.21.15.87:0/103141886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8517fe0 0x55dcc851b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f7244ff9700 0 client.15423 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.475+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 >> 172.21.15.87:0/103141886 conn(0x55dcc843f100 msgr2=0x55dcc850c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.476+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 shutdown_connections 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.476+0000 7f726ddabb80 1 -- 172.21.15.87:0/103141886 wait complete. 2024-06-06T07:48:15.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.476+0000 7f726ddabb80 20 client.15423 trim_cache size 0 max 16384 2024-06-06T07:48:15.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 Processor -- start 2024-06-06T07:48:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 -- start start 2024-06-06T07:48:15.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc85122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc851a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1b2f30 con 0x55dcc8517fe0 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1b07c0 con 0x55dcc8512820 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f726ddabb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1ae610 con 0x55dcc8518900 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49440/0 (socket says 172.21.15.87:49440) 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 -- 172.21.15.87:0/3058787396 learned_addr learned my addr 172.21.15.87:0/3058787396 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f72612c4700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc85122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7261ac5700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc851a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 -- 172.21.15.87:0/3058787396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 msgr2=0x55dcc851a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc851a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 -- 172.21.15.87:0/3058787396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 msgr2=0x55dcc85122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc85122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f7260ac3700 1 -- 172.21.15.87:0/3058787396 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc8519ee0 con 0x55dcc8517fe0 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.498+0000 7f72612c4700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc85122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.499+0000 7f7260ac3700 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f724c00eed0 tx=0x7f724c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76b7f3f87331030 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.499+0000 7f7246ffd700 1 -- 172.21.15.87:0/3058787396 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c01ede0 con 0x55dcc8517fe0 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.499+0000 7f7246ffd700 1 -- 172.21.15.87:0/3058787396 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f724c010910 con 0x55dcc8517fe0 2024-06-06T07:48:15.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.499+0000 7f7246ffd700 1 -- 172.21.15.87:0/3058787396 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c01da20 con 0x55dcc8517fe0 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- 172.21.15.87:0/3058787396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 msgr2=0x55dcc8519570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f724c00eed0 tx=0x7f724c00af90 comp rx=0 tx=0).stop 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- 172.21.15.87:0/3058787396 shutdown_connections 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8512820 0x55dcc85122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc851a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 --2- 172.21.15.87:0/3058787396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8517fe0 0x55dcc8519570 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- 172.21.15.87:0/3058787396 >> 172.21.15.87:0/3058787396 conn(0x55dcc843f100 msgr2=0x55dcc851d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- 172.21.15.87:0/3058787396 shutdown_connections 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- 172.21.15.87:0/3058787396 wait complete. 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 -- start start 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.500+0000 7f726ddabb80 1 Processor -- start 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc8521b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aec80 con 0x55dcc8512820 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1b00f0 con 0x55dcc8517fe0 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1af350 con 0x55dcc8518900 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f726ddabb80 10 client.0 _ll_register_callbacks cb 0x55dcc83f9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49452/0 (socket says 172.21.15.87:49452) 2024-06-06T07:48:15.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 -- 172.21.15.87:0/793443770 learned_addr learned my addr 172.21.15.87:0/793443770 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f7261ac5700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc8521b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f7260ac3700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 -- 172.21.15.87:0/793443770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 msgr2=0x55dcc8521b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc8521b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 -- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 msgr2=0x55dcc85215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.501+0000 7f72612c4700 1 -- 172.21.15.87:0/793443770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc8522050 con 0x55dcc8512820 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f72612c4700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f72580cdc80 tx=0x7f7258088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120988ae17b2e073 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258022700 con 0x55dcc8512820 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f72580880b0 con 0x55dcc8512820 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724c010910 con 0x55dcc8512820 2024-06-06T07:48:15.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72580882b0 con 0x55dcc8512820 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 10 client.15429 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcc85977a0 con 0x55dcc8512820 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f724c010910 con 0x55dcc8512820 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f72580cdee0 con 0x55dcc8512820 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 20 client.15429 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc8597f50 0x55dcc859a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 4 client.15429 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.502+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcc846b350 con 0x55dcc8597f50 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.503+0000 7f7260ac3700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc8597f50 0x55dcc859a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.503+0000 7f7260ac3700 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc8597f50 0x55dcc859a340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3d52c91519da0d server_cookie=29322e7616487daa in_seq=0 out_seq=0 2024-06-06T07:48:15.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.503+0000 7f7244ff9700 10 client.15429 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f7244ff9700 1 -- 172.21.15.87:0/793443770 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcc846b350 con 0x55dcc8597f50 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f7244ff9700 10 client.15429 handle_command_reply: tid=0 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 client.15429 shutdown 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 msgr2=0x55dcc8524a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f72580cdc80 tx=0x7f7258088040 comp rx=0 tx=0).stop 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 shutdown_connections 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55dcc8597f50 0x55dcc859a340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dcc8518900 0x55dcc8521b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55dcc8512820 0x55dcc8524a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 --2- 172.21.15.87:0/793443770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcc8517fe0 0x55dcc85215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 >> 172.21.15.87:0/793443770 conn(0x55dcc843f100 msgr2=0x55dcc851da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.504+0000 7f7244ff9700 0 client.15429 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 shutdown_connections 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f726ddabb80 1 -- 172.21.15.87:0/793443770 wait complete. 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f726ddabb80 20 client.15429 trim_cache size 0 max 16384 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "a2a646fc-574c-4818-b9dd-fa464fe93a70", 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-06-06T07:48:15.509 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 msgr2=0x7f72480a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 0x7f72480a5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f724c005ea0 tx=0x7f724c005e30 comp rx=0 tx=0).stop 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 msgr2=0x7f725c103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f7258003730 comp rx=0 tx=0).stop 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 shutdown_connections 2024-06-06T07:48:15.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c1073f0 0x7f725c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72480a2eb0 0x7f72480a5370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725c101380 0x7f725c103360 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 --2- 172.21.15.87:0/875337500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f725c101d40 0x7f725c1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.505+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 >> 172.21.15.87:0/875337500 conn(0x7f725c078300 msgr2=0x7f725c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.506+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 shutdown_connections 2024-06-06T07:48:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.506+0000 7f7263528700 1 -- 172.21.15.87:0/875337500 wait complete. 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 -- 172.21.15.87:0/474476906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b41069d0 msgr2=0x7f38b4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 --2- 172.21.15.87:0/474476906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b41069d0 0x7f38b4106db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002a410 comp rx=0 tx=0).stop 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 -- 172.21.15.87:0/474476906 shutdown_connections 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 --2- 172.21.15.87:0/474476906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 --2- 172.21.15.87:0/474476906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b406a180 0x7f38b406a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 --2- 172.21.15.87:0/474476906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b41069d0 0x7f38b4106db0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 -- 172.21.15.87:0/474476906 >> 172.21.15.87:0/474476906 conn(0x7f38b4077680 msgr2=0x7f38b4077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 -- 172.21.15.87:0/474476906 shutdown_connections 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.594+0000 7f38bbeca640 1 -- 172.21.15.87:0/474476906 wait complete. 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 Processor -- start 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 -- start start 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b41069d0 0x7f38b419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4111d60 con 0x7f38b406a180 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4111be0 con 0x7f38b41069d0 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38bbeca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4111ee0 con 0x7f38b406ab40 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49456/0 (socket says 172.21.15.87:49456) 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 -- 172.21.15.87:0/1833431318 learned_addr learned my addr 172.21.15.87:0/1833431318 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38ba467640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b41069d0 0x7f38b419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9465640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 -- 172.21.15.87:0/1833431318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 msgr2=0x7f38b41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 -- 172.21.15.87:0/1833431318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b41069d0 msgr2=0x7f38b419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b41069d0 0x7f38b419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 -- 172.21.15.87:0/1833431318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b419d170 con 0x7f38b406a180 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9465640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.595+0000 7f38b9c66640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006010 tx=0x7f38b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.596+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b003f070 con 0x7f38b406a180 2024-06-06T07:48:15.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.596+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b419d400 con 0x7f38b406a180 2024-06-06T07:48:15.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.596+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b0031c30 con 0x7f38b406a180 2024-06-06T07:48:15.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.596+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0035030 con 0x7f38b406a180 2024-06-06T07:48:15.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.596+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b41a4cf0 con 0x7f38b406a180 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.597+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b41030d0 con 0x7f38b406a180 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.598+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f38b0042070 con 0x7f38b406a180 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.598+0000 7f38aaffd640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 0x7f38a00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.598+0000 7f38b9465640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 0x7f38a00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.599+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f38b00f0710 con 0x7f38b406a180 2024-06-06T07:48:15.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.599+0000 7f38b9465640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 0x7f38a00a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005d00 tx=0x7f38a4005ed0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:15.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.602+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38b0031de0 con 0x7f38b406a180 2024-06-06T07:48:15.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.806+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f38b40fecc0 con 0x7f38a00a2ec0 2024-06-06T07:48:15.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.812+0000 7f38aaffd640 1 -- 172.21.15.87:0/1833431318 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f38b40fecc0 con 0x7f38a00a2ec0 2024-06-06T07:48:15.812 INFO:teuthology.orchestra.run.smithi087.stdout:/ 1m 6m3h 2024-06-06T07:48:15.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 msgr2=0x7f38a00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 0x7f38a00a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005d00 tx=0x7f38a4005ed0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 msgr2=0x7f38b4198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f38b0006010 tx=0x7f38b0003c10 comp rx=0 tx=0).stop 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 shutdown_connections 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38a00a2ec0 0x7f38a00a5380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38b41069d0 0x7f38b419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b406ab40 0x7f38b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 --2- 172.21.15.87:0/1833431318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38b406a180 0x7f38b4198180 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 >> 172.21.15.87:0/1833431318 conn(0x7f38b4077680 msgr2=0x7f38b40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 shutdown_connections 2024-06-06T07:48:15.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:15.814+0000 7f38bbeca640 1 -- 172.21.15.87:0/1833431318 wait complete. 2024-06-06T07:48:15.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-06T07:48:15.890 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-06T07:48:15.901 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-06T07:48:15.919 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T07:48:15.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-06T07:48:15.933 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-06T07:48:15.947 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T07:48:15.947 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: scrub queued for path: / 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: scrub summary: active paths [/] 2024-06-06T07:48:16.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:16 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T07:48:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: scrub queued for path: / 2024-06-06T07:48:16.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:16.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: scrub summary: active paths [/] 2024-06-06T07:48:16.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:16 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:16.353 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:48:16.418 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: scrub queued for path: / 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: scrub summary: active paths [/] 2024-06-06T07:48:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:16 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:48:17.297 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:17 smithi087 ceph-mon[32276]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:17.297 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:17 smithi087 ceph-mon[32276]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:17 smithi181 ceph-mon[36790]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:17 smithi181 ceph-mon[36790]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:17 smithi195 ceph-mon[37040]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:17 smithi195 ceph-mon[37040]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:48:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:18 smithi181 ceph-mon[36790]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 43 op/s 2024-06-06T07:48:18.310 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:18 smithi087 ceph-mon[32276]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 43 op/s 2024-06-06T07:48:18.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 -- 172.21.15.87:0/3376600252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 msgr2=0x7f1aa806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 --2- 172.21.15.87:0/3376600252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa806a5f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009990 tx=0x7f1aa4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 -- 172.21.15.87:0/3376600252 shutdown_connections 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 --2- 172.21.15.87:0/3376600252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 0x7f1aa810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 --2- 172.21.15.87:0/3376600252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 0x7f1aa81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 --2- 172.21.15.87:0/3376600252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa806a5f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.311+0000 7f1aafb41700 1 -- 172.21.15.87:0/3376600252 >> 172.21.15.87:0/3376600252 conn(0x7f1aa8078340 msgr2=0x7f1aa8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.312+0000 7f1aafb41700 1 -- 172.21.15.87:0/3376600252 shutdown_connections 2024-06-06T07:48:18.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.312+0000 7f1aafb41700 1 -- 172.21.15.87:0/3376600252 wait complete. 2024-06-06T07:48:18.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.312+0000 7f1aafb41700 1 Processor -- start 2024-06-06T07:48:18.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.312+0000 7f1aafb41700 1 -- start start 2024-06-06T07:48:18.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:18.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 0x7f1aa819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:18.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 0x7f1aa8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81127d0 con 0x7f1aa806a170 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8112650 con 0x7f1aa806ab30 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aafb41700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8112950 con 0x7f1aa8108500 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41070/0 (socket says 172.21.15.87:41070) 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 -- 172.21.15.87:0/240015352 learned_addr learned my addr 172.21.15.87:0/240015352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad0dc700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 0x7f1aa819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aae0de700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 0x7f1aa8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 -- 172.21.15.87:0/240015352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 msgr2=0x7f1aa8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 0x7f1aa8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 -- 172.21.15.87:0/240015352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 msgr2=0x7f1aa819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 0x7f1aa819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 -- 172.21.15.87:0/240015352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81991d0 con 0x7f1aa806a170 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.313+0000 7f1aad8dd700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.314+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00cbd0 con 0x7f1aa806a170 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.314+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81994c0 con 0x7f1aa806a170 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.314+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9c00cd70 con 0x7f1aa806a170 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.314+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c010430 con 0x7f1aa806a170 2024-06-06T07:48:18.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.314+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a5780 con 0x7f1aa806a170 2024-06-06T07:48:18.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.315+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa8069180 con 0x7f1aa806a170 2024-06-06T07:48:18.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.316+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f1a9c004750 con 0x7f1aa806a170 2024-06-06T07:48:18.317 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:18.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.317+0000 7f1a9affd700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 0x7f1a940a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:18.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.317+0000 7f1aad0dc700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 0x7f1a940a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:18.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.317+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f1a9c0cfcf0 con 0x7f1aa806a170 2024-06-06T07:48:18.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.317+0000 7f1aad0dc700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 0x7f1a940a53f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8078940 tx=0x7f1aa4005fd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:18.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.321+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a9c09c290 con 0x7f1aa806a170 2024-06-06T07:48:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:18 smithi195 ceph-mon[37040]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 43 op/s 2024-06-06T07:48:18.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.521+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aa80628d0 con 0x7f1a940a2f30 2024-06-06T07:48:18.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.531+0000 7f1a9affd700 1 -- 172.21.15.87:0/240015352 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339560 (secure 0 0 0) 0x7f1aa80628d0 con 0x7f1a940a2f30 2024-06-06T07:48:18.535 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 msgr2=0x7f1a940a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 0x7f1a940a53f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8078940 tx=0x7f1aa4005fd0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 msgr2=0x7f1aa819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ecd0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 shutdown_connections 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1a940a2f30 0x7f1a940a53f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.535+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1aa8108500 0x7f1aa8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.536+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1aa806a170 0x7f1aa819e8b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.536+0000 7f1aafb41700 1 --2- 172.21.15.87:0/240015352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa806ab30 0x7f1aa819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:18.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.536+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 >> 172.21.15.87:0/240015352 conn(0x7f1aa8078340 msgr2=0x7f1aa80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:18.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.536+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 shutdown_connections 2024-06-06T07:48:18.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:18.536+0000 7f1aafb41700 1 -- 172.21.15.87:0/240015352 wait complete. 2024-06-06T07:48:18.539 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:19.338 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-06-06T07:48:18.410946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389890,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1771,"num_read_kb":1316,"num_write":636,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392272,"kb_used_data":17780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124549552,"statfs":{"total":1151940427776,"available":1151538741248,"internally_reserved":0,"allocated":18206720,"data_stored":11105636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200322,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":445,"num_read_kb":495,"num_write":150,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002801"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470157+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:48:01.470157+0000","last_peered":"2024-06-06T07:48:01.470157+0000","last_clean":"2024-06-06T07:48:01.470157+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:48:01.470157+0000","last_undegraded":"2024-06-06T07:48:01.470157+0000","last_fullsized":"2024-06-06T07:48:01.470157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469833+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469833+0000","last_peered":"2024-06-06T07:48:01.469833+0000","last_clean":"2024-06-06T07:48:01.469833+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469833+0000","last_undegraded":"2024-06-06T07:48:01.469833+0000","last_fullsized":"2024-06-06T07:48:01.469833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469631+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469631+0000","last_peered":"2024-06-06T07:48:01.469631+0000","last_clean":"2024-06-06T07:48:01.469631+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469631+0000","last_undegraded":"2024-06-06T07:48:01.469631+0000","last_fullsized":"2024-06-06T07:48:01.469631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470016+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:48:01.470016+0000","last_peered":"2024-06-06T07:48:01.470016+0000","last_clean":"2024-06-06T07:48:01.470016+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:48:01.470016+0000","last_undegraded":"2024-06-06T07:48:01.470016+0000","last_fullsized":"2024-06-06T07:48:01.470016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:06.403648+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:06.403648+0000","last_peered":"2024-06-06T07:48:06.403648+0000","last_clean":"2024-06-06T07:48:06.403648+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:06.403648+0000","last_undegraded":"2024-06-06T07:48:06.403648+0000","last_fullsized":"2024-06-06T07:48:06.403648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469840+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469840+0000","last_peered":"2024-06-06T07:48:01.469840+0000","last_clean":"2024-06-06T07:48:01.469840+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469840+0000","last_undegraded":"2024-06-06T07:48:01.469840+0000","last_fullsized":"2024-06-06T07:48:01.469840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:14.523840+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:14.523840+0000","last_peered":"2024-06-06T07:48:14.523840+0000","last_clean":"2024-06-06T07:48:14.523840+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:14.523840+0000","last_undegraded":"2024-06-06T07:48:14.523840+0000","last_fullsized":"2024-06-06T07:48:14.523840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469934+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:48:01.469934+0000","last_peered":"2024-06-06T07:48:01.469934+0000","last_clean":"2024-06-06T07:48:01.469934+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:48:01.469934+0000","last_undegraded":"2024-06-06T07:48:01.469934+0000","last_fullsized":"2024-06-06T07:48:01.469934+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469765+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469765+0000","last_peered":"2024-06-06T07:48:01.469765+0000","last_clean":"2024-06-06T07:48:01.469765+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469765+0000","last_undegraded":"2024-06-06T07:48:01.469765+0000","last_fullsized":"2024-06-06T07:48:01.469765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":243,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:14.523284+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:14.523284+0000","last_peered":"2024-06-06T07:48:14.523284+0000","last_clean":"2024-06-06T07:48:14.523284+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:14.523284+0000","last_undegraded":"2024-06-06T07:48:14.523284+0000","last_fullsized":"2024-06-06T07:48:14.523284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":487,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.524713+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:15.524713+0000","last_peered":"2024-06-06T07:48:15.524713+0000","last_clean":"2024-06-06T07:48:15.524713+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:15.524713+0000","last_undegraded":"2024-06-06T07:48:15.524713+0000","last_fullsized":"2024-06-06T07:48:15.524713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":270,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501627+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501627+0000","last_peered":"2024-06-06T07:48:00.501627+0000","last_clean":"2024-06-06T07:48:00.501627+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501627+0000","last_undegraded":"2024-06-06T07:48:00.501627+0000","last_fullsized":"2024-06-06T07:48:00.501627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469762+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:48:01.469762+0000","last_peered":"2024-06-06T07:48:01.469762+0000","last_clean":"2024-06-06T07:48:01.469762+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:48:01.469762+0000","last_undegraded":"2024-06-06T07:48:01.469762+0000","last_fullsized":"2024-06-06T07:48:01.469762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":251,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:14.576809+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:14.576809+0000","last_peered":"2024-06-06T07:48:14.576809+0000","last_clean":"2024-06-06T07:48:14.576809+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:14.576809+0000","last_undegraded":"2024-06-06T07:48:14.576809+0000","last_fullsized":"2024-06-06T07:48:14.576809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469638+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:48:01.469638+0000","last_peered":"2024-06-06T07:48:01.469638+0000","last_clean":"2024-06-06T07:48:01.469638+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:48:01.469638+0000","last_undegraded":"2024-06-06T07:48:01.469638+0000","last_fullsized":"2024-06-06T07:48:01.469638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469871+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:48:01.469871+0000","last_peered":"2024-06-06T07:48:01.469871+0000","last_clean":"2024-06-06T07:48:01.469871+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:48:01.469871+0000","last_undegraded":"2024-06-06T07:48:01.469871+0000","last_fullsized":"2024-06-06T07:48:01.469871+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469888+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:48:01.469888+0000","last_peered":"2024-06-06T07:48:01.469888+0000","last_clean":"2024-06-06T07:48:01.469888+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:48:01.469888+0000","last_undegraded":"2024-06-06T07:48:01.469888+0000","last_fullsized":"2024-06-06T07:48:01.469888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501744+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501744+0000","last_peered":"2024-06-06T07:48:00.501744+0000","last_clean":"2024-06-06T07:48:00.501744+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501744+0000","last_undegraded":"2024-06-06T07:48:00.501744+0000","last_fullsized":"2024-06-06T07:48:00.501744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395632+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395632+0000","last_peered":"2024-06-06T07:48:00.395632+0000","last_clean":"2024-06-06T07:48:00.395632+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395632+0000","last_undegraded":"2024-06-06T07:48:00.395632+0000","last_fullsized":"2024-06-06T07:48:00.395632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501561+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501561+0000","last_peered":"2024-06-06T07:48:00.501561+0000","last_clean":"2024-06-06T07:48:00.501561+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501561+0000","last_undegraded":"2024-06-06T07:48:00.501561+0000","last_fullsized":"2024-06-06T07:48:00.501561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498755+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498755+0000","last_peered":"2024-06-06T07:48:00.498755+0000","last_clean":"2024-06-06T07:48:00.498755+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498755+0000","last_undegraded":"2024-06-06T07:48:00.498755+0000","last_fullsized":"2024-06-06T07:48:00.498755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":246,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:14.575929+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:14.575929+0000","last_peered":"2024-06-06T07:48:14.575929+0000","last_clean":"2024-06-06T07:48:14.575929+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:14.575929+0000","last_undegraded":"2024-06-06T07:48:14.575929+0000","last_fullsized":"2024-06-06T07:48:14.575929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491306+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491306+0000","last_peered":"2024-06-06T07:48:00.491306+0000","last_clean":"2024-06-06T07:48:00.491306+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491306+0000","last_undegraded":"2024-06-06T07:48:00.491306+0000","last_fullsized":"2024-06-06T07:48:00.491306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554779+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554779+0000","last_peered":"2024-06-06T07:48:00.554779+0000","last_clean":"2024-06-06T07:48:00.554779+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554779+0000","last_undegraded":"2024-06-06T07:48:00.554779+0000","last_fullsized":"2024-06-06T07:48:00.554779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":426,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:02.523875+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:02.523875+0000","last_peered":"2024-06-06T07:48:02.523875+0000","last_clean":"2024-06-06T07:48:02.523875+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:02.523875+0000","last_undegraded":"2024-06-06T07:48:02.523875+0000","last_fullsized":"2024-06-06T07:48:02.523875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469689+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:48:01.469689+0000","last_peered":"2024-06-06T07:48:01.469689+0000","last_clean":"2024-06-06T07:48:01.469689+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:48:01.469689+0000","last_undegraded":"2024-06-06T07:48:01.469689+0000","last_fullsized":"2024-06-06T07:48:01.469689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491397+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491397+0000","last_peered":"2024-06-06T07:48:00.491397+0000","last_clean":"2024-06-06T07:48:00.491397+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491397+0000","last_undegraded":"2024-06-06T07:48:00.491397+0000","last_fullsized":"2024-06-06T07:48:00.491397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393994+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393994+0000","last_peered":"2024-06-06T07:48:00.393994+0000","last_clean":"2024-06-06T07:48:00.393994+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393994+0000","last_undegraded":"2024-06-06T07:48:00.393994+0000","last_fullsized":"2024-06-06T07:48:00.393994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491724+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.491724+0000","last_peered":"2024-06-06T07:48:00.491724+0000","last_clean":"2024-06-06T07:48:00.491724+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.491724+0000","last_undegraded":"2024-06-06T07:48:00.491724+0000","last_fullsized":"2024-06-06T07:48:00.491724+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.813155+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:15.813155+0000","last_peered":"2024-06-06T07:48:15.813155+0000","last_clean":"2024-06-06T07:48:15.813155+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:15.813155+0000","last_undegraded":"2024-06-06T07:48:15.813155+0000","last_fullsized":"2024-06-06T07:48:15.813155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491172+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491172+0000","last_peered":"2024-06-06T07:48:00.491172+0000","last_clean":"2024-06-06T07:48:00.491172+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491172+0000","last_undegraded":"2024-06-06T07:48:00.491172+0000","last_fullsized":"2024-06-06T07:48:00.491172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392527+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392527+0000","last_peered":"2024-06-06T07:48:00.392527+0000","last_clean":"2024-06-06T07:48:00.392527+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392527+0000","last_undegraded":"2024-06-06T07:48:00.392527+0000","last_fullsized":"2024-06-06T07:48:00.392527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555126+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555126+0000","last_peered":"2024-06-06T07:48:00.555126+0000","last_clean":"2024-06-06T07:48:00.555126+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555126+0000","last_undegraded":"2024-06-06T07:48:00.555126+0000","last_fullsized":"2024-06-06T07:48:00.555126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469845+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469845+0000","last_peered":"2024-06-06T07:48:01.469845+0000","last_clean":"2024-06-06T07:48:01.469845+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469845+0000","last_undegraded":"2024-06-06T07:48:01.469845+0000","last_fullsized":"2024-06-06T07:48:01.469845+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581713+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581713+0000","last_peered":"2024-06-06T07:48:00.581713+0000","last_clean":"2024-06-06T07:48:00.581713+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581713+0000","last_undegraded":"2024-06-06T07:48:00.581713+0000","last_fullsized":"2024-06-06T07:48:00.581713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490919+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490919+0000","last_peered":"2024-06-06T07:48:00.490919+0000","last_clean":"2024-06-06T07:48:00.490919+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490919+0000","last_undegraded":"2024-06-06T07:48:00.490919+0000","last_fullsized":"2024-06-06T07:48:00.490919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491031+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491031+0000","last_peered":"2024-06-06T07:48:00.491031+0000","last_clean":"2024-06-06T07:48:00.491031+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491031+0000","last_undegraded":"2024-06-06T07:48:00.491031+0000","last_fullsized":"2024-06-06T07:48:00.491031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469686+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:48:01.469686+0000","last_peered":"2024-06-06T07:48:01.469686+0000","last_clean":"2024-06-06T07:48:01.469686+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:48:01.469686+0000","last_undegraded":"2024-06-06T07:48:01.469686+0000","last_fullsized":"2024-06-06T07:48:01.469686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392587+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:00.392587+0000","last_peered":"2024-06-06T07:48:00.392587+0000","last_clean":"2024-06-06T07:48:00.392587+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:00.392587+0000","last_undegraded":"2024-06-06T07:48:00.392587+0000","last_fullsized":"2024-06-06T07:48:00.392587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581922+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581922+0000","last_peered":"2024-06-06T07:48:00.581922+0000","last_clean":"2024-06-06T07:48:00.581922+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581922+0000","last_undegraded":"2024-06-06T07:48:00.581922+0000","last_fullsized":"2024-06-06T07:48:00.581922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.456155+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:15.456155+0000","last_peered":"2024-06-06T07:48:15.456155+0000","last_clean":"2024-06-06T07:48:15.456155+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:15.456155+0000","last_undegraded":"2024-06-06T07:48:15.456155+0000","last_fullsized":"2024-06-06T07:48:15.456155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469838+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:48:01.469838+0000","last_peered":"2024-06-06T07:48:01.469838+0000","last_clean":"2024-06-06T07:48:01.469838+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:48:01.469838+0000","last_undegraded":"2024-06-06T07:48:01.469838+0000","last_fullsized":"2024-06-06T07:48:01.469838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392803+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392803+0000","last_peered":"2024-06-06T07:48:00.392803+0000","last_clean":"2024-06-06T07:48:00.392803+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392803+0000","last_undegraded":"2024-06-06T07:48:00.392803+0000","last_fullsized":"2024-06-06T07:48:00.392803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392560+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392560+0000","last_peered":"2024-06-06T07:48:00.392560+0000","last_clean":"2024-06-06T07:48:00.392560+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392560+0000","last_undegraded":"2024-06-06T07:48:00.392560+0000","last_fullsized":"2024-06-06T07:48:00.392560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469841+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469841+0000","last_peered":"2024-06-06T07:48:01.469841+0000","last_clean":"2024-06-06T07:48:01.469841+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469841+0000","last_undegraded":"2024-06-06T07:48:01.469841+0000","last_fullsized":"2024-06-06T07:48:01.469841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393816+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393816+0000","last_peered":"2024-06-06T07:48:00.393816+0000","last_clean":"2024-06-06T07:48:00.393816+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393816+0000","last_undegraded":"2024-06-06T07:48:00.393816+0000","last_fullsized":"2024-06-06T07:48:00.393816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470032+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:48:01.470032+0000","last_peered":"2024-06-06T07:48:01.470032+0000","last_clean":"2024-06-06T07:48:01.470032+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:48:01.470032+0000","last_undegraded":"2024-06-06T07:48:01.470032+0000","last_fullsized":"2024-06-06T07:48:01.470032+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393093+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393093+0000","last_peered":"2024-06-06T07:48:00.393093+0000","last_clean":"2024-06-06T07:48:00.393093+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393093+0000","last_undegraded":"2024-06-06T07:48:00.393093+0000","last_fullsized":"2024-06-06T07:48:00.393093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491388+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491388+0000","last_peered":"2024-06-06T07:48:00.491388+0000","last_clean":"2024-06-06T07:48:00.491388+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491388+0000","last_undegraded":"2024-06-06T07:48:00.491388+0000","last_fullsized":"2024-06-06T07:48:00.491388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470014+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:48:01.470014+0000","last_peered":"2024-06-06T07:48:01.470014+0000","last_clean":"2024-06-06T07:48:01.470014+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:48:01.470014+0000","last_undegraded":"2024-06-06T07:48:01.470014+0000","last_fullsized":"2024-06-06T07:48:01.470014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799288,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1534,"num_read_kb":986,"num_write":433,"num_write_kb":2369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2397864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383747,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":580137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645384,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29276,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715876,"statfs":{"total":95995035648,"available":95965057024,"internally_reserved":0,"allocated":2191360,"data_stored":1572518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808908,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070545,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1178217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495708,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":647597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1249280,"data_stored":649094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37300,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707852,"statfs":{"total":95995035648,"available":95956840448,"internally_reserved":0,"allocated":1888256,"data_stored":1297057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":581294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37512,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707640,"statfs":{"total":95995035648,"available":95956623360,"internally_reserved":0,"allocated":2105344,"data_stored":1500808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:19.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:48:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:20.214 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:20 smithi087 ceph-mon[32276]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T07:48:20.215 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:20 smithi087 ceph-mon[32276]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:20 smithi181 ceph-mon[36790]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T07:48:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:20 smithi181 ceph-mon[36790]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:20 smithi195 ceph-mon[37040]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T07:48:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:20 smithi195 ceph-mon[37040]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/3479652312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f81073f0 msgr2=0x7fe4f81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/3479652312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f81073f0 0x7fe4f81077d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/3479652312 shutdown_connections 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/3479652312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101d40 0x7fe4f810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/3479652312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f81073f0 0x7fe4f81077d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/3479652312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101380 0x7fe4f8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/3479652312 >> 172.21.15.87:0/3479652312 conn(0x7fe4f8078300 msgr2=0x7fe4f8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/3479652312 shutdown_connections 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.057+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/3479652312 wait complete. 2024-06-06T07:48:22.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 Processor -- start 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 -- start start 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 0x7fe4f8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 0x7fe4f81a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8111760 con 0x7fe4f8101d40 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81115e0 con 0x7fe4f81073f0 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4fcbe2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81118e0 con 0x7fe4f8101380 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41092/0 (socket says 172.21.15.87:41092) 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 -- 172.21.15.87:0/337089711 learned_addr learned my addr 172.21.15.87:0/337089711 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4f659c700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 0x7fe4f8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 -- 172.21.15.87:0/337089711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 msgr2=0x7fe4f8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 0x7fe4f81a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 0x7fe4f8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 -- 172.21.15.87:0/337089711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 msgr2=0x7fe4f81a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 0x7fe4f81a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4edd9b700 1 -- 172.21.15.87:0/337089711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f81a5220 con 0x7fe4f8101d40 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4f659c700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 0x7fe4f8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.058+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 0x7fe4f81a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000b530 tx=0x7fe4e000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4effff700 1 -- 172.21.15.87:0/337089711 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0015020 con 0x7fe4f8101d40 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f81a5450 con 0x7fe4f8101d40 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4effff700 1 -- 172.21.15.87:0/337089711 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0011d10 con 0x7fe4f8101d40 2024-06-06T07:48:22.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4effff700 1 -- 172.21.15.87:0/337089711 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0010e50 con 0x7fe4f8101d40 2024-06-06T07:48:22.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.059+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a59e0 con 0x7fe4f8101d40 2024-06-06T07:48:22.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.062+0000 7fe4effff700 1 -- 172.21.15.87:0/337089711 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fe4e0018020 con 0x7fe4f8101d40 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.062+0000 7fe4effff700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 0x7fe4e40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.062+0000 7fe4f659c700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 0x7fe4e40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.062+0000 7fe4effff700 1 -- 172.21.15.87:0/337089711 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fe4e00cf250 con 0x7fe4f8101d40 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe4f659c700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 0x7fe4e40a5510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b5c0 tx=0x7fe4e8005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 Processor -- start 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 -- start start 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63178880 0x559f63172260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559f63064f60 con 0x559f63177f60 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f630c3850 con 0x559f631727a0 2024-06-06T07:48:22.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.063+0000 7fe507465b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f6302b460 con 0x559f63178880 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41102/0 (socket says 172.21.15.87:41102) 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 -- 172.21.15.87:0/2975901290 learned_addr learned my addr 172.21.15.87:0/2975901290 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63178880 0x559f63172260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 -- 172.21.15.87:0/2975901290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63178880 msgr2=0x559f63172260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63178880 0x559f63172260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 -- 172.21.15.87:0/2975901290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 msgr2=0x559f63172bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 -- 172.21.15.87:0/2975901290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f630128d0 con 0x559f63177f60 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80059d0 tx=0x7fe4e801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a5a5d325bf9f18 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4ecd99700 1 -- 172.21.15.87:0/2975901290 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003740 con 0x559f63177f60 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4ecd99700 1 -- 172.21.15.87:0/2975901290 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8003e40 con 0x559f63177f60 2024-06-06T07:48:22.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.064+0000 7fe4ecd99700 1 -- 172.21.15.87:0/2975901290 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800f750 con 0x559f63177f60 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- 172.21.15.87:0/2975901290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 msgr2=0x559f63178340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80059d0 tx=0x7fe4e801e000 comp rx=0 tx=0).stop 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- 172.21.15.87:0/2975901290 shutdown_connections 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63177f60 0x559f63178340 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 --2- 172.21.15.87:0/2975901290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63178880 0x559f63172260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- 172.21.15.87:0/2975901290 >> 172.21.15.87:0/2975901290 conn(0x559f6309f080 msgr2=0x559f6316d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- 172.21.15.87:0/2975901290 shutdown_connections 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- 172.21.15.87:0/2975901290 wait complete. 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 -- start start 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.065+0000 7fe507465b80 1 Processor -- start 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f631727a0 0x559f6317e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63177f60 0x559f6317b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559f63063fc0 con 0x559f63178880 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f630bf810 con 0x559f63177f60 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f6317f200 con 0x559f631727a0 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe507465b80 10 client.0 _ll_register_callbacks cb 0x559f630595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41106/0 (socket says 172.21.15.87:41106) 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/2112696875 learned_addr learned my addr 172.21.15.87:0/2112696875 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f631727a0 0x559f6317e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63177f60 0x559f6317b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/2112696875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f631727a0 msgr2=0x559f6317e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f631727a0 0x559f6317e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63177f60 msgr2=0x559f6317b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63177f60 0x559f6317b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/2112696875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f63089750 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000fe90 tx=0x7fe4f000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aef83395962b867 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.066+0000 7fe4ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000d9c0 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fe4f000db60 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f630d0d50 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4f000dd60 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 10 client.15453 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f63189a10 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559f630d0d50 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe4f00185b0 con 0x559f63178880 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:22.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 20 client.15453 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f6318e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 4 client.15453 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f630cb2d0 con 0x559f6318c0a0 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.067+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f6318e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.068+0000 7fe4f659c700 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f6318e490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fa7fa3ebbb8f97 server_cookie=4c605a81e22c0485 in_seq=0 out_seq=0 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.068+0000 7fe4ddffb700 10 client.15453 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe4ddffb700 1 -- 172.21.15.87:0/2112696875 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f630cb2d0 con 0x559f6318c0a0 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe4ddffb700 10 client.15453 handle_command_reply: tid=0 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 client.15453 shutdown 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 msgr2=0x559f6317be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000fe90 tx=0x7fe4f000fec0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 shutdown_connections 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f6318e490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe4ddffb700 0 client.15453 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317be30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f631727a0 0x559f6317e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 --2- 172.21.15.87:0/2112696875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63177f60 0x559f6317b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 >> 172.21.15.87:0/2112696875 conn(0x559f6309f080 msgr2=0x559f6316c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 shutdown_connections 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 1 -- 172.21.15.87:0/2112696875 wait complete. 2024-06-06T07:48:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.069+0000 7fe507465b80 20 client.15453 trim_cache size 0 max 16384 2024-06-06T07:48:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.089+0000 7fe507465b80 1 Processor -- start 2024-06-06T07:48:22.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.089+0000 7fe507465b80 1 -- start start 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f6317be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81b1110 con 0x559f63178880 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ae9a0 con 0x559f631727a0 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe507465b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ac7f0 con 0x559f63177f60 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41122/0 (socket says 172.21.15.87:41122) 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/3057166736 learned_addr learned my addr 172.21.15.87:0/3057166736 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f6317be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f659c700 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/3057166736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 msgr2=0x559f6317be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f6317be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/3057166736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 msgr2=0x559f63172260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 -- 172.21.15.87:0/3057166736 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f63179e80 con 0x559f63178880 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0018a20 tx=0x7fe4f00071f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166c5c516e90bcb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.090+0000 7fe4dffff700 1 -- 172.21.15.87:0/3057166736 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000d7c0 con 0x559f63178880 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe4dffff700 1 -- 172.21.15.87:0/3057166736 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f0007420 con 0x559f63178880 2024-06-06T07:48:22.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe4dffff700 1 -- 172.21.15.87:0/3057166736 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f002f980 con 0x559f63178880 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 -- 172.21.15.87:0/3057166736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 msgr2=0x559f6317d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0018a20 tx=0x7fe4f00071f0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 -- 172.21.15.87:0/3057166736 shutdown_connections 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f631727a0 0x559f63172260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f63178880 0x559f6317d830 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 --2- 172.21.15.87:0/3057166736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f6317be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 -- 172.21.15.87:0/3057166736 >> 172.21.15.87:0/3057166736 conn(0x559f6309f080 msgr2=0x559f6317d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 -- 172.21.15.87:0/3057166736 shutdown_connections 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.091+0000 7fe507465b80 1 -- 172.21.15.87:0/3057166736 wait complete. 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 -- start start 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 Processor -- start 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f63193ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 0x559f63194430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ace60 con 0x559f631727a0 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ae2d0 con 0x559f63178880 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ad530 con 0x559f63177f60 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe507465b80 10 client.0 _ll_register_callbacks cb 0x559f630595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41126/0 (socket says 172.21.15.87:41126) 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 -- 172.21.15.87:0/579779435 learned_addr learned my addr 172.21.15.87:0/579779435 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f63193ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 0x559f63194430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 -- 172.21.15.87:0/579779435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 msgr2=0x559f63193ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f63193ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 -- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 msgr2=0x559f63194430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:22.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 0x559f63194430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 -- 172.21.15.87:0/579779435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f631f5d00 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f63193ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f6d9d700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 0x559f63194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4f659c700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005010 tx=0x7fe4e80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17b773f7bb8ba19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.092+0000 7fe4ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003c30 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fe4e80050c0 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f6307ef50 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4e8046050 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 10 client.15459 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f631f6fe0 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559f6307ef50 con 0x559f631727a0 2024-06-06T07:48:22.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe4e8041030 con 0x559f631727a0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 20 client.15459 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f631f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 4 client.15459 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.093+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f62f45250 con 0x559f6318c0a0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f631f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4ddffb700 10 client.15459 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4edd9b700 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f631f9330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3582ea9af5e5dbfe server_cookie=e88f46a48a9459bc in_seq=0 out_seq=0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4ddffb700 1 -- 172.21.15.87:0/579779435 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f62f45250 con 0x559f6318c0a0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4ddffb700 10 client.15459 handle_command_reply: tid=0 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 client.15459 shutdown 2024-06-06T07:48:22.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 msgr2=0x559f631939b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005010 tx=0x7fe4e80095c0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 shutdown_connections 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f6318c0a0 0x559f631f9330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe4ddffb700 0 client.15459 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f63178880 0x559f63194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f631727a0 0x559f631939b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 --2- 172.21.15.87:0/579779435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f63177f60 0x559f63193ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.094+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 >> 172.21.15.87:0/579779435 conn(0x559f6309f080 msgr2=0x559f6317d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 shutdown_connections 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe507465b80 1 -- 172.21.15.87:0/579779435 wait complete. 2024-06-06T07:48:22.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe507465b80 20 client.15459 trim_cache size 0 max 16384 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "a2a646fc-574c-4818-b9dd-fa464fe93a70": { 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "a2a646fc-574c-4818-b9dd-fa464fe93a70", 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,force" 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:48:22.101 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 msgr2=0x7fe4e40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 0x7fe4e40a5510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b5c0 tx=0x7fe4e8005fb0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 msgr2=0x7fe4f819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000b530 tx=0x7fe4e000b8f0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 shutdown_connections 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f81073f0 0x7fe4f81a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe4e40a3050 0x7fe4e40a5510 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4f8101380 0x7fe4f8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 --2- 172.21.15.87:0/337089711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4f8101d40 0x7fe4f819a1a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.095+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 >> 172.21.15.87:0/337089711 conn(0x7fe4f8078300 msgr2=0x7fe4f8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.096+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 shutdown_connections 2024-06-06T07:48:22.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.096+0000 7fe4fcbe2700 1 -- 172.21.15.87:0/337089711 wait complete. 2024-06-06T07:48:22.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:22 smithi087 ceph-mon[32276]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:22.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- 172.21.15.87:0/3437849194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 msgr2=0x7f40d4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 --2- 172.21.15.87:0/3437849194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d4102990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009990 tx=0x7f40c8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- 172.21.15.87:0/3437849194 shutdown_connections 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 --2- 172.21.15.87:0/3437849194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4108510 0x7f40d41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 --2- 172.21.15.87:0/3437849194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4102ed0 0x7f40d410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 --2- 172.21.15.87:0/3437849194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d4102990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- 172.21.15.87:0/3437849194 >> 172.21.15.87:0/3437849194 conn(0x7f40d40795b0 msgr2=0x7f40d40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- 172.21.15.87:0/3437849194 shutdown_connections 2024-06-06T07:48:22.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- 172.21.15.87:0/3437849194 wait complete. 2024-06-06T07:48:22.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 Processor -- start 2024-06-06T07:48:22.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.297+0000 7f40da39f700 1 -- start start 2024-06-06T07:48:22.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 0x7f40d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 0x7f40d4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4112760 con 0x7f40d4102510 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f40d41125e0 con 0x7f40d4102ed0 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40da39f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40d41128e0 con 0x7f40d4108510 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 0x7f40d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41142/0 (socket says 172.21.15.87:41142) 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d3fff700 1 -- 172.21.15.87:0/814099559 learned_addr learned my addr 172.21.15.87:0/814099559 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d893c700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 0x7f40d4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 -- 172.21.15.87:0/814099559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 msgr2=0x7f40d4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 0x7f40d4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 -- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 msgr2=0x7f40d419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d37fe700 1 -- 172.21.15.87:0/814099559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d41991a0 con 0x7f40d4102ed0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d893c700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 0x7f40d4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.298+0000 7f40d3fff700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40d37fe700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 0x7f40d419ed70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40c80059d0 tx=0x7f40c80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8027070 con 0x7f40d4102ed0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c8004050 con 0x7f40d4102ed0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8022c00 con 0x7f40d4102ed0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d4199430 con 0x7f40d4102ed0 2024-06-06T07:48:22.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.299+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d41a5700 con 0x7f40d4102ed0 2024-06-06T07:48:22.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.300+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d4103bd0 con 0x7f40d4102ed0 2024-06-06T07:48:22.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.301+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f40c802a070 con 0x7f40d4102ed0 2024-06-06T07:48:22.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.302+0000 7f40d17fa700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 0x7f40bc0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:22.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.302+0000 7f40d3fff700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 0x7f40bc0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:22.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.302+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f40c80d0f50 con 0x7f40d4102ed0 2024-06-06T07:48:22.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.303+0000 7f40d3fff700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 0x7f40bc0a53f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40d4100bd0 tx=0x7f40c400b500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:22.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.306+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40c8031070 con 0x7f40d4102ed0 2024-06-06T07:48:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:22 smithi181 ceph-mon[36790]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:22.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.505+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40d4199f20 con 0x7f40bc0a2f30 2024-06-06T07:48:22.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:22 smithi195 ceph-mon[37040]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:22.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.514+0000 7f40d17fa700 1 -- 172.21.15.87:0/814099559 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339561 (secure 0 0 0) 0x7f40d4199f20 con 0x7f40bc0a2f30 2024-06-06T07:48:22.519 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 msgr2=0x7f40bc0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 0x7f40bc0a53f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40d4100bd0 tx=0x7f40c400b500 comp rx=0 tx=0).stop 2024-06-06T07:48:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 msgr2=0x7f40d419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 0x7f40d419ed70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40c80059d0 tx=0x7f40c80047b0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 shutdown_connections 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40bc0a2f30 0x7f40bc0a53f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d4108510 0x7f40d4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d4102510 0x7f40d419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 --2- 172.21.15.87:0/814099559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d4102ed0 0x7f40d419ed70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 >> 172.21.15.87:0/814099559 conn(0x7f40d40795b0 msgr2=0x7f40d4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 shutdown_connections 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:22.517+0000 7f40da39f700 1 -- 172.21.15.87:0/814099559 wait complete. 2024-06-06T07:48:22.522 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:23.011 INFO:tasks.cephfs.filesystem:scrub status for tag:a2a646fc-574c-4818-b9dd-fa464fe93a70 - {'path': '/', 'tag': 'a2a646fc-574c-4818-b9dd-fa464fe93a70', 'options': 'recursive,force'} 2024-06-06T07:48:23.838 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-06-06T07:48:22.412307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389890,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1801,"num_read_kb":1331,"num_write":636,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392272,"kb_used_data":17780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124549552,"statfs":{"total":1151940427776,"available":1151538741248,"internally_reserved":0,"allocated":18206720,"data_stored":11105636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":346,"num_read_kb":280,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003144"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470157+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:48:01.470157+0000","last_peered":"2024-06-06T07:48:01.470157+0000","last_clean":"2024-06-06T07:48:01.470157+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:48:01.470157+0000","last_undegraded":"2024-06-06T07:48:01.470157+0000","last_fullsized":"2024-06-06T07:48:01.470157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469833+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469833+0000","last_peered":"2024-06-06T07:48:01.469833+0000","last_clean":"2024-06-06T07:48:01.469833+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469833+0000","last_undegraded":"2024-06-06T07:48:01.469833+0000","last_fullsized":"2024-06-06T07:48:01.469833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469631+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469631+0000","last_peered":"2024-06-06T07:48:01.469631+0000","last_clean":"2024-06-06T07:48:01.469631+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469631+0000","last_undegraded":"2024-06-06T07:48:01.469631+0000","last_fullsized":"2024-06-06T07:48:01.469631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470016+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:48:01.470016+0000","last_peered":"2024-06-06T07:48:01.470016+0000","last_clean":"2024-06-06T07:48:01.470016+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:48:01.470016+0000","last_undegraded":"2024-06-06T07:48:01.470016+0000","last_fullsized":"2024-06-06T07:48:01.470016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:06.403648+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:06.403648+0000","last_peered":"2024-06-06T07:48:06.403648+0000","last_clean":"2024-06-06T07:48:06.403648+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:06.403648+0000","last_undegraded":"2024-06-06T07:48:06.403648+0000","last_fullsized":"2024-06-06T07:48:06.403648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469840+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469840+0000","last_peered":"2024-06-06T07:48:01.469840+0000","last_clean":"2024-06-06T07:48:01.469840+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469840+0000","last_undegraded":"2024-06-06T07:48:01.469840+0000","last_fullsized":"2024-06-06T07:48:01.469840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":258,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.531551+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:19.531551+0000","last_peered":"2024-06-06T07:48:19.531551+0000","last_clean":"2024-06-06T07:48:19.531551+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:19.531551+0000","last_undegraded":"2024-06-06T07:48:19.531551+0000","last_fullsized":"2024-06-06T07:48:19.531551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469990+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:48:01.469990+0000","last_peered":"2024-06-06T07:48:01.469990+0000","last_clean":"2024-06-06T07:48:01.469990+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:48:01.469990+0000","last_undegraded":"2024-06-06T07:48:01.469990+0000","last_fullsized":"2024-06-06T07:48:01.469990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469765+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469765+0000","last_peered":"2024-06-06T07:48:01.469765+0000","last_clean":"2024-06-06T07:48:01.469765+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469765+0000","last_undegraded":"2024-06-06T07:48:01.469765+0000","last_fullsized":"2024-06-06T07:48:01.469765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":248,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.530915+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:19.530915+0000","last_peered":"2024-06-06T07:48:19.530915+0000","last_clean":"2024-06-06T07:48:19.530915+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:19.530915+0000","last_undegraded":"2024-06-06T07:48:19.530915+0000","last_fullsized":"2024-06-06T07:48:19.530915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":197,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":497,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:20.532578+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:20.532578+0000","last_peered":"2024-06-06T07:48:20.532578+0000","last_clean":"2024-06-06T07:48:20.532578+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:20.532578+0000","last_undegraded":"2024-06-06T07:48:20.532578+0000","last_fullsized":"2024-06-06T07:48:20.532578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":408,"num_read_kb":275,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501627+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501627+0000","last_peered":"2024-06-06T07:48:00.501627+0000","last_clean":"2024-06-06T07:48:00.501627+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501627+0000","last_undegraded":"2024-06-06T07:48:00.501627+0000","last_fullsized":"2024-06-06T07:48:00.501627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469762+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:48:01.469762+0000","last_peered":"2024-06-06T07:48:01.469762+0000","last_clean":"2024-06-06T07:48:01.469762+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:48:01.469762+0000","last_undegraded":"2024-06-06T07:48:01.469762+0000","last_fullsized":"2024-06-06T07:48:01.469762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":256,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.585881+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:19.585881+0000","last_peered":"2024-06-06T07:48:19.585881+0000","last_clean":"2024-06-06T07:48:19.585881+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:19.585881+0000","last_undegraded":"2024-06-06T07:48:19.585881+0000","last_fullsized":"2024-06-06T07:48:19.585881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":204,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469638+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:48:01.469638+0000","last_peered":"2024-06-06T07:48:01.469638+0000","last_clean":"2024-06-06T07:48:01.469638+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:48:01.469638+0000","last_undegraded":"2024-06-06T07:48:01.469638+0000","last_fullsized":"2024-06-06T07:48:01.469638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469871+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:48:01.469871+0000","last_peered":"2024-06-06T07:48:01.469871+0000","last_clean":"2024-06-06T07:48:01.469871+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:48:01.469871+0000","last_undegraded":"2024-06-06T07:48:01.469871+0000","last_fullsized":"2024-06-06T07:48:01.469871+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469888+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:48:01.469888+0000","last_peered":"2024-06-06T07:48:01.469888+0000","last_clean":"2024-06-06T07:48:01.469888+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:48:01.469888+0000","last_undegraded":"2024-06-06T07:48:01.469888+0000","last_fullsized":"2024-06-06T07:48:01.469888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501744+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501744+0000","last_peered":"2024-06-06T07:48:00.501744+0000","last_clean":"2024-06-06T07:48:00.501744+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501744+0000","last_undegraded":"2024-06-06T07:48:00.501744+0000","last_fullsized":"2024-06-06T07:48:00.501744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395632+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395632+0000","last_peered":"2024-06-06T07:48:00.395632+0000","last_clean":"2024-06-06T07:48:00.395632+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395632+0000","last_undegraded":"2024-06-06T07:48:00.395632+0000","last_fullsized":"2024-06-06T07:48:00.395632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501561+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501561+0000","last_peered":"2024-06-06T07:48:00.501561+0000","last_clean":"2024-06-06T07:48:00.501561+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501561+0000","last_undegraded":"2024-06-06T07:48:00.501561+0000","last_fullsized":"2024-06-06T07:48:00.501561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498755+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498755+0000","last_peered":"2024-06-06T07:48:00.498755+0000","last_clean":"2024-06-06T07:48:00.498755+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498755+0000","last_undegraded":"2024-06-06T07:48:00.498755+0000","last_fullsized":"2024-06-06T07:48:00.498755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":251,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.584850+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:19.584850+0000","last_peered":"2024-06-06T07:48:19.584850+0000","last_clean":"2024-06-06T07:48:19.584850+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:19.584850+0000","last_undegraded":"2024-06-06T07:48:19.584850+0000","last_fullsized":"2024-06-06T07:48:19.584850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":197,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491306+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491306+0000","last_peered":"2024-06-06T07:48:00.491306+0000","last_clean":"2024-06-06T07:48:00.491306+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491306+0000","last_undegraded":"2024-06-06T07:48:00.491306+0000","last_fullsized":"2024-06-06T07:48:00.491306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554779+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554779+0000","last_peered":"2024-06-06T07:48:00.554779+0000","last_clean":"2024-06-06T07:48:00.554779+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554779+0000","last_undegraded":"2024-06-06T07:48:00.554779+0000","last_fullsized":"2024-06-06T07:48:00.554779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":426,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:02.523875+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:02.523875+0000","last_peered":"2024-06-06T07:48:02.523875+0000","last_clean":"2024-06-06T07:48:02.523875+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:02.523875+0000","last_undegraded":"2024-06-06T07:48:02.523875+0000","last_fullsized":"2024-06-06T07:48:02.523875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469745+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:48:01.469745+0000","last_peered":"2024-06-06T07:48:01.469745+0000","last_clean":"2024-06-06T07:48:01.469745+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:48:01.469745+0000","last_undegraded":"2024-06-06T07:48:01.469745+0000","last_fullsized":"2024-06-06T07:48:01.469745+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491397+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491397+0000","last_peered":"2024-06-06T07:48:00.491397+0000","last_clean":"2024-06-06T07:48:00.491397+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491397+0000","last_undegraded":"2024-06-06T07:48:00.491397+0000","last_fullsized":"2024-06-06T07:48:00.491397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393994+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393994+0000","last_peered":"2024-06-06T07:48:00.393994+0000","last_clean":"2024-06-06T07:48:00.393994+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393994+0000","last_undegraded":"2024-06-06T07:48:00.393994+0000","last_fullsized":"2024-06-06T07:48:00.393994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491724+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.491724+0000","last_peered":"2024-06-06T07:48:00.491724+0000","last_clean":"2024-06-06T07:48:00.491724+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.491724+0000","last_undegraded":"2024-06-06T07:48:00.491724+0000","last_fullsized":"2024-06-06T07:48:00.491724+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.813155+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:15.813155+0000","last_peered":"2024-06-06T07:48:15.813155+0000","last_clean":"2024-06-06T07:48:15.813155+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:15.813155+0000","last_undegraded":"2024-06-06T07:48:15.813155+0000","last_fullsized":"2024-06-06T07:48:15.813155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491229+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491229+0000","last_peered":"2024-06-06T07:48:00.491229+0000","last_clean":"2024-06-06T07:48:00.491229+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491229+0000","last_undegraded":"2024-06-06T07:48:00.491229+0000","last_fullsized":"2024-06-06T07:48:00.491229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392527+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392527+0000","last_peered":"2024-06-06T07:48:00.392527+0000","last_clean":"2024-06-06T07:48:00.392527+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392527+0000","last_undegraded":"2024-06-06T07:48:00.392527+0000","last_fullsized":"2024-06-06T07:48:00.392527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555126+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555126+0000","last_peered":"2024-06-06T07:48:00.555126+0000","last_clean":"2024-06-06T07:48:00.555126+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555126+0000","last_undegraded":"2024-06-06T07:48:00.555126+0000","last_fullsized":"2024-06-06T07:48:00.555126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469894+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469894+0000","last_peered":"2024-06-06T07:48:01.469894+0000","last_clean":"2024-06-06T07:48:01.469894+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469894+0000","last_undegraded":"2024-06-06T07:48:01.469894+0000","last_fullsized":"2024-06-06T07:48:01.469894+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581713+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581713+0000","last_peered":"2024-06-06T07:48:00.581713+0000","last_clean":"2024-06-06T07:48:00.581713+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581713+0000","last_undegraded":"2024-06-06T07:48:00.581713+0000","last_fullsized":"2024-06-06T07:48:00.581713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490987+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490987+0000","last_peered":"2024-06-06T07:48:00.490987+0000","last_clean":"2024-06-06T07:48:00.490987+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490987+0000","last_undegraded":"2024-06-06T07:48:00.490987+0000","last_fullsized":"2024-06-06T07:48:00.490987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491084+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491084+0000","last_peered":"2024-06-06T07:48:00.491084+0000","last_clean":"2024-06-06T07:48:00.491084+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491084+0000","last_undegraded":"2024-06-06T07:48:00.491084+0000","last_fullsized":"2024-06-06T07:48:00.491084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469761+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:48:01.469761+0000","last_peered":"2024-06-06T07:48:01.469761+0000","last_clean":"2024-06-06T07:48:01.469761+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:48:01.469761+0000","last_undegraded":"2024-06-06T07:48:01.469761+0000","last_fullsized":"2024-06-06T07:48:01.469761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.243949+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:01.243949+0000","last_peered":"2024-06-06T07:48:01.243949+0000","last_clean":"2024-06-06T07:48:01.243949+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:01.243949+0000","last_undegraded":"2024-06-06T07:48:01.243949+0000","last_fullsized":"2024-06-06T07:48:01.243949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581922+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581922+0000","last_peered":"2024-06-06T07:48:00.581922+0000","last_clean":"2024-06-06T07:48:00.581922+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581922+0000","last_undegraded":"2024-06-06T07:48:00.581922+0000","last_fullsized":"2024-06-06T07:48:00.581922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.456155+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:15.456155+0000","last_peered":"2024-06-06T07:48:15.456155+0000","last_clean":"2024-06-06T07:48:15.456155+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:15.456155+0000","last_undegraded":"2024-06-06T07:48:15.456155+0000","last_fullsized":"2024-06-06T07:48:15.456155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469838+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:48:01.469838+0000","last_peered":"2024-06-06T07:48:01.469838+0000","last_clean":"2024-06-06T07:48:01.469838+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:48:01.469838+0000","last_undegraded":"2024-06-06T07:48:01.469838+0000","last_fullsized":"2024-06-06T07:48:01.469838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392803+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392803+0000","last_peered":"2024-06-06T07:48:00.392803+0000","last_clean":"2024-06-06T07:48:00.392803+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392803+0000","last_undegraded":"2024-06-06T07:48:00.392803+0000","last_fullsized":"2024-06-06T07:48:00.392803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392560+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392560+0000","last_peered":"2024-06-06T07:48:00.392560+0000","last_clean":"2024-06-06T07:48:00.392560+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392560+0000","last_undegraded":"2024-06-06T07:48:00.392560+0000","last_fullsized":"2024-06-06T07:48:00.392560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469841+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469841+0000","last_peered":"2024-06-06T07:48:01.469841+0000","last_clean":"2024-06-06T07:48:01.469841+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469841+0000","last_undegraded":"2024-06-06T07:48:01.469841+0000","last_fullsized":"2024-06-06T07:48:01.469841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393816+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393816+0000","last_peered":"2024-06-06T07:48:00.393816+0000","last_clean":"2024-06-06T07:48:00.393816+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393816+0000","last_undegraded":"2024-06-06T07:48:00.393816+0000","last_fullsized":"2024-06-06T07:48:00.393816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:48:01.470081+0000","last_peered":"2024-06-06T07:48:01.470081+0000","last_clean":"2024-06-06T07:48:01.470081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:48:01.470081+0000","last_undegraded":"2024-06-06T07:48:01.470081+0000","last_fullsized":"2024-06-06T07:48:01.470081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393137+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393137+0000","last_peered":"2024-06-06T07:48:00.393137+0000","last_clean":"2024-06-06T07:48:00.393137+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393137+0000","last_undegraded":"2024-06-06T07:48:00.393137+0000","last_fullsized":"2024-06-06T07:48:00.393137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491436+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491436+0000","last_peered":"2024-06-06T07:48:00.491436+0000","last_clean":"2024-06-06T07:48:00.491436+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491436+0000","last_undegraded":"2024-06-06T07:48:00.491436+0000","last_fullsized":"2024-06-06T07:48:00.491436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470014+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:48:01.470014+0000","last_peered":"2024-06-06T07:48:01.470014+0000","last_clean":"2024-06-06T07:48:01.470014+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:48:01.470014+0000","last_undegraded":"2024-06-06T07:48:01.470014+0000","last_fullsized":"2024-06-06T07:48:01.470014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799288,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1564,"num_read_kb":1001,"num_write":433,"num_write_kb":2369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2397864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383748,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":580137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645385,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29276,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715876,"statfs":{"total":95995035648,"available":95965057024,"internally_reserved":0,"allocated":2191360,"data_stored":1572518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332183,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1178217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495709,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":647597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1249280,"data_stored":649094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37300,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707852,"statfs":{"total":95995035648,"available":95956840448,"internally_reserved":0,"allocated":1888256,"data_stored":1297057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":581294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37512,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707640,"statfs":{"total":95995035648,"available":95956623360,"internally_reserved":0,"allocated":2105344,"data_stored":1500808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:23.843 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T07:48:23.843 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T07:48:24.079 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:24 smithi087 ceph-mon[32276]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T07:48:24.079 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:24 smithi087 ceph-mon[32276]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:24.095 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:24 smithi181 ceph-mon[36790]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T07:48:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:24 smithi181 ceph-mon[36790]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:24 smithi195 ceph-mon[37040]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T07:48:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:24 smithi195 ceph-mon[37040]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 -- 172.21.15.87:0/262221270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101fd0 msgr2=0x7fc6cc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/262221270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc105fd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0099c0 tx=0x7fc6bc009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 -- 172.21.15.87:0/262221270 shutdown_connections 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/262221270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc1066b0 0x7fc6cc10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/262221270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc101620 0x7fc6cc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/262221270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc105fd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 -- 172.21.15.87:0/262221270 >> 172.21.15.87:0/262221270 conn(0x7fc6cc078360 msgr2=0x7fc6cc078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 -- 172.21.15.87:0/262221270 shutdown_connections 2024-06-06T07:48:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.907+0000 7fc6d15a1700 1 -- 172.21.15.87:0/262221270 wait complete. 2024-06-06T07:48:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.908+0000 7fc6d15a1700 1 Processor -- start 2024-06-06T07:48:25.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.908+0000 7fc6d15a1700 1 -- start start 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.908+0000 7fc6d15a1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.908+0000 7fc6d15a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.908+0000 7fc6d15a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1066b0 0x7fc6cc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6d15a1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc111690 con 0x7fc6cc101620 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6d15a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc111510 con 0x7fc6cc101fd0 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6d15a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc111810 con 0x7fc6cc1066b0 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41160/0 (socket says 172.21.15.87:41160) 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 -- 172.21.15.87:0/3015696043 learned_addr learned my addr 172.21.15.87:0/3015696043 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6ca7fc700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6cb7fe700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1066b0 0x7fc6cc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:25.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 -- 172.21.15.87:0/3015696043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1066b0 msgr2=0x7fc6cc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1066b0 0x7fc6cc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 -- 172.21.15.87:0/3015696043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc101fd0 msgr2=0x7fc6cc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 -- 172.21.15.87:0/3015696043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc1a7150 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6caffd700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c970 tx=0x7fc6c000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0007aa0 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6c0004d10 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc1a7440 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000f450 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.909+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc1a7900 con 0x7fc6cc101620 2024-06-06T07:48:25.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.910+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6cc069180 con 0x7fc6cc101620 2024-06-06T07:48:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.912+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc6c0004750 con 0x7fc6cc101620 2024-06-06T07:48:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.912+0000 7fc6b3fff700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 0x7fc6b40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.912+0000 7fc6ca7fc700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 0x7fc6b40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.912+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc6c00cf960 con 0x7fc6cc101620 2024-06-06T07:48:25.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.913+0000 7fc6ca7fc700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 0x7fc6b40a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc005fd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:25.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:25.917+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc6c009bff0 con 0x7fc6cc101620 2024-06-06T07:48:26.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.108+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6cc0628d0 con 0x7fc6b40a2eb0 2024-06-06T07:48:26.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.117+0000 7fc6b3fff700 1 -- 172.21.15.87:0/3015696043 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339561 (secure 0 0 0) 0x7fc6cc0628d0 con 0x7fc6b40a2eb0 2024-06-06T07:48:26.122 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 msgr2=0x7fc6b40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 0x7fc6b40a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc005fd0 comp rx=0 tx=0).stop 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 msgr2=0x7fc6cc197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c970 tx=0x7fc6c000cd30 comp rx=0 tx=0).stop 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 shutdown_connections 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc6b40a2eb0 0x7fc6b40a5370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:26.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6cc1066b0 0x7fc6cc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6cc101620 0x7fc6cc197940 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 --2- 172.21.15.87:0/3015696043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6cc101fd0 0x7fc6cc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 >> 172.21.15.87:0/3015696043 conn(0x7fc6cc078360 msgr2=0x7fc6cc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 shutdown_connections 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:26.122+0000 7fc6d15a1700 1 -- 172.21.15.87:0/3015696043 wait complete. 2024-06-06T07:48:26.127 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-06-06T07:48:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:26 smithi181 ceph-mon[36790]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:26 smithi181 ceph-mon[36790]: scrub summary: idle 2024-06-06T07:48:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:26 smithi087 ceph-mon[32276]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:26 smithi087 ceph-mon[32276]: scrub summary: idle 2024-06-06T07:48:26.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:26 smithi195 ceph-mon[37040]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T07:48:26.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:26 smithi195 ceph-mon[37040]: scrub summary: idle 2024-06-06T07:48:26.840 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-06-06T07:48:24.412993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389890,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1801,"num_read_kb":1331,"num_write":636,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181452,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392272,"kb_used_data":17780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124549552,"statfs":{"total":1151940427776,"available":1151538741248,"internally_reserved":0,"allocated":18206720,"data_stored":11105636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":315,"num_read_kb":262,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003469"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470157+0000","last_change":"2024-06-06T07:44:00.348525+0000","last_active":"2024-06-06T07:48:01.470157+0000","last_peered":"2024-06-06T07:48:01.470157+0000","last_clean":"2024-06-06T07:48:01.470157+0000","last_became_active":"2024-06-06T07:44:00.348170+0000","last_became_peered":"2024-06-06T07:44:00.348170+0000","last_unstale":"2024-06-06T07:48:01.470157+0000","last_undegraded":"2024-06-06T07:48:01.470157+0000","last_fullsized":"2024-06-06T07:48:01.470157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:21:22.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469833+0000","last_change":"2024-06-06T07:44:00.344349+0000","last_active":"2024-06-06T07:48:01.469833+0000","last_peered":"2024-06-06T07:48:01.469833+0000","last_clean":"2024-06-06T07:48:01.469833+0000","last_became_active":"2024-06-06T07:44:00.343917+0000","last_became_peered":"2024-06-06T07:44:00.343917+0000","last_unstale":"2024-06-06T07:48:01.469833+0000","last_undegraded":"2024-06-06T07:48:01.469833+0000","last_fullsized":"2024-06-06T07:48:01.469833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:15:32.587850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469631+0000","last_change":"2024-06-06T07:44:00.348878+0000","last_active":"2024-06-06T07:48:01.469631+0000","last_peered":"2024-06-06T07:48:01.469631+0000","last_clean":"2024-06-06T07:48:01.469631+0000","last_became_active":"2024-06-06T07:44:00.348561+0000","last_became_peered":"2024-06-06T07:44:00.348561+0000","last_unstale":"2024-06-06T07:48:01.469631+0000","last_undegraded":"2024-06-06T07:48:01.469631+0000","last_fullsized":"2024-06-06T07:48:01.469631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:37:57.576279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555225+0000","last_change":"2024-06-06T07:44:00.351772+0000","last_active":"2024-06-06T07:48:00.555225+0000","last_peered":"2024-06-06T07:48:00.555225+0000","last_clean":"2024-06-06T07:48:00.555225+0000","last_became_active":"2024-06-06T07:44:00.351331+0000","last_became_peered":"2024-06-06T07:44:00.351331+0000","last_unstale":"2024-06-06T07:48:00.555225+0000","last_undegraded":"2024-06-06T07:48:00.555225+0000","last_fullsized":"2024-06-06T07:48:00.555225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:58:08.642032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491025+0000","last_change":"2024-06-06T07:44:00.333362+0000","last_active":"2024-06-06T07:48:00.491025+0000","last_peered":"2024-06-06T07:48:00.491025+0000","last_clean":"2024-06-06T07:48:00.491025+0000","last_became_active":"2024-06-06T07:44:00.332998+0000","last_became_peered":"2024-06-06T07:44:00.332998+0000","last_unstale":"2024-06-06T07:48:00.491025+0000","last_undegraded":"2024-06-06T07:48:00.491025+0000","last_fullsized":"2024-06-06T07:48:00.491025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:14.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501533+0000","last_change":"2024-06-06T07:44:00.341727+0000","last_active":"2024-06-06T07:48:00.501533+0000","last_peered":"2024-06-06T07:48:00.501533+0000","last_clean":"2024-06-06T07:48:00.501533+0000","last_became_active":"2024-06-06T07:44:00.341390+0000","last_became_peered":"2024-06-06T07:44:00.341390+0000","last_unstale":"2024-06-06T07:48:00.501533+0000","last_undegraded":"2024-06-06T07:48:00.501533+0000","last_fullsized":"2024-06-06T07:48:00.501533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:53:17.944718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395447+0000","last_change":"2024-06-06T07:44:00.338874+0000","last_active":"2024-06-06T07:48:00.395447+0000","last_peered":"2024-06-06T07:48:00.395447+0000","last_clean":"2024-06-06T07:48:00.395447+0000","last_became_active":"2024-06-06T07:44:00.338540+0000","last_became_peered":"2024-06-06T07:44:00.338540+0000","last_unstale":"2024-06-06T07:48:00.395447+0000","last_undegraded":"2024-06-06T07:48:00.395447+0000","last_fullsized":"2024-06-06T07:48:00.395447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:42:34.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490872+0000","last_change":"2024-06-06T07:44:00.339834+0000","last_active":"2024-06-06T07:48:00.490872+0000","last_peered":"2024-06-06T07:48:00.490872+0000","last_clean":"2024-06-06T07:48:00.490872+0000","last_became_active":"2024-06-06T07:44:00.339279+0000","last_became_peered":"2024-06-06T07:44:00.339279+0000","last_unstale":"2024-06-06T07:48:00.490872+0000","last_undegraded":"2024-06-06T07:48:00.490872+0000","last_fullsized":"2024-06-06T07:48:00.490872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:15.826070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488888+0000","last_change":"2024-06-06T07:44:47.255204+0000","last_active":"2024-06-06T07:48:00.488888+0000","last_peered":"2024-06-06T07:48:00.488888+0000","last_clean":"2024-06-06T07:48:00.488888+0000","last_became_active":"2024-06-06T07:44:47.254947+0000","last_became_peered":"2024-06-06T07:44:47.254947+0000","last_unstale":"2024-06-06T07:48:00.488888+0000","last_undegraded":"2024-06-06T07:48:00.488888+0000","last_fullsized":"2024-06-06T07:48:00.488888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:36.613600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501891+0000","last_change":"2024-06-06T07:44:00.343070+0000","last_active":"2024-06-06T07:48:00.501891+0000","last_peered":"2024-06-06T07:48:00.501891+0000","last_clean":"2024-06-06T07:48:00.501891+0000","last_became_active":"2024-06-06T07:44:00.342730+0000","last_became_peered":"2024-06-06T07:44:00.342730+0000","last_unstale":"2024-06-06T07:48:00.501891+0000","last_undegraded":"2024-06-06T07:48:00.501891+0000","last_fullsized":"2024-06-06T07:48:00.501891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:43:15.631154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396051+0000","last_change":"2024-06-06T07:44:00.345927+0000","last_active":"2024-06-06T07:48:00.396051+0000","last_peered":"2024-06-06T07:48:00.396051+0000","last_clean":"2024-06-06T07:48:00.396051+0000","last_became_active":"2024-06-06T07:44:00.343985+0000","last_became_peered":"2024-06-06T07:44:00.343985+0000","last_unstale":"2024-06-06T07:48:00.396051+0000","last_undegraded":"2024-06-06T07:48:00.396051+0000","last_fullsized":"2024-06-06T07:48:00.396051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:58.014166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555046+0000","last_change":"2024-06-06T07:44:00.343634+0000","last_active":"2024-06-06T07:48:00.555046+0000","last_peered":"2024-06-06T07:48:00.555046+0000","last_clean":"2024-06-06T07:48:00.555046+0000","last_became_active":"2024-06-06T07:44:00.343310+0000","last_became_peered":"2024-06-06T07:44:00.343310+0000","last_unstale":"2024-06-06T07:48:00.555046+0000","last_undegraded":"2024-06-06T07:48:00.555046+0000","last_fullsized":"2024-06-06T07:48:00.555046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:56.741245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393739+0000","last_change":"2024-06-06T07:44:00.339229+0000","last_active":"2024-06-06T07:48:00.393739+0000","last_peered":"2024-06-06T07:48:00.393739+0000","last_clean":"2024-06-06T07:48:00.393739+0000","last_became_active":"2024-06-06T07:44:00.338901+0000","last_became_peered":"2024-06-06T07:44:00.338901+0000","last_unstale":"2024-06-06T07:48:00.393739+0000","last_undegraded":"2024-06-06T07:48:00.393739+0000","last_fullsized":"2024-06-06T07:48:00.393739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:55:57.927440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469435+0000","last_change":"2024-06-06T07:44:00.345273+0000","last_active":"2024-06-06T07:48:01.469435+0000","last_peered":"2024-06-06T07:48:01.469435+0000","last_clean":"2024-06-06T07:48:01.469435+0000","last_became_active":"2024-06-06T07:44:00.344941+0000","last_became_peered":"2024-06-06T07:44:00.344941+0000","last_unstale":"2024-06-06T07:48:01.469435+0000","last_undegraded":"2024-06-06T07:48:01.469435+0000","last_fullsized":"2024-06-06T07:48:01.469435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:54.491866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469466+0000","last_change":"2024-06-06T07:44:47.251796+0000","last_active":"2024-06-06T07:48:01.469466+0000","last_peered":"2024-06-06T07:48:01.469466+0000","last_clean":"2024-06-06T07:48:01.469466+0000","last_became_active":"2024-06-06T07:44:47.251453+0000","last_became_peered":"2024-06-06T07:44:47.251453+0000","last_unstale":"2024-06-06T07:48:01.469466+0000","last_undegraded":"2024-06-06T07:48:01.469466+0000","last_fullsized":"2024-06-06T07:48:01.469466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:26.151227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581588+0000","last_change":"2024-06-06T07:44:00.342752+0000","last_active":"2024-06-06T07:48:00.581588+0000","last_peered":"2024-06-06T07:48:00.581588+0000","last_clean":"2024-06-06T07:48:00.581588+0000","last_became_active":"2024-06-06T07:44:00.342366+0000","last_became_peered":"2024-06-06T07:44:00.342366+0000","last_unstale":"2024-06-06T07:48:00.581588+0000","last_undegraded":"2024-06-06T07:48:00.581588+0000","last_fullsized":"2024-06-06T07:48:00.581588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:14:20.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470016+0000","last_change":"2024-06-06T07:44:00.349445+0000","last_active":"2024-06-06T07:48:01.470016+0000","last_peered":"2024-06-06T07:48:01.470016+0000","last_clean":"2024-06-06T07:48:01.470016+0000","last_became_active":"2024-06-06T07:44:00.349184+0000","last_became_peered":"2024-06-06T07:44:00.349184+0000","last_unstale":"2024-06-06T07:48:01.470016+0000","last_undegraded":"2024-06-06T07:48:01.470016+0000","last_fullsized":"2024-06-06T07:48:01.470016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:40.631580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488994+0000","last_change":"2024-06-06T07:44:00.343869+0000","last_active":"2024-06-06T07:48:00.488994+0000","last_peered":"2024-06-06T07:48:00.488994+0000","last_clean":"2024-06-06T07:48:00.488994+0000","last_became_active":"2024-06-06T07:44:00.343546+0000","last_became_peered":"2024-06-06T07:44:00.343546+0000","last_unstale":"2024-06-06T07:48:00.488994+0000","last_undegraded":"2024-06-06T07:48:00.488994+0000","last_fullsized":"2024-06-06T07:48:00.488994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:12:30.973328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392986+0000","last_change":"2024-06-06T07:44:00.343505+0000","last_active":"2024-06-06T07:48:00.392986+0000","last_peered":"2024-06-06T07:48:00.392986+0000","last_clean":"2024-06-06T07:48:00.392986+0000","last_became_active":"2024-06-06T07:44:00.343043+0000","last_became_peered":"2024-06-06T07:44:00.343043+0000","last_unstale":"2024-06-06T07:48:00.392986+0000","last_undegraded":"2024-06-06T07:48:00.392986+0000","last_fullsized":"2024-06-06T07:48:00.392986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:39.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392441+0000","last_change":"2024-06-06T07:44:11.401074+0000","last_active":"2024-06-06T07:48:00.392441+0000","last_peered":"2024-06-06T07:48:00.392441+0000","last_clean":"2024-06-06T07:48:00.392441+0000","last_became_active":"2024-06-06T07:44:11.400614+0000","last_became_peered":"2024-06-06T07:44:11.400614+0000","last_unstale":"2024-06-06T07:48:00.392441+0000","last_undegraded":"2024-06-06T07:48:00.392441+0000","last_fullsized":"2024-06-06T07:48:00.392441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:18.674630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:06.403648+0000","last_change":"2024-06-06T07:43:57.215254+0000","last_active":"2024-06-06T07:48:06.403648+0000","last_peered":"2024-06-06T07:48:06.403648+0000","last_clean":"2024-06-06T07:48:06.403648+0000","last_became_active":"2024-06-06T07:43:57.214997+0000","last_became_peered":"2024-06-06T07:43:57.214997+0000","last_unstale":"2024-06-06T07:48:06.403648+0000","last_undegraded":"2024-06-06T07:48:06.403648+0000","last_fullsized":"2024-06-06T07:48:06.403648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:19.244660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491426+0000","last_change":"2024-06-06T07:44:00.342744+0000","last_active":"2024-06-06T07:48:00.491426+0000","last_peered":"2024-06-06T07:48:00.491426+0000","last_clean":"2024-06-06T07:48:00.491426+0000","last_became_active":"2024-06-06T07:44:00.342391+0000","last_became_peered":"2024-06-06T07:44:00.342391+0000","last_unstale":"2024-06-06T07:48:00.491426+0000","last_undegraded":"2024-06-06T07:48:00.491426+0000","last_fullsized":"2024-06-06T07:48:00.491426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:35:14.990578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554607+0000","last_change":"2024-06-06T07:44:11.393131+0000","last_active":"2024-06-06T07:48:00.554607+0000","last_peered":"2024-06-06T07:48:00.554607+0000","last_clean":"2024-06-06T07:48:00.554607+0000","last_became_active":"2024-06-06T07:44:11.392488+0000","last_became_peered":"2024-06-06T07:44:11.392488+0000","last_unstale":"2024-06-06T07:48:00.554607+0000","last_undegraded":"2024-06-06T07:48:00.554607+0000","last_fullsized":"2024-06-06T07:48:00.554607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:35:04.433793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491670+0000","last_change":"2024-06-06T07:43:57.225624+0000","last_active":"2024-06-06T07:48:00.491670+0000","last_peered":"2024-06-06T07:48:00.491670+0000","last_clean":"2024-06-06T07:48:00.491670+0000","last_became_active":"2024-06-06T07:43:57.225025+0000","last_became_peered":"2024-06-06T07:43:57.225025+0000","last_unstale":"2024-06-06T07:48:00.491670+0000","last_undegraded":"2024-06-06T07:48:00.491670+0000","last_fullsized":"2024-06-06T07:48:00.491670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:20:25.552878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469840+0000","last_change":"2024-06-06T07:44:00.350481+0000","last_active":"2024-06-06T07:48:01.469840+0000","last_peered":"2024-06-06T07:48:01.469840+0000","last_clean":"2024-06-06T07:48:01.469840+0000","last_became_active":"2024-06-06T07:44:00.350018+0000","last_became_peered":"2024-06-06T07:44:00.350018+0000","last_unstale":"2024-06-06T07:48:01.469840+0000","last_undegraded":"2024-06-06T07:48:01.469840+0000","last_fullsized":"2024-06-06T07:48:01.469840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:02.650023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554625+0000","last_change":"2024-06-06T07:44:11.394847+0000","last_active":"2024-06-06T07:48:00.554625+0000","last_peered":"2024-06-06T07:48:00.554625+0000","last_clean":"2024-06-06T07:48:00.554625+0000","last_became_active":"2024-06-06T07:44:11.393516+0000","last_became_peered":"2024-06-06T07:44:11.393516+0000","last_unstale":"2024-06-06T07:48:00.554625+0000","last_undegraded":"2024-06-06T07:48:00.554625+0000","last_fullsized":"2024-06-06T07:48:00.554625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:19:01.859313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491216+0000","last_change":"2024-06-06T07:43:57.217062+0000","last_active":"2024-06-06T07:48:00.491216+0000","last_peered":"2024-06-06T07:48:00.491216+0000","last_clean":"2024-06-06T07:48:00.491216+0000","last_became_active":"2024-06-06T07:43:57.216664+0000","last_became_peered":"2024-06-06T07:43:57.216664+0000","last_unstale":"2024-06-06T07:48:00.491216+0000","last_undegraded":"2024-06-06T07:48:00.491216+0000","last_fullsized":"2024-06-06T07:48:00.491216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:33:36.606891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581889+0000","last_change":"2024-06-06T07:44:00.346737+0000","last_active":"2024-06-06T07:48:00.581889+0000","last_peered":"2024-06-06T07:48:00.581889+0000","last_clean":"2024-06-06T07:48:00.581889+0000","last_became_active":"2024-06-06T07:44:00.346350+0000","last_became_peered":"2024-06-06T07:44:00.346350+0000","last_unstale":"2024-06-06T07:48:00.581889+0000","last_undegraded":"2024-06-06T07:48:00.581889+0000","last_fullsized":"2024-06-06T07:48:00.581889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:21.251279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554600+0000","last_change":"2024-06-06T07:44:11.394776+0000","last_active":"2024-06-06T07:48:00.554600+0000","last_peered":"2024-06-06T07:48:00.554600+0000","last_clean":"2024-06-06T07:48:00.554600+0000","last_became_active":"2024-06-06T07:44:11.393514+0000","last_became_peered":"2024-06-06T07:44:11.393514+0000","last_unstale":"2024-06-06T07:48:00.554600+0000","last_undegraded":"2024-06-06T07:48:00.554600+0000","last_fullsized":"2024-06-06T07:48:00.554600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:59:18.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491101+0000","last_change":"2024-06-06T07:43:57.216178+0000","last_active":"2024-06-06T07:48:00.491101+0000","last_peered":"2024-06-06T07:48:00.491101+0000","last_clean":"2024-06-06T07:48:00.491101+0000","last_became_active":"2024-06-06T07:43:57.215704+0000","last_became_peered":"2024-06-06T07:43:57.215704+0000","last_unstale":"2024-06-06T07:48:00.491101+0000","last_undegraded":"2024-06-06T07:48:00.491101+0000","last_fullsized":"2024-06-06T07:48:00.491101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:21.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498903+0000","last_change":"2024-06-06T07:44:47.257530+0000","last_active":"2024-06-06T07:48:00.498903+0000","last_peered":"2024-06-06T07:48:00.498903+0000","last_clean":"2024-06-06T07:48:00.498903+0000","last_became_active":"2024-06-06T07:44:47.257083+0000","last_became_peered":"2024-06-06T07:44:47.257083+0000","last_unstale":"2024-06-06T07:48:00.498903+0000","last_undegraded":"2024-06-06T07:48:00.498903+0000","last_fullsized":"2024-06-06T07:48:00.498903+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:06:14.050044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392822+0000","last_change":"2024-06-06T07:44:11.396898+0000","last_active":"2024-06-06T07:48:00.392822+0000","last_peered":"2024-06-06T07:48:00.392822+0000","last_clean":"2024-06-06T07:48:00.392822+0000","last_became_active":"2024-06-06T07:44:11.396332+0000","last_became_peered":"2024-06-06T07:44:11.396332+0000","last_unstale":"2024-06-06T07:48:00.392822+0000","last_undegraded":"2024-06-06T07:48:00.392822+0000","last_fullsized":"2024-06-06T07:48:00.392822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:24.188603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":258,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.531551+0000","last_change":"2024-06-06T07:43:57.217955+0000","last_active":"2024-06-06T07:48:19.531551+0000","last_peered":"2024-06-06T07:48:19.531551+0000","last_clean":"2024-06-06T07:48:19.531551+0000","last_became_active":"2024-06-06T07:43:57.217580+0000","last_became_peered":"2024-06-06T07:43:57.217580+0000","last_unstale":"2024-06-06T07:48:19.531551+0000","last_undegraded":"2024-06-06T07:48:19.531551+0000","last_fullsized":"2024-06-06T07:48:19.531551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:54.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491077+0000","last_change":"2024-06-06T07:44:00.335925+0000","last_active":"2024-06-06T07:48:00.491077+0000","last_peered":"2024-06-06T07:48:00.491077+0000","last_clean":"2024-06-06T07:48:00.491077+0000","last_became_active":"2024-06-06T07:44:00.335144+0000","last_became_peered":"2024-06-06T07:44:00.335144+0000","last_unstale":"2024-06-06T07:48:00.491077+0000","last_undegraded":"2024-06-06T07:48:00.491077+0000","last_fullsized":"2024-06-06T07:48:00.491077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:25:32.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392816+0000","last_change":"2024-06-06T07:44:11.404081+0000","last_active":"2024-06-06T07:48:00.392816+0000","last_peered":"2024-06-06T07:48:00.392816+0000","last_clean":"2024-06-06T07:48:00.392816+0000","last_became_active":"2024-06-06T07:44:11.403629+0000","last_became_peered":"2024-06-06T07:44:11.403629+0000","last_unstale":"2024-06-06T07:48:00.392816+0000","last_undegraded":"2024-06-06T07:48:00.392816+0000","last_fullsized":"2024-06-06T07:48:00.392816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:27.308167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501495+0000","last_change":"2024-06-06T07:43:57.217625+0000","last_active":"2024-06-06T07:48:00.501495+0000","last_peered":"2024-06-06T07:48:00.501495+0000","last_clean":"2024-06-06T07:48:00.501495+0000","last_became_active":"2024-06-06T07:43:57.217097+0000","last_became_peered":"2024-06-06T07:43:57.217097+0000","last_unstale":"2024-06-06T07:48:00.501495+0000","last_undegraded":"2024-06-06T07:48:00.501495+0000","last_fullsized":"2024-06-06T07:48:00.501495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:55.283386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498810+0000","last_change":"2024-06-06T07:44:00.345706+0000","last_active":"2024-06-06T07:48:00.498810+0000","last_peered":"2024-06-06T07:48:00.498810+0000","last_clean":"2024-06-06T07:48:00.498810+0000","last_became_active":"2024-06-06T07:44:00.345315+0000","last_became_peered":"2024-06-06T07:44:00.345315+0000","last_unstale":"2024-06-06T07:48:00.498810+0000","last_undegraded":"2024-06-06T07:48:00.498810+0000","last_fullsized":"2024-06-06T07:48:00.498810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:12:09.291271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469990+0000","last_change":"2024-06-06T07:44:11.403608+0000","last_active":"2024-06-06T07:48:01.469990+0000","last_peered":"2024-06-06T07:48:01.469990+0000","last_clean":"2024-06-06T07:48:01.469990+0000","last_became_active":"2024-06-06T07:44:11.403167+0000","last_became_peered":"2024-06-06T07:44:11.403167+0000","last_unstale":"2024-06-06T07:48:01.469990+0000","last_undegraded":"2024-06-06T07:48:01.469990+0000","last_fullsized":"2024-06-06T07:48:01.469990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:58.985392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393609+0000","last_change":"2024-06-06T07:43:57.211356+0000","last_active":"2024-06-06T07:48:00.393609+0000","last_peered":"2024-06-06T07:48:00.393609+0000","last_clean":"2024-06-06T07:48:00.393609+0000","last_became_active":"2024-06-06T07:43:57.210969+0000","last_became_peered":"2024-06-06T07:43:57.210969+0000","last_unstale":"2024-06-06T07:48:00.393609+0000","last_undegraded":"2024-06-06T07:48:00.393609+0000","last_fullsized":"2024-06-06T07:48:00.393609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:10.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491058+0000","last_change":"2024-06-06T07:44:00.341006+0000","last_active":"2024-06-06T07:48:00.491058+0000","last_peered":"2024-06-06T07:48:00.491058+0000","last_clean":"2024-06-06T07:48:00.491058+0000","last_became_active":"2024-06-06T07:44:00.340530+0000","last_became_peered":"2024-06-06T07:44:00.340530+0000","last_unstale":"2024-06-06T07:48:00.491058+0000","last_undegraded":"2024-06-06T07:48:00.491058+0000","last_fullsized":"2024-06-06T07:48:00.491058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:52.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469765+0000","last_change":"2024-06-06T07:44:11.397490+0000","last_active":"2024-06-06T07:48:01.469765+0000","last_peered":"2024-06-06T07:48:01.469765+0000","last_clean":"2024-06-06T07:48:01.469765+0000","last_became_active":"2024-06-06T07:44:11.397007+0000","last_became_peered":"2024-06-06T07:44:11.397007+0000","last_unstale":"2024-06-06T07:48:01.469765+0000","last_undegraded":"2024-06-06T07:48:01.469765+0000","last_fullsized":"2024-06-06T07:48:01.469765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:17.556327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498460+0000","last_change":"2024-06-06T07:43:57.219128+0000","last_active":"2024-06-06T07:48:00.498460+0000","last_peered":"2024-06-06T07:48:00.498460+0000","last_clean":"2024-06-06T07:48:00.498460+0000","last_became_active":"2024-06-06T07:43:57.218776+0000","last_became_peered":"2024-06-06T07:43:57.218776+0000","last_unstale":"2024-06-06T07:48:00.498460+0000","last_undegraded":"2024-06-06T07:48:00.498460+0000","last_fullsized":"2024-06-06T07:48:00.498460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:40.421403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488528+0000","last_change":"2024-06-06T07:44:00.334150+0000","last_active":"2024-06-06T07:48:00.488528+0000","last_peered":"2024-06-06T07:48:00.488528+0000","last_clean":"2024-06-06T07:48:00.488528+0000","last_became_active":"2024-06-06T07:44:00.333831+0000","last_became_peered":"2024-06-06T07:44:00.333831+0000","last_unstale":"2024-06-06T07:48:00.488528+0000","last_undegraded":"2024-06-06T07:48:00.488528+0000","last_fullsized":"2024-06-06T07:48:00.488528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:14:08.547313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491001+0000","last_change":"2024-06-06T07:44:11.396427+0000","last_active":"2024-06-06T07:48:00.491001+0000","last_peered":"2024-06-06T07:48:00.491001+0000","last_clean":"2024-06-06T07:48:00.491001+0000","last_became_active":"2024-06-06T07:44:11.396036+0000","last_became_peered":"2024-06-06T07:44:11.396036+0000","last_unstale":"2024-06-06T07:48:00.491001+0000","last_undegraded":"2024-06-06T07:48:00.491001+0000","last_fullsized":"2024-06-06T07:48:00.491001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:36:39.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":248,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.530915+0000","last_change":"2024-06-06T07:43:57.216758+0000","last_active":"2024-06-06T07:48:19.530915+0000","last_peered":"2024-06-06T07:48:19.530915+0000","last_clean":"2024-06-06T07:48:19.530915+0000","last_became_active":"2024-06-06T07:43:57.216354+0000","last_became_peered":"2024-06-06T07:43:57.216354+0000","last_unstale":"2024-06-06T07:48:19.530915+0000","last_undegraded":"2024-06-06T07:48:19.530915+0000","last_fullsized":"2024-06-06T07:48:19.530915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:24:34.867363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":197,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582202+0000","last_change":"2024-06-06T07:44:00.335538+0000","last_active":"2024-06-06T07:48:00.582202+0000","last_peered":"2024-06-06T07:48:00.582202+0000","last_clean":"2024-06-06T07:48:00.582202+0000","last_became_active":"2024-06-06T07:44:00.335148+0000","last_became_peered":"2024-06-06T07:44:00.335148+0000","last_unstale":"2024-06-06T07:48:00.582202+0000","last_undegraded":"2024-06-06T07:48:00.582202+0000","last_fullsized":"2024-06-06T07:48:00.582202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:42:53.205141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490900+0000","last_change":"2024-06-06T07:44:48.202386+0000","last_active":"2024-06-06T07:48:00.490900+0000","last_peered":"2024-06-06T07:48:00.490900+0000","last_clean":"2024-06-06T07:48:00.490900+0000","last_became_active":"2024-06-06T07:44:48.201757+0000","last_became_peered":"2024-06-06T07:44:48.201757+0000","last_unstale":"2024-06-06T07:48:00.490900+0000","last_undegraded":"2024-06-06T07:48:00.490900+0000","last_fullsized":"2024-06-06T07:48:00.490900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:56:10.287509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395950+0000","last_change":"2024-06-06T07:43:57.220240+0000","last_active":"2024-06-06T07:48:00.395950+0000","last_peered":"2024-06-06T07:48:00.395950+0000","last_clean":"2024-06-06T07:48:00.395950+0000","last_became_active":"2024-06-06T07:43:57.219827+0000","last_became_peered":"2024-06-06T07:43:57.219827+0000","last_unstale":"2024-06-06T07:48:00.395950+0000","last_undegraded":"2024-06-06T07:48:00.395950+0000","last_fullsized":"2024-06-06T07:48:00.395950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:39.737611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469751+0000","last_change":"2024-06-06T07:44:00.347239+0000","last_active":"2024-06-06T07:48:01.469751+0000","last_peered":"2024-06-06T07:48:01.469751+0000","last_clean":"2024-06-06T07:48:01.469751+0000","last_became_active":"2024-06-06T07:44:00.346928+0000","last_became_peered":"2024-06-06T07:44:00.346928+0000","last_unstale":"2024-06-06T07:48:01.469751+0000","last_undegraded":"2024-06-06T07:48:01.469751+0000","last_fullsized":"2024-06-06T07:48:01.469751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:09:36.295108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554775+0000","last_change":"2024-06-06T07:44:11.388542+0000","last_active":"2024-06-06T07:48:00.554775+0000","last_peered":"2024-06-06T07:48:00.554775+0000","last_clean":"2024-06-06T07:48:00.554775+0000","last_became_active":"2024-06-06T07:44:11.388105+0000","last_became_peered":"2024-06-06T07:44:11.388105+0000","last_unstale":"2024-06-06T07:48:00.554775+0000","last_undegraded":"2024-06-06T07:48:00.554775+0000","last_fullsized":"2024-06-06T07:48:00.554775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:16.877681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498871+0000","last_change":"2024-06-06T07:43:57.227344+0000","last_active":"2024-06-06T07:48:00.498871+0000","last_peered":"2024-06-06T07:48:00.498871+0000","last_clean":"2024-06-06T07:48:00.498871+0000","last_became_active":"2024-06-06T07:43:57.226921+0000","last_became_peered":"2024-06-06T07:43:57.226921+0000","last_unstale":"2024-06-06T07:48:00.498871+0000","last_undegraded":"2024-06-06T07:48:00.498871+0000","last_fullsized":"2024-06-06T07:48:00.498871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:48:14.068447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581996+0000","last_change":"2024-06-06T07:44:47.261019+0000","last_active":"2024-06-06T07:48:00.581996+0000","last_peered":"2024-06-06T07:48:00.581996+0000","last_clean":"2024-06-06T07:48:00.581996+0000","last_became_active":"2024-06-06T07:44:47.260639+0000","last_became_peered":"2024-06-06T07:44:47.260639+0000","last_unstale":"2024-06-06T07:48:00.581996+0000","last_undegraded":"2024-06-06T07:48:00.581996+0000","last_fullsized":"2024-06-06T07:48:00.581996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:52.888789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488486+0000","last_change":"2024-06-06T07:44:11.398440+0000","last_active":"2024-06-06T07:48:00.488486+0000","last_peered":"2024-06-06T07:48:00.488486+0000","last_clean":"2024-06-06T07:48:00.488486+0000","last_became_active":"2024-06-06T07:44:11.398077+0000","last_became_peered":"2024-06-06T07:44:11.398077+0000","last_unstale":"2024-06-06T07:48:00.488486+0000","last_undegraded":"2024-06-06T07:48:00.488486+0000","last_fullsized":"2024-06-06T07:48:00.488486+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:38:41.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":497,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:20.532578+0000","last_change":"2024-06-06T07:43:57.217834+0000","last_active":"2024-06-06T07:48:20.532578+0000","last_peered":"2024-06-06T07:48:20.532578+0000","last_clean":"2024-06-06T07:48:20.532578+0000","last_became_active":"2024-06-06T07:43:57.217493+0000","last_became_peered":"2024-06-06T07:43:57.217493+0000","last_unstale":"2024-06-06T07:48:20.532578+0000","last_undegraded":"2024-06-06T07:48:20.532578+0000","last_fullsized":"2024-06-06T07:48:20.532578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:57:40.532251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":408,"num_read_kb":275,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491357+0000","last_change":"2024-06-06T07:44:00.339601+0000","last_active":"2024-06-06T07:48:00.491357+0000","last_peered":"2024-06-06T07:48:00.491357+0000","last_clean":"2024-06-06T07:48:00.491357+0000","last_became_active":"2024-06-06T07:44:00.339139+0000","last_became_peered":"2024-06-06T07:44:00.339139+0000","last_unstale":"2024-06-06T07:48:00.491357+0000","last_undegraded":"2024-06-06T07:48:00.491357+0000","last_fullsized":"2024-06-06T07:48:00.491357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:47.231514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501627+0000","last_change":"2024-06-06T07:45:48.233044+0000","last_active":"2024-06-06T07:48:00.501627+0000","last_peered":"2024-06-06T07:48:00.501627+0000","last_clean":"2024-06-06T07:48:00.501627+0000","last_became_active":"2024-06-06T07:45:48.232191+0000","last_became_peered":"2024-06-06T07:45:48.232191+0000","last_unstale":"2024-06-06T07:48:00.501627+0000","last_undegraded":"2024-06-06T07:48:00.501627+0000","last_fullsized":"2024-06-06T07:48:00.501627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.963487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392777+0000","last_change":"2024-06-06T07:43:57.218188+0000","last_active":"2024-06-06T07:48:00.392777+0000","last_peered":"2024-06-06T07:48:00.392777+0000","last_clean":"2024-06-06T07:48:00.392777+0000","last_became_active":"2024-06-06T07:43:57.217808+0000","last_became_peered":"2024-06-06T07:43:57.217808+0000","last_unstale":"2024-06-06T07:48:00.392777+0000","last_undegraded":"2024-06-06T07:48:00.392777+0000","last_fullsized":"2024-06-06T07:48:00.392777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:29.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393638+0000","last_change":"2024-06-06T07:44:00.337234+0000","last_active":"2024-06-06T07:48:00.393638+0000","last_peered":"2024-06-06T07:48:00.393638+0000","last_clean":"2024-06-06T07:48:00.393638+0000","last_became_active":"2024-06-06T07:44:00.336955+0000","last_became_peered":"2024-06-06T07:44:00.336955+0000","last_unstale":"2024-06-06T07:48:00.393638+0000","last_undegraded":"2024-06-06T07:48:00.393638+0000","last_fullsized":"2024-06-06T07:48:00.393638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:12.357049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469762+0000","last_change":"2024-06-06T07:44:11.400349+0000","last_active":"2024-06-06T07:48:01.469762+0000","last_peered":"2024-06-06T07:48:01.469762+0000","last_clean":"2024-06-06T07:48:01.469762+0000","last_became_active":"2024-06-06T07:44:11.399841+0000","last_became_peered":"2024-06-06T07:44:11.399841+0000","last_unstale":"2024-06-06T07:48:01.469762+0000","last_undegraded":"2024-06-06T07:48:01.469762+0000","last_fullsized":"2024-06-06T07:48:01.469762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:21.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498649+0000","last_change":"2024-06-06T07:43:57.216628+0000","last_active":"2024-06-06T07:48:00.498649+0000","last_peered":"2024-06-06T07:48:00.498649+0000","last_clean":"2024-06-06T07:48:00.498649+0000","last_became_active":"2024-06-06T07:43:57.216252+0000","last_became_peered":"2024-06-06T07:43:57.216252+0000","last_unstale":"2024-06-06T07:48:00.498649+0000","last_undegraded":"2024-06-06T07:48:00.498649+0000","last_fullsized":"2024-06-06T07:48:00.498649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:26:09.668755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395623+0000","last_change":"2024-06-06T07:44:00.347404+0000","last_active":"2024-06-06T07:48:00.395623+0000","last_peered":"2024-06-06T07:48:00.395623+0000","last_clean":"2024-06-06T07:48:00.395623+0000","last_became_active":"2024-06-06T07:44:00.347043+0000","last_became_peered":"2024-06-06T07:44:00.347043+0000","last_unstale":"2024-06-06T07:48:00.395623+0000","last_undegraded":"2024-06-06T07:48:00.395623+0000","last_fullsized":"2024-06-06T07:48:00.395623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:28:30.178220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489334+0000","last_change":"2024-06-06T07:44:11.403896+0000","last_active":"2024-06-06T07:48:00.489334+0000","last_peered":"2024-06-06T07:48:00.489334+0000","last_clean":"2024-06-06T07:48:00.489334+0000","last_became_active":"2024-06-06T07:44:11.403480+0000","last_became_peered":"2024-06-06T07:44:11.403480+0000","last_unstale":"2024-06-06T07:48:00.489334+0000","last_undegraded":"2024-06-06T07:48:00.489334+0000","last_fullsized":"2024-06-06T07:48:00.489334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:51:54.860070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":256,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.585881+0000","last_change":"2024-06-06T07:43:57.218296+0000","last_active":"2024-06-06T07:48:19.585881+0000","last_peered":"2024-06-06T07:48:19.585881+0000","last_clean":"2024-06-06T07:48:19.585881+0000","last_became_active":"2024-06-06T07:43:57.217915+0000","last_became_peered":"2024-06-06T07:43:57.217915+0000","last_unstale":"2024-06-06T07:48:19.585881+0000","last_undegraded":"2024-06-06T07:48:19.585881+0000","last_fullsized":"2024-06-06T07:48:19.585881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:23:57.690682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":204,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395631+0000","last_change":"2024-06-06T07:44:47.253158+0000","last_active":"2024-06-06T07:48:00.395631+0000","last_peered":"2024-06-06T07:48:00.395631+0000","last_clean":"2024-06-06T07:48:00.395631+0000","last_became_active":"2024-06-06T07:44:47.252721+0000","last_became_peered":"2024-06-06T07:44:47.252721+0000","last_unstale":"2024-06-06T07:48:00.395631+0000","last_undegraded":"2024-06-06T07:48:00.395631+0000","last_fullsized":"2024-06-06T07:48:00.395631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:35.495969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394202+0000","last_change":"2024-06-06T07:44:11.397005+0000","last_active":"2024-06-06T07:48:00.394202+0000","last_peered":"2024-06-06T07:48:00.394202+0000","last_clean":"2024-06-06T07:48:00.394202+0000","last_became_active":"2024-06-06T07:44:11.396601+0000","last_became_peered":"2024-06-06T07:44:11.396601+0000","last_unstale":"2024-06-06T07:48:00.394202+0000","last_undegraded":"2024-06-06T07:48:00.394202+0000","last_fullsized":"2024-06-06T07:48:00.394202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:38:36.760300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469638+0000","last_change":"2024-06-06T07:45:48.248446+0000","last_active":"2024-06-06T07:48:01.469638+0000","last_peered":"2024-06-06T07:48:01.469638+0000","last_clean":"2024-06-06T07:48:01.469638+0000","last_became_active":"2024-06-06T07:45:48.231530+0000","last_became_peered":"2024-06-06T07:45:48.231530+0000","last_unstale":"2024-06-06T07:48:01.469638+0000","last_undegraded":"2024-06-06T07:48:01.469638+0000","last_fullsized":"2024-06-06T07:48:01.469638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:11:11.671652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498602+0000","last_change":"2024-06-06T07:44:00.335108+0000","last_active":"2024-06-06T07:48:00.498602+0000","last_peered":"2024-06-06T07:48:00.498602+0000","last_clean":"2024-06-06T07:48:00.498602+0000","last_became_active":"2024-06-06T07:44:00.334720+0000","last_became_peered":"2024-06-06T07:44:00.334720+0000","last_unstale":"2024-06-06T07:48:00.498602+0000","last_undegraded":"2024-06-06T07:48:00.498602+0000","last_fullsized":"2024-06-06T07:48:00.498602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:08:43.047861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498515+0000","last_change":"2024-06-06T07:44:11.400218+0000","last_active":"2024-06-06T07:48:00.498515+0000","last_peered":"2024-06-06T07:48:00.498515+0000","last_clean":"2024-06-06T07:48:00.498515+0000","last_became_active":"2024-06-06T07:44:11.399439+0000","last_became_peered":"2024-06-06T07:44:11.399439+0000","last_unstale":"2024-06-06T07:48:00.498515+0000","last_undegraded":"2024-06-06T07:48:00.498515+0000","last_fullsized":"2024-06-06T07:48:00.498515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:34:17.425428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469871+0000","last_change":"2024-06-06T07:45:48.252101+0000","last_active":"2024-06-06T07:48:01.469871+0000","last_peered":"2024-06-06T07:48:01.469871+0000","last_clean":"2024-06-06T07:48:01.469871+0000","last_became_active":"2024-06-06T07:45:48.234253+0000","last_became_peered":"2024-06-06T07:45:48.234253+0000","last_unstale":"2024-06-06T07:48:01.469871+0000","last_undegraded":"2024-06-06T07:48:01.469871+0000","last_fullsized":"2024-06-06T07:48:01.469871+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:47:43.488642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469888+0000","last_change":"2024-06-06T07:44:00.342313+0000","last_active":"2024-06-06T07:48:01.469888+0000","last_peered":"2024-06-06T07:48:01.469888+0000","last_clean":"2024-06-06T07:48:01.469888+0000","last_became_active":"2024-06-06T07:44:00.341668+0000","last_became_peered":"2024-06-06T07:44:00.341668+0000","last_unstale":"2024-06-06T07:48:01.469888+0000","last_undegraded":"2024-06-06T07:48:01.469888+0000","last_fullsized":"2024-06-06T07:48:01.469888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:54.547668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490977+0000","last_change":"2024-06-06T07:44:11.463538+0000","last_active":"2024-06-06T07:48:00.490977+0000","last_peered":"2024-06-06T07:48:00.490977+0000","last_clean":"2024-06-06T07:48:00.490977+0000","last_became_active":"2024-06-06T07:44:11.462754+0000","last_became_peered":"2024-06-06T07:44:11.462754+0000","last_unstale":"2024-06-06T07:48:00.490977+0000","last_undegraded":"2024-06-06T07:48:00.490977+0000","last_fullsized":"2024-06-06T07:48:00.490977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:04:48.829785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.488843+0000","last_change":"2024-06-06T07:43:57.225379+0000","last_active":"2024-06-06T07:48:00.488843+0000","last_peered":"2024-06-06T07:48:00.488843+0000","last_clean":"2024-06-06T07:48:00.488843+0000","last_became_active":"2024-06-06T07:43:57.225050+0000","last_became_peered":"2024-06-06T07:43:57.225050+0000","last_unstale":"2024-06-06T07:48:00.488843+0000","last_undegraded":"2024-06-06T07:48:00.488843+0000","last_fullsized":"2024-06-06T07:48:00.488843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:01:51.988591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501744+0000","last_change":"2024-06-06T07:44:00.338485+0000","last_active":"2024-06-06T07:48:00.501744+0000","last_peered":"2024-06-06T07:48:00.501744+0000","last_clean":"2024-06-06T07:48:00.501744+0000","last_became_active":"2024-06-06T07:44:00.338126+0000","last_became_peered":"2024-06-06T07:44:00.338126+0000","last_unstale":"2024-06-06T07:48:00.501744+0000","last_undegraded":"2024-06-06T07:48:00.501744+0000","last_fullsized":"2024-06-06T07:48:00.501744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:53.861508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394098+0000","last_change":"2024-06-06T07:44:11.401699+0000","last_active":"2024-06-06T07:48:00.394098+0000","last_peered":"2024-06-06T07:48:00.394098+0000","last_clean":"2024-06-06T07:48:00.394098+0000","last_became_active":"2024-06-06T07:44:11.401310+0000","last_became_peered":"2024-06-06T07:44:11.401310+0000","last_unstale":"2024-06-06T07:48:00.394098+0000","last_undegraded":"2024-06-06T07:48:00.394098+0000","last_fullsized":"2024-06-06T07:48:00.394098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:47:18.592401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395632+0000","last_change":"2024-06-06T07:43:57.219305+0000","last_active":"2024-06-06T07:48:00.395632+0000","last_peered":"2024-06-06T07:48:00.395632+0000","last_clean":"2024-06-06T07:48:00.395632+0000","last_became_active":"2024-06-06T07:43:57.218874+0000","last_became_peered":"2024-06-06T07:43:57.218874+0000","last_unstale":"2024-06-06T07:48:00.395632+0000","last_undegraded":"2024-06-06T07:48:00.395632+0000","last_fullsized":"2024-06-06T07:48:00.395632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:36:14.186324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.501561+0000","last_change":"2024-06-06T07:44:00.347802+0000","last_active":"2024-06-06T07:48:00.501561+0000","last_peered":"2024-06-06T07:48:00.501561+0000","last_clean":"2024-06-06T07:48:00.501561+0000","last_became_active":"2024-06-06T07:44:00.347436+0000","last_became_peered":"2024-06-06T07:44:00.347436+0000","last_unstale":"2024-06-06T07:48:00.501561+0000","last_undegraded":"2024-06-06T07:48:00.501561+0000","last_fullsized":"2024-06-06T07:48:00.501561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:37.918585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393723+0000","last_change":"2024-06-06T07:44:11.463052+0000","last_active":"2024-06-06T07:48:00.393723+0000","last_peered":"2024-06-06T07:48:00.393723+0000","last_clean":"2024-06-06T07:48:00.393723+0000","last_became_active":"2024-06-06T07:44:11.462663+0000","last_became_peered":"2024-06-06T07:44:11.462663+0000","last_unstale":"2024-06-06T07:48:00.393723+0000","last_undegraded":"2024-06-06T07:48:00.393723+0000","last_fullsized":"2024-06-06T07:48:00.393723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:08:59.541960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498755+0000","last_change":"2024-06-06T07:44:00.345750+0000","last_active":"2024-06-06T07:48:00.498755+0000","last_peered":"2024-06-06T07:48:00.498755+0000","last_clean":"2024-06-06T07:48:00.498755+0000","last_became_active":"2024-06-06T07:44:00.345375+0000","last_became_peered":"2024-06-06T07:44:00.345375+0000","last_unstale":"2024-06-06T07:48:00.498755+0000","last_undegraded":"2024-06-06T07:48:00.498755+0000","last_fullsized":"2024-06-06T07:48:00.498755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:14:30.396918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":251,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:19.584850+0000","last_change":"2024-06-06T07:43:57.225336+0000","last_active":"2024-06-06T07:48:19.584850+0000","last_peered":"2024-06-06T07:48:19.584850+0000","last_clean":"2024-06-06T07:48:19.584850+0000","last_became_active":"2024-06-06T07:43:57.224938+0000","last_became_peered":"2024-06-06T07:43:57.224938+0000","last_unstale":"2024-06-06T07:48:19.584850+0000","last_undegraded":"2024-06-06T07:48:19.584850+0000","last_fullsized":"2024-06-06T07:48:19.584850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:44:56.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":197,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395714+0000","last_change":"2024-06-06T07:44:47.255514+0000","last_active":"2024-06-06T07:48:00.395714+0000","last_peered":"2024-06-06T07:48:00.395714+0000","last_clean":"2024-06-06T07:48:00.395714+0000","last_became_active":"2024-06-06T07:44:47.255076+0000","last_became_peered":"2024-06-06T07:44:47.255076+0000","last_unstale":"2024-06-06T07:48:00.395714+0000","last_undegraded":"2024-06-06T07:48:00.395714+0000","last_fullsized":"2024-06-06T07:48:00.395714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:18:41.108519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491306+0000","last_change":"2024-06-06T07:44:00.330059+0000","last_active":"2024-06-06T07:48:00.491306+0000","last_peered":"2024-06-06T07:48:00.491306+0000","last_clean":"2024-06-06T07:48:00.491306+0000","last_became_active":"2024-06-06T07:44:00.329739+0000","last_became_peered":"2024-06-06T07:44:00.329739+0000","last_unstale":"2024-06-06T07:48:00.491306+0000","last_undegraded":"2024-06-06T07:48:00.491306+0000","last_fullsized":"2024-06-06T07:48:00.491306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:47.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554779+0000","last_change":"2024-06-06T07:43:57.216236+0000","last_active":"2024-06-06T07:48:00.554779+0000","last_peered":"2024-06-06T07:48:00.554779+0000","last_clean":"2024-06-06T07:48:00.554779+0000","last_became_active":"2024-06-06T07:43:57.215843+0000","last_became_peered":"2024-06-06T07:43:57.215843+0000","last_unstale":"2024-06-06T07:48:00.554779+0000","last_undegraded":"2024-06-06T07:48:00.554779+0000","last_fullsized":"2024-06-06T07:48:00.554779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:18:39.043688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":426,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:02.523875+0000","last_change":"2024-06-06T07:42:34.925814+0000","last_active":"2024-06-06T07:48:02.523875+0000","last_peered":"2024-06-06T07:48:02.523875+0000","last_clean":"2024-06-06T07:48:02.523875+0000","last_became_active":"2024-06-06T07:42:34.925294+0000","last_became_peered":"2024-06-06T07:42:34.925294+0000","last_unstale":"2024-06-06T07:48:02.523875+0000","last_undegraded":"2024-06-06T07:48:02.523875+0000","last_fullsized":"2024-06-06T07:48:02.523875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:39:11.095468+0000","last_clean_scrub_stamp":"2024-06-06T07:39:11.095468+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:23.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498926+0000","last_change":"2024-06-06T07:44:47.260890+0000","last_active":"2024-06-06T07:48:00.498926+0000","last_peered":"2024-06-06T07:48:00.498926+0000","last_clean":"2024-06-06T07:48:00.498926+0000","last_became_active":"2024-06-06T07:44:47.260363+0000","last_became_peered":"2024-06-06T07:44:47.260363+0000","last_unstale":"2024-06-06T07:48:00.498926+0000","last_undegraded":"2024-06-06T07:48:00.498926+0000","last_fullsized":"2024-06-06T07:48:00.498926+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:43:47.413168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393525+0000","last_change":"2024-06-06T07:44:00.342044+0000","last_active":"2024-06-06T07:48:00.393525+0000","last_peered":"2024-06-06T07:48:00.393525+0000","last_clean":"2024-06-06T07:48:00.393525+0000","last_became_active":"2024-06-06T07:44:00.341384+0000","last_became_peered":"2024-06-06T07:44:00.341384+0000","last_unstale":"2024-06-06T07:48:00.393525+0000","last_undegraded":"2024-06-06T07:48:00.393525+0000","last_fullsized":"2024-06-06T07:48:00.393525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:52:04.461921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395560+0000","last_change":"2024-06-06T07:43:57.219111+0000","last_active":"2024-06-06T07:48:00.395560+0000","last_peered":"2024-06-06T07:48:00.395560+0000","last_clean":"2024-06-06T07:48:00.395560+0000","last_became_active":"2024-06-06T07:43:57.218684+0000","last_became_peered":"2024-06-06T07:43:57.218684+0000","last_unstale":"2024-06-06T07:48:00.395560+0000","last_undegraded":"2024-06-06T07:48:00.395560+0000","last_fullsized":"2024-06-06T07:48:00.395560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:37:22.267737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469745+0000","last_change":"2024-06-06T07:44:11.398009+0000","last_active":"2024-06-06T07:48:01.469745+0000","last_peered":"2024-06-06T07:48:01.469745+0000","last_clean":"2024-06-06T07:48:01.469745+0000","last_became_active":"2024-06-06T07:44:11.397547+0000","last_became_peered":"2024-06-06T07:44:11.397547+0000","last_unstale":"2024-06-06T07:48:01.469745+0000","last_undegraded":"2024-06-06T07:48:01.469745+0000","last_fullsized":"2024-06-06T07:48:01.469745+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:00.367211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491397+0000","last_change":"2024-06-06T07:43:57.225293+0000","last_active":"2024-06-06T07:48:00.491397+0000","last_peered":"2024-06-06T07:48:00.491397+0000","last_clean":"2024-06-06T07:48:00.491397+0000","last_became_active":"2024-06-06T07:43:57.225055+0000","last_became_peered":"2024-06-06T07:43:57.225055+0000","last_unstale":"2024-06-06T07:48:00.491397+0000","last_undegraded":"2024-06-06T07:48:00.491397+0000","last_fullsized":"2024-06-06T07:48:00.491397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:36.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393994+0000","last_change":"2024-06-06T07:44:00.341085+0000","last_active":"2024-06-06T07:48:00.393994+0000","last_peered":"2024-06-06T07:48:00.393994+0000","last_clean":"2024-06-06T07:48:00.393994+0000","last_became_active":"2024-06-06T07:44:00.340742+0000","last_became_peered":"2024-06-06T07:44:00.340742+0000","last_unstale":"2024-06-06T07:48:00.393994+0000","last_undegraded":"2024-06-06T07:48:00.393994+0000","last_fullsized":"2024-06-06T07:48:00.393994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:05:03.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491724+0000","last_change":"2024-06-06T07:44:11.462801+0000","last_active":"2024-06-06T07:48:00.491724+0000","last_peered":"2024-06-06T07:48:00.491724+0000","last_clean":"2024-06-06T07:48:00.491724+0000","last_became_active":"2024-06-06T07:44:11.462370+0000","last_became_peered":"2024-06-06T07:44:11.462370+0000","last_unstale":"2024-06-06T07:48:00.491724+0000","last_undegraded":"2024-06-06T07:48:00.491724+0000","last_fullsized":"2024-06-06T07:48:00.491724+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:23.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.813155+0000","last_change":"2024-06-06T07:43:57.211993+0000","last_active":"2024-06-06T07:48:15.813155+0000","last_peered":"2024-06-06T07:48:15.813155+0000","last_clean":"2024-06-06T07:48:15.813155+0000","last_became_active":"2024-06-06T07:43:57.211614+0000","last_became_peered":"2024-06-06T07:43:57.211614+0000","last_unstale":"2024-06-06T07:48:15.813155+0000","last_undegraded":"2024-06-06T07:48:15.813155+0000","last_fullsized":"2024-06-06T07:48:15.813155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:37.282878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491229+0000","last_change":"2024-06-06T07:44:00.337359+0000","last_active":"2024-06-06T07:48:00.491229+0000","last_peered":"2024-06-06T07:48:00.491229+0000","last_clean":"2024-06-06T07:48:00.491229+0000","last_became_active":"2024-06-06T07:44:00.336613+0000","last_became_peered":"2024-06-06T07:44:00.336613+0000","last_unstale":"2024-06-06T07:48:00.491229+0000","last_undegraded":"2024-06-06T07:48:00.491229+0000","last_fullsized":"2024-06-06T07:48:00.491229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.073352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392527+0000","last_change":"2024-06-06T07:44:11.394055+0000","last_active":"2024-06-06T07:48:00.392527+0000","last_peered":"2024-06-06T07:48:00.392527+0000","last_clean":"2024-06-06T07:48:00.392527+0000","last_became_active":"2024-06-06T07:44:11.393476+0000","last_became_peered":"2024-06-06T07:44:11.393476+0000","last_unstale":"2024-06-06T07:48:00.392527+0000","last_undegraded":"2024-06-06T07:48:00.392527+0000","last_fullsized":"2024-06-06T07:48:00.392527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:26:50.582448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.555126+0000","last_change":"2024-06-06T07:43:57.531891+0000","last_active":"2024-06-06T07:48:00.555126+0000","last_peered":"2024-06-06T07:48:00.555126+0000","last_clean":"2024-06-06T07:48:00.555126+0000","last_became_active":"2024-06-06T07:43:57.531509+0000","last_became_peered":"2024-06-06T07:43:57.531509+0000","last_unstale":"2024-06-06T07:48:00.555126+0000","last_undegraded":"2024-06-06T07:48:00.555126+0000","last_fullsized":"2024-06-06T07:48:00.555126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:12.767171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469894+0000","last_change":"2024-06-06T07:44:48.201542+0000","last_active":"2024-06-06T07:48:01.469894+0000","last_peered":"2024-06-06T07:48:01.469894+0000","last_clean":"2024-06-06T07:48:01.469894+0000","last_became_active":"2024-06-06T07:44:48.201255+0000","last_became_peered":"2024-06-06T07:44:48.201255+0000","last_unstale":"2024-06-06T07:48:01.469894+0000","last_undegraded":"2024-06-06T07:48:01.469894+0000","last_fullsized":"2024-06-06T07:48:01.469894+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:28.975123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554966+0000","last_change":"2024-06-06T07:44:11.464107+0000","last_active":"2024-06-06T07:48:00.554966+0000","last_peered":"2024-06-06T07:48:00.554966+0000","last_clean":"2024-06-06T07:48:00.554966+0000","last_became_active":"2024-06-06T07:44:11.463315+0000","last_became_peered":"2024-06-06T07:44:11.463315+0000","last_unstale":"2024-06-06T07:48:00.554966+0000","last_undegraded":"2024-06-06T07:48:00.554966+0000","last_fullsized":"2024-06-06T07:48:00.554966+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:12.853250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581713+0000","last_change":"2024-06-06T07:43:57.227240+0000","last_active":"2024-06-06T07:48:00.581713+0000","last_peered":"2024-06-06T07:48:00.581713+0000","last_clean":"2024-06-06T07:48:00.581713+0000","last_became_active":"2024-06-06T07:43:57.226932+0000","last_became_peered":"2024-06-06T07:43:57.226932+0000","last_unstale":"2024-06-06T07:48:00.581713+0000","last_undegraded":"2024-06-06T07:48:00.581713+0000","last_fullsized":"2024-06-06T07:48:00.581713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:26.600831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.490987+0000","last_change":"2024-06-06T07:44:00.339546+0000","last_active":"2024-06-06T07:48:00.490987+0000","last_peered":"2024-06-06T07:48:00.490987+0000","last_clean":"2024-06-06T07:48:00.490987+0000","last_became_active":"2024-06-06T07:44:00.339182+0000","last_became_peered":"2024-06-06T07:44:00.339182+0000","last_unstale":"2024-06-06T07:48:00.490987+0000","last_undegraded":"2024-06-06T07:48:00.490987+0000","last_fullsized":"2024-06-06T07:48:00.490987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:02:20.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.489207+0000","last_change":"2024-06-06T07:44:11.403766+0000","last_active":"2024-06-06T07:48:00.489207+0000","last_peered":"2024-06-06T07:48:00.489207+0000","last_clean":"2024-06-06T07:48:00.489207+0000","last_became_active":"2024-06-06T07:44:11.403268+0000","last_became_peered":"2024-06-06T07:44:11.403268+0000","last_unstale":"2024-06-06T07:48:00.489207+0000","last_undegraded":"2024-06-06T07:48:00.489207+0000","last_fullsized":"2024-06-06T07:48:00.489207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:32.120023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491084+0000","last_change":"2024-06-06T07:43:57.215884+0000","last_active":"2024-06-06T07:48:00.491084+0000","last_peered":"2024-06-06T07:48:00.491084+0000","last_clean":"2024-06-06T07:48:00.491084+0000","last_became_active":"2024-06-06T07:43:57.215619+0000","last_became_peered":"2024-06-06T07:43:57.215619+0000","last_unstale":"2024-06-06T07:48:00.491084+0000","last_undegraded":"2024-06-06T07:48:00.491084+0000","last_fullsized":"2024-06-06T07:48:00.491084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.062322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469761+0000","last_change":"2024-06-06T07:44:00.341038+0000","last_active":"2024-06-06T07:48:01.469761+0000","last_peered":"2024-06-06T07:48:01.469761+0000","last_clean":"2024-06-06T07:48:01.469761+0000","last_became_active":"2024-06-06T07:44:00.340661+0000","last_became_peered":"2024-06-06T07:44:00.340661+0000","last_unstale":"2024-06-06T07:48:01.469761+0000","last_undegraded":"2024-06-06T07:48:01.469761+0000","last_fullsized":"2024-06-06T07:48:01.469761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.925068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554950+0000","last_change":"2024-06-06T07:44:11.391402+0000","last_active":"2024-06-06T07:48:00.554950+0000","last_peered":"2024-06-06T07:48:00.554950+0000","last_clean":"2024-06-06T07:48:00.554950+0000","last_became_active":"2024-06-06T07:44:11.390950+0000","last_became_peered":"2024-06-06T07:44:11.390950+0000","last_unstale":"2024-06-06T07:48:00.554950+0000","last_undegraded":"2024-06-06T07:48:00.554950+0000","last_fullsized":"2024-06-06T07:48:00.554950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:35.140194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.243949+0000","last_change":"2024-06-06T07:43:57.211901+0000","last_active":"2024-06-06T07:48:01.243949+0000","last_peered":"2024-06-06T07:48:01.243949+0000","last_clean":"2024-06-06T07:48:01.243949+0000","last_became_active":"2024-06-06T07:43:57.211557+0000","last_became_peered":"2024-06-06T07:43:57.211557+0000","last_unstale":"2024-06-06T07:48:01.243949+0000","last_undegraded":"2024-06-06T07:48:01.243949+0000","last_fullsized":"2024-06-06T07:48:01.243949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:23:29.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491092+0000","last_change":"2024-06-06T07:44:00.339514+0000","last_active":"2024-06-06T07:48:00.491092+0000","last_peered":"2024-06-06T07:48:00.491092+0000","last_clean":"2024-06-06T07:48:00.491092+0000","last_became_active":"2024-06-06T07:44:00.339178+0000","last_became_peered":"2024-06-06T07:44:00.339178+0000","last_unstale":"2024-06-06T07:48:00.491092+0000","last_undegraded":"2024-06-06T07:48:00.491092+0000","last_fullsized":"2024-06-06T07:48:00.491092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:10:56.367646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.554910+0000","last_change":"2024-06-06T07:44:11.463085+0000","last_active":"2024-06-06T07:48:00.554910+0000","last_peered":"2024-06-06T07:48:00.554910+0000","last_clean":"2024-06-06T07:48:00.554910+0000","last_became_active":"2024-06-06T07:44:11.462622+0000","last_became_peered":"2024-06-06T07:44:11.462622+0000","last_unstale":"2024-06-06T07:48:00.554910+0000","last_undegraded":"2024-06-06T07:48:00.554910+0000","last_fullsized":"2024-06-06T07:48:00.554910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:22.179536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491042+0000","last_change":"2024-06-06T07:43:57.217311+0000","last_active":"2024-06-06T07:48:00.491042+0000","last_peered":"2024-06-06T07:48:00.491042+0000","last_clean":"2024-06-06T07:48:00.491042+0000","last_became_active":"2024-06-06T07:43:57.216899+0000","last_became_peered":"2024-06-06T07:43:57.216899+0000","last_unstale":"2024-06-06T07:48:00.491042+0000","last_undegraded":"2024-06-06T07:48:00.491042+0000","last_fullsized":"2024-06-06T07:48:00.491042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:43:32.138037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395681+0000","last_change":"2024-06-06T07:44:00.345343+0000","last_active":"2024-06-06T07:48:00.395681+0000","last_peered":"2024-06-06T07:48:00.395681+0000","last_clean":"2024-06-06T07:48:00.395681+0000","last_became_active":"2024-06-06T07:44:00.344412+0000","last_became_peered":"2024-06-06T07:44:00.344412+0000","last_unstale":"2024-06-06T07:48:00.395681+0000","last_undegraded":"2024-06-06T07:48:00.395681+0000","last_fullsized":"2024-06-06T07:48:00.395681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:29:04.484935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581689+0000","last_change":"2024-06-06T07:44:11.398090+0000","last_active":"2024-06-06T07:48:00.581689+0000","last_peered":"2024-06-06T07:48:00.581689+0000","last_clean":"2024-06-06T07:48:00.581689+0000","last_became_active":"2024-06-06T07:44:11.397419+0000","last_became_peered":"2024-06-06T07:44:11.397419+0000","last_unstale":"2024-06-06T07:48:00.581689+0000","last_undegraded":"2024-06-06T07:48:00.581689+0000","last_fullsized":"2024-06-06T07:48:00.581689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:49:47.805685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.581922+0000","last_change":"2024-06-06T07:45:48.236752+0000","last_active":"2024-06-06T07:48:00.581922+0000","last_peered":"2024-06-06T07:48:00.581922+0000","last_clean":"2024-06-06T07:48:00.581922+0000","last_became_active":"2024-06-06T07:45:48.231049+0000","last_became_peered":"2024-06-06T07:45:48.231049+0000","last_unstale":"2024-06-06T07:48:00.581922+0000","last_undegraded":"2024-06-06T07:48:00.581922+0000","last_fullsized":"2024-06-06T07:48:00.581922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:16.337218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395917+0000","last_change":"2024-06-06T07:44:00.344931+0000","last_active":"2024-06-06T07:48:00.395917+0000","last_peered":"2024-06-06T07:48:00.395917+0000","last_clean":"2024-06-06T07:48:00.395917+0000","last_became_active":"2024-06-06T07:44:00.343285+0000","last_became_peered":"2024-06-06T07:44:00.343285+0000","last_unstale":"2024-06-06T07:48:00.395917+0000","last_undegraded":"2024-06-06T07:48:00.395917+0000","last_fullsized":"2024-06-06T07:48:00.395917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:44:34.747398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469487+0000","last_change":"2024-06-06T07:44:11.398054+0000","last_active":"2024-06-06T07:48:01.469487+0000","last_peered":"2024-06-06T07:48:01.469487+0000","last_clean":"2024-06-06T07:48:01.469487+0000","last_became_active":"2024-06-06T07:44:11.397513+0000","last_became_peered":"2024-06-06T07:44:11.397513+0000","last_unstale":"2024-06-06T07:48:01.469487+0000","last_undegraded":"2024-06-06T07:48:01.469487+0000","last_fullsized":"2024-06-06T07:48:01.469487+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:58.608321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:15.456155+0000","last_change":"2024-06-06T07:43:57.216426+0000","last_active":"2024-06-06T07:48:15.456155+0000","last_peered":"2024-06-06T07:48:15.456155+0000","last_clean":"2024-06-06T07:48:15.456155+0000","last_became_active":"2024-06-06T07:43:57.216019+0000","last_became_peered":"2024-06-06T07:43:57.216019+0000","last_unstale":"2024-06-06T07:48:15.456155+0000","last_undegraded":"2024-06-06T07:48:15.456155+0000","last_fullsized":"2024-06-06T07:48:15.456155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:16:11.663187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.395589+0000","last_change":"2024-06-06T07:44:00.336349+0000","last_active":"2024-06-06T07:48:00.395589+0000","last_peered":"2024-06-06T07:48:00.395589+0000","last_clean":"2024-06-06T07:48:00.395589+0000","last_became_active":"2024-06-06T07:44:00.335960+0000","last_became_peered":"2024-06-06T07:44:00.335960+0000","last_unstale":"2024-06-06T07:48:00.395589+0000","last_undegraded":"2024-06-06T07:48:00.395589+0000","last_fullsized":"2024-06-06T07:48:00.395589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:31.125378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498315+0000","last_change":"2024-06-06T07:44:11.394903+0000","last_active":"2024-06-06T07:48:00.498315+0000","last_peered":"2024-06-06T07:48:00.498315+0000","last_clean":"2024-06-06T07:48:00.498315+0000","last_became_active":"2024-06-06T07:44:11.394399+0000","last_became_peered":"2024-06-06T07:44:11.394399+0000","last_unstale":"2024-06-06T07:48:00.498315+0000","last_undegraded":"2024-06-06T07:48:00.498315+0000","last_fullsized":"2024-06-06T07:48:00.498315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:44:10.368159+0000","last_clean_scrub_stamp":"2024-06-06T07:44:10.368159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:07:14.635022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.394100+0000","last_change":"2024-06-06T07:45:48.235157+0000","last_active":"2024-06-06T07:48:00.394100+0000","last_peered":"2024-06-06T07:48:00.394100+0000","last_clean":"2024-06-06T07:48:00.394100+0000","last_became_active":"2024-06-06T07:45:48.232061+0000","last_became_peered":"2024-06-06T07:45:48.232061+0000","last_unstale":"2024-06-06T07:48:00.394100+0000","last_undegraded":"2024-06-06T07:48:00.394100+0000","last_fullsized":"2024-06-06T07:48:00.394100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:56.190423+0000","last_clean_scrub_stamp":"2024-06-06T07:43:56.190423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:01:22.102198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.396165+0000","last_change":"2024-06-06T07:44:00.340260+0000","last_active":"2024-06-06T07:48:00.396165+0000","last_peered":"2024-06-06T07:48:00.396165+0000","last_clean":"2024-06-06T07:48:00.396165+0000","last_became_active":"2024-06-06T07:44:00.339909+0000","last_became_peered":"2024-06-06T07:44:00.339909+0000","last_unstale":"2024-06-06T07:48:00.396165+0000","last_undegraded":"2024-06-06T07:48:00.396165+0000","last_fullsized":"2024-06-06T07:48:00.396165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:04:53.251668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469838+0000","last_change":"2024-06-06T07:44:00.342018+0000","last_active":"2024-06-06T07:48:01.469838+0000","last_peered":"2024-06-06T07:48:01.469838+0000","last_clean":"2024-06-06T07:48:01.469838+0000","last_became_active":"2024-06-06T07:44:00.341550+0000","last_became_peered":"2024-06-06T07:44:00.341550+0000","last_unstale":"2024-06-06T07:48:01.469838+0000","last_undegraded":"2024-06-06T07:48:01.469838+0000","last_fullsized":"2024-06-06T07:48:01.469838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:28:17.021958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392803+0000","last_change":"2024-06-06T07:44:00.347729+0000","last_active":"2024-06-06T07:48:00.392803+0000","last_peered":"2024-06-06T07:48:00.392803+0000","last_clean":"2024-06-06T07:48:00.392803+0000","last_became_active":"2024-06-06T07:44:00.347373+0000","last_became_peered":"2024-06-06T07:44:00.347373+0000","last_unstale":"2024-06-06T07:48:00.392803+0000","last_undegraded":"2024-06-06T07:48:00.392803+0000","last_fullsized":"2024-06-06T07:48:00.392803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:55.687025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.392560+0000","last_change":"2024-06-06T07:44:00.346058+0000","last_active":"2024-06-06T07:48:00.392560+0000","last_peered":"2024-06-06T07:48:00.392560+0000","last_clean":"2024-06-06T07:48:00.392560+0000","last_became_active":"2024-06-06T07:44:00.343496+0000","last_became_peered":"2024-06-06T07:44:00.343496+0000","last_unstale":"2024-06-06T07:48:00.392560+0000","last_undegraded":"2024-06-06T07:48:00.392560+0000","last_fullsized":"2024-06-06T07:48:00.392560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:34:49.113992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.469841+0000","last_change":"2024-06-06T07:44:00.347277+0000","last_active":"2024-06-06T07:48:01.469841+0000","last_peered":"2024-06-06T07:48:01.469841+0000","last_clean":"2024-06-06T07:48:01.469841+0000","last_became_active":"2024-06-06T07:44:00.346953+0000","last_became_peered":"2024-06-06T07:44:00.346953+0000","last_unstale":"2024-06-06T07:48:01.469841+0000","last_undegraded":"2024-06-06T07:48:01.469841+0000","last_fullsized":"2024-06-06T07:48:01.469841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:55.330260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393816+0000","last_change":"2024-06-06T07:44:00.341203+0000","last_active":"2024-06-06T07:48:00.393816+0000","last_peered":"2024-06-06T07:48:00.393816+0000","last_clean":"2024-06-06T07:48:00.393816+0000","last_became_active":"2024-06-06T07:44:00.340876+0000","last_became_peered":"2024-06-06T07:44:00.340876+0000","last_unstale":"2024-06-06T07:48:00.393816+0000","last_undegraded":"2024-06-06T07:48:00.393816+0000","last_fullsized":"2024-06-06T07:48:00.393816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:47:59.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470081+0000","last_change":"2024-06-06T07:44:47.257583+0000","last_active":"2024-06-06T07:48:01.470081+0000","last_peered":"2024-06-06T07:48:01.470081+0000","last_clean":"2024-06-06T07:48:01.470081+0000","last_became_active":"2024-06-06T07:44:47.257195+0000","last_became_peered":"2024-06-06T07:44:47.257195+0000","last_unstale":"2024-06-06T07:48:01.470081+0000","last_undegraded":"2024-06-06T07:48:01.470081+0000","last_fullsized":"2024-06-06T07:48:01.470081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:02.726604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393137+0000","last_change":"2024-06-06T07:44:00.350400+0000","last_active":"2024-06-06T07:48:00.393137+0000","last_peered":"2024-06-06T07:48:00.393137+0000","last_clean":"2024-06-06T07:48:00.393137+0000","last_became_active":"2024-06-06T07:44:00.350023+0000","last_became_peered":"2024-06-06T07:44:00.350023+0000","last_unstale":"2024-06-06T07:48:00.393137+0000","last_undegraded":"2024-06-06T07:48:00.393137+0000","last_fullsized":"2024-06-06T07:48:00.393137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:13.329814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491436+0000","last_change":"2024-06-06T07:44:00.342574+0000","last_active":"2024-06-06T07:48:00.491436+0000","last_peered":"2024-06-06T07:48:00.491436+0000","last_clean":"2024-06-06T07:48:00.491436+0000","last_became_active":"2024-06-06T07:44:00.342233+0000","last_became_peered":"2024-06-06T07:44:00.342233+0000","last_unstale":"2024-06-06T07:48:00.491436+0000","last_undegraded":"2024-06-06T07:48:00.491436+0000","last_fullsized":"2024-06-06T07:48:00.491436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:09:49.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.498512+0000","last_change":"2024-06-06T07:44:00.333323+0000","last_active":"2024-06-06T07:48:00.498512+0000","last_peered":"2024-06-06T07:48:00.498512+0000","last_clean":"2024-06-06T07:48:00.498512+0000","last_became_active":"2024-06-06T07:44:00.332409+0000","last_became_peered":"2024-06-06T07:44:00.332409+0000","last_unstale":"2024-06-06T07:48:00.498512+0000","last_undegraded":"2024-06-06T07:48:00.498512+0000","last_fullsized":"2024-06-06T07:48:00.498512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:10.666128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.491106+0000","last_change":"2024-06-06T07:44:00.343222+0000","last_active":"2024-06-06T07:48:00.491106+0000","last_peered":"2024-06-06T07:48:00.491106+0000","last_clean":"2024-06-06T07:48:00.491106+0000","last_became_active":"2024-06-06T07:44:00.342833+0000","last_became_peered":"2024-06-06T07:44:00.342833+0000","last_unstale":"2024-06-06T07:48:00.491106+0000","last_undegraded":"2024-06-06T07:48:00.491106+0000","last_fullsized":"2024-06-06T07:48:00.491106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:17:36.991901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.582402+0000","last_change":"2024-06-06T07:44:00.337380+0000","last_active":"2024-06-06T07:48:00.582402+0000","last_peered":"2024-06-06T07:48:00.582402+0000","last_clean":"2024-06-06T07:48:00.582402+0000","last_became_active":"2024-06-06T07:44:00.336973+0000","last_became_peered":"2024-06-06T07:44:00.336973+0000","last_unstale":"2024-06-06T07:48:00.582402+0000","last_undegraded":"2024-06-06T07:48:00.582402+0000","last_fullsized":"2024-06-06T07:48:00.582402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:33:07.346183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:00.393817+0000","last_change":"2024-06-06T07:44:00.337317+0000","last_active":"2024-06-06T07:48:00.393817+0000","last_peered":"2024-06-06T07:48:00.393817+0000","last_clean":"2024-06-06T07:48:00.393817+0000","last_became_active":"2024-06-06T07:44:00.336936+0000","last_became_peered":"2024-06-06T07:44:00.336936+0000","last_unstale":"2024-06-06T07:48:00.393817+0000","last_undegraded":"2024-06-06T07:48:00.393817+0000","last_fullsized":"2024-06-06T07:48:00.393817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:54.640986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T07:48:01.470014+0000","last_change":"2024-06-06T07:44:00.341994+0000","last_active":"2024-06-06T07:48:01.470014+0000","last_peered":"2024-06-06T07:48:01.470014+0000","last_clean":"2024-06-06T07:48:01.470014+0000","last_became_active":"2024-06-06T07:44:00.341344+0000","last_became_peered":"2024-06-06T07:44:00.341344+0000","last_unstale":"2024-06-06T07:48:01.470014+0000","last_undegraded":"2024-06-06T07:48:01.470014+0000","last_fullsized":"2024-06-06T07:48:01.470014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T07:43:59.315725+0000","last_clean_scrub_stamp":"2024-06-06T07:43:59.315725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:07.197173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799288,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1564,"num_read_kb":1001,"num_write":433,"num_write_kb":2369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2397864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383748,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":580137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645385,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29276,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715876,"statfs":{"total":95995035648,"available":95965057024,"internally_reserved":0,"allocated":2191360,"data_stored":1572518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":973455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":973241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332183,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712248,"statfs":{"total":95995035648,"available":95961341952,"internally_reserved":0,"allocated":1765376,"data_stored":1178217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495709,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1150976,"data_stored":575415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":647597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":576803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40716,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704436,"statfs":{"total":95995035648,"available":95953342464,"internally_reserved":0,"allocated":1249280,"data_stored":649094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37300,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707852,"statfs":{"total":95995035648,"available":95956840448,"internally_reserved":0,"allocated":1888256,"data_stored":1297057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":581294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37512,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707640,"statfs":{"total":95995035648,"available":95956623360,"internally_reserved":0,"allocated":2105344,"data_stored":1500808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:48:26.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:27.084 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:27.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:27 smithi087 ceph-mon[32276]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:27.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:27 smithi181 ceph-mon[36790]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:27.411 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:27 smithi195 ceph-mon[37040]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:48:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:28 smithi181 ceph-mon[36790]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T07:48:28.353 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:28 smithi087 ceph-mon[32276]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T07:48:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:28 smithi195 ceph-mon[37040]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3435755937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101620 msgr2=0x7f7d68101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3435755937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101620 0x7f7d68101a00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099c0 tx=0x7f7d5c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3435755937 shutdown_connections 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3435755937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d6810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3435755937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101620 0x7f7d68101a00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3435755937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101fd0 0x7f7d68105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3435755937 >> 172.21.15.87:0/3435755937 conn(0x7f7d68078340 msgr2=0x7f7d68078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3435755937 shutdown_connections 2024-06-06T07:48:28.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.916+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3435755937 wait complete. 2024-06-06T07:48:28.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 Processor -- start 2024-06-06T07:48:28.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 -- start start 2024-06-06T07:48:28.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 0x7f7d6819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d68198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d681116b0 con 0x7f7d68101fd0 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68111530 con 0x7f7d681066b0 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d6d38b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68111830 con 0x7f7d68101620 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d667fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 0x7f7d6819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:33898/0 (socket says 172.21.15.87:33898) 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.917+0000 7f7d66ffd700 1 -- 172.21.15.87:0/3207411394 learned_addr learned my addr 172.21.15.87:0/3207411394 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d677fe700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d68198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 -- 172.21.15.87:0/3207411394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 msgr2=0x7f7d6819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:28.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 -- 172.21.15.87:0/3207411394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 msgr2=0x7f7d68198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d68198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 -- 172.21.15.87:0/3207411394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d68199220 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d677fe700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d68198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d66ffd700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d667fc700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 0x7f7d6819ee40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9c0 tx=0x7f7d5800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cbd0 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d68199510 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d58004500 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58005040 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.918+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d681a57d0 con 0x7f7d68101fd0 2024-06-06T07:48:28.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.919+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d68069180 con 0x7f7d68101fd0 2024-06-06T07:48:28.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.921+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f7d58005340 con 0x7f7d68101fd0 2024-06-06T07:48:28.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.921+0000 7f7d4ffff700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 0x7f7d500a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:28.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.921+0000 7f7d66ffd700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 0x7f7d500a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:28.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.921+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7d580cf900 con 0x7f7d68101fd0 2024-06-06T07:48:28.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.922+0000 7f7d66ffd700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 0x7f7d500a53f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009990 tx=0x7f7d5c005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:28.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:28.926+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d5809bfd0 con 0x7f7d68101fd0 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- 172.21.15.87:0/2298499858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 msgr2=0x7fb09c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 --2- 172.21.15.87:0/2298499858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c100d50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb084009930 tx=0x7fb08402a420 comp rx=0 tx=0).stop 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- 172.21.15.87:0/2298499858 shutdown_connections 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 --2- 172.21.15.87:0/2298499858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 0x7fb09c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 --2- 172.21.15.87:0/2298499858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c100d50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 --2- 172.21.15.87:0/2298499858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 0x7fb09c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- 172.21.15.87:0/2298499858 >> 172.21.15.87:0/2298499858 conn(0x7fb09c077a20 msgr2=0x7fb09c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- 172.21.15.87:0/2298499858 shutdown_connections 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- 172.21.15.87:0/2298499858 wait complete. 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 Processor -- start 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.066+0000 7fb0a131b640 1 -- start start 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 0x7fb09c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 0x7fb09c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c110ca0 con 0x7fb09c1008d0 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c110b20 con 0x7fb09c101290 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c110e20 con 0x7fb09c106890 2024-06-06T07:48:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58506/0 (socket says 172.21.15.87:58506) 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 -- 172.21.15.87:0/3178470946 learned_addr learned my addr 172.21.15.87:0/3178470946 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 0x7fb09c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb093fff640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 0x7fb09c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 -- 172.21.15.87:0/3178470946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 msgr2=0x7fb09c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 0x7fb09c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 -- 172.21.15.87:0/3178470946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 msgr2=0x7fb09c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 0x7fb09c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 -- 172.21.15.87:0/3178470946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c19d240 con 0x7fb09c1008d0 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb09affd640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d770 tx=0x7fb08c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb098ff9640 1 -- 172.21.15.87:0/3178470946 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c017070 con 0x7fb09c1008d0 2024-06-06T07:48:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c19d530 con 0x7fb09c1008d0 2024-06-06T07:48:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb098ff9640 1 -- 172.21.15.87:0/3178470946 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c005500 con 0x7fb09c1008d0 2024-06-06T07:48:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb098ff9640 1 -- 172.21.15.87:0/3178470946 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c004e00 con 0x7fb09c1008d0 2024-06-06T07:48:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.067+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c1a4dc0 con 0x7fb09c1008d0 2024-06-06T07:48:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.070+0000 7fb098ff9640 1 -- 172.21.15.87:0/3178470946 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fb08c004fa0 con 0x7fb09c1008d0 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.070+0000 7fb0a402f740 1 Processor -- start 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 -- start start 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151170 0x55aad8151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55aad80b54f0 con 0x55aad8151a90 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aad8053870 con 0x55aad8151170 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb0a402f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aad80b14b0 con 0x55aad815c350 2024-06-06T07:48:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58520/0 (socket says 172.21.15.87:58520) 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 -- 172.21.15.87:0/4186397823 learned_addr learned my addr 172.21.15.87:0/4186397823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb098ff9640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 0x7fb0880a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb093fff640 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151170 0x55aad8151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 0x7fb0880a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb098ff9640 1 -- 172.21.15.87:0/3178470946 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb08c0cf4a0 con 0x7fb09c1008d0 2024-06-06T07:48:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 -- 172.21.15.87:0/4186397823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 msgr2=0x55aad815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 -- 172.21.15.87:0/4186397823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151170 msgr2=0x55aad8151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151170 0x55aad8151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.071+0000 7fb09affd640 1 -- 172.21.15.87:0/4186397823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad806f9b0 con 0x55aad8151a90 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.072+0000 7fb09affd640 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cc730 tx=0x7fb08c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f83e4ca5a358d14 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.072+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 0x7fb0880a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb094005c40 tx=0x7fb094009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.072+0000 7fb0917fa640 1 -- 172.21.15.87:0/4186397823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c020070 con 0x55aad8151a90 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.072+0000 7fb0917fa640 1 -- 172.21.15.87:0/4186397823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c0046c0 con 0x55aad8151a90 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.072+0000 7fb0917fa640 1 -- 172.21.15.87:0/4186397823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0a4010 con 0x55aad8151a90 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- 172.21.15.87:0/4186397823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 msgr2=0x55aad815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cc730 tx=0x7fb08c009c30 comp rx=0 tx=0).stop 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- 172.21.15.87:0/4186397823 shutdown_connections 2024-06-06T07:48:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151a90 0x55aad815be10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- 172.21.15.87:0/4186397823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151170 0x55aad8151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- 172.21.15.87:0/4186397823 >> 172.21.15.87:0/4186397823 conn(0x55aad814d680 msgr2=0x55aad814da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- 172.21.15.87:0/4186397823 shutdown_connections 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- 172.21.15.87:0/4186397823 wait complete. 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- start start 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 Processor -- start 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151170 0x55aad8159370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55aad815ef20 con 0x55aad815c350 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aad803eae0 con 0x55aad8151a90 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aad815f0a0 con 0x55aad8151170 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb0a402f740 10 client.0 _ll_register_callbacks cb 0x55aad80779b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.073+0000 7fb09b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58532/0 (socket says 172.21.15.87:58532) 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 -- 172.21.15.87:0/1230169129 learned_addr learned my addr 172.21.15.87:0/1230169129 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb093fff640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09affd640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151170 0x55aad8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 -- 172.21.15.87:0/1230169129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151170 msgr2=0x55aad8159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151170 0x55aad8159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 -- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 msgr2=0x55aad81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 -- 172.21.15.87:0/1230169129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad7dc9210 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb094011e60 tx=0x7fb094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f7f85471c3bf69 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0827fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09402d060 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb0940047c0 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aad80c8f10 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb094034050 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0a402f740 10 client.15489 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aad7e95620 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.074+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aad80c8f10 con 0x55aad815c350 2024-06-06T07:48:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb094027030 con 0x55aad815c350 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 20 client.15489 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad816c5b0 0x55aad816e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 4 client.15489 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad80c2c20 con 0x55aad816c5b0 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb09affd640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad816c5b0 0x55aad816e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb0827fc640 10 client.15489 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.075+0000 7fb09affd640 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad816c5b0 0x55aad816e9a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=149b0056c5faf4f0 server_cookie=2eb5da200f302076 in_seq=0 out_seq=0 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0827fc640 1 -- 172.21.15.87:0/1230169129 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aad80c2c20 con 0x55aad816c5b0 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0827fc640 10 client.15489 handle_command_reply: tid=0 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 client.15489 shutdown 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 msgr2=0x55aad8159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb094011e60 tx=0x7fb094004a40 comp rx=0 tx=0).stop 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 shutdown_connections 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad816c5b0 0x55aad816e9a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad815c350 0x55aad8159df0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0827fc640 0 client.15489 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1230169129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151170 0x55aad8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 >> 172.21.15.87:0/1230169129 conn(0x55aad814d680 msgr2=0x55aad8153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 shutdown_connections 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 1 -- 172.21.15.87:0/1230169129 wait complete. 2024-06-06T07:48:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.076+0000 7fb0a402f740 20 client.15489 trim_cache size 0 max 16384 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 Processor -- start 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 -- start start 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151a90 0x55aad81747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad815c350 0x55aad8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b17c0 con 0x55aad8151170 2024-06-06T07:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1af090 con 0x55aad815c350 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.091+0000 7fb0a402f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1acee0 con 0x55aad8151a90 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58538/0 (socket says 172.21.15.87:58538) 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 -- 172.21.15.87:0/1205205952 learned_addr learned my addr 172.21.15.87:0/1205205952 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb093fff640 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151a90 0x55aad81747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad815c350 0x55aad8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 -- 172.21.15.87:0/1205205952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151a90 msgr2=0x55aad81747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151a90 0x55aad81747d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 -- 172.21.15.87:0/1205205952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad815c350 msgr2=0x55aad8152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad815c350 0x55aad8152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 -- 172.21.15.87:0/1205205952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad816b150 con 0x55aad8151170 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb09affd640 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cce20 tx=0x7fb08c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff2c9e61981d027 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb090ff9640 1 -- 172.21.15.87:0/1205205952 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c020040 con 0x55aad8151170 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb090ff9640 1 -- 172.21.15.87:0/1205205952 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c09b3a0 con 0x55aad8151170 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.092+0000 7fb090ff9640 1 -- 172.21.15.87:0/1205205952 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c097a90 con 0x55aad8151170 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- 172.21.15.87:0/1205205952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 msgr2=0x55aad8174290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cce20 tx=0x7fb08c0bb040 comp rx=0 tx=0).stop 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- 172.21.15.87:0/1205205952 shutdown_connections 2024-06-06T07:48:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad815c350 0x55aad8152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad8151a90 0x55aad81747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 --2- 172.21.15.87:0/1205205952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad8174290 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- 172.21.15.87:0/1205205952 >> 172.21.15.87:0/1205205952 conn(0x55aad814d680 msgr2=0x55aad81545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- 172.21.15.87:0/1205205952 shutdown_connections 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- 172.21.15.87:0/1205205952 wait complete. 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 -- start start 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.093+0000 7fb0a402f740 1 Processor -- start 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad8160250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ad550 con 0x55aad8151170 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ae9c0 con 0x55aad8151a90 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1adc20 con 0x55aad815c350 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 10 client.0 _ll_register_callbacks cb 0x7fb094011d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58548/0 (socket says 172.21.15.87:58548) 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 -- 172.21.15.87:0/2364384016 learned_addr learned my addr 172.21.15.87:0/2364384016 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad8160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb093fff640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 -- 172.21.15.87:0/2364384016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 msgr2=0x55aad8160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad8160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 -- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 msgr2=0x55aad815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad815fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 -- 172.21.15.87:0/2364384016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad8160a50 con 0x55aad8151170 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09b7fe640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad8160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb09affd640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb08c097800 tx=0x7fb08c09bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa18b15eee0311c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0827fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c020070 con 0x55aad8151170 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb08c097a90 con 0x55aad8151170 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.094+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aad80bcf70 con 0x55aad8151170 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb08c0a4010 con 0x55aad8151170 2024-06-06T07:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aad81d71c0 con 0x55aad8151170 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aad80bcf70 con 0x55aad8151170 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb08c0b41c0 con 0x55aad8151170 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 20 client.15495 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad81d7390 0x55aad81d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 4 client.15495 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aad803a8e0 con 0x55aad81d7390 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.095+0000 7fb093fff640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad81d7390 0x55aad81d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.096+0000 7fb0827fc640 10 client.15495 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.096+0000 7fb093fff640 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad81d7390 0x55aad81d9780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3bde740c257135 server_cookie=b7967c9e1347a92d in_seq=0 out_seq=0 2024-06-06T07:48:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0827fc640 1 -- 172.21.15.87:0/2364384016 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aad803a8e0 con 0x55aad81d7390 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0827fc640 10 client.15495 handle_command_reply: tid=0 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 client.15495 shutdown 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 msgr2=0x55aad81649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb08c097800 tx=0x7fb08c09bb30 comp rx=0 tx=0).stop 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 shutdown_connections 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55aad81d7390 0x55aad81d9780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aad815c350 0x55aad8160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0827fc640 0 client.15495 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad8151a90 0x55aad815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 --2- 172.21.15.87:0/2364384016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aad8151170 0x55aad81649a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 >> 172.21.15.87:0/2364384016 conn(0x55aad814d680 msgr2=0x55aad8152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 shutdown_connections 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 1 -- 172.21.15.87:0/2364384016 wait complete. 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.108+0000 7fb0a402f740 20 client.15495 trim_cache size 0 max 16384 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:48:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:48:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 msgr2=0x7fb0880a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 0x7fb0880a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb094005c40 tx=0x7fb094009500 comp rx=0 tx=0).stop 2024-06-06T07:48:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 msgr2=0x7fb09c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d770 tx=0x7fb08c00dc30 comp rx=0 tx=0).stop 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 shutdown_connections 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0880a2ec0 0x7fb0880a5380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09c106890 0x7fb09c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb09c101290 0x7fb09c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 --2- 172.21.15.87:0/3178470946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb09c1008d0 0x7fb09c198250 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 >> 172.21.15.87:0/3178470946 conn(0x7fb09c077a20 msgr2=0x7fb09c069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 shutdown_connections 2024-06-06T07:48:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:29.109+0000 7fb0a131b640 1 -- 172.21.15.87:0/3178470946 wait complete. 2024-06-06T07:48:29.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.179+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6802dca0 con 0x7f7d68101fd0 2024-06-06T07:48:29.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.180+0000 7f7d4ffff700 1 -- 172.21.15.87:0/3207411394 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f7d5809b8d0 con 0x7f7d68101fd0 2024-06-06T07:48:29.180 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:29.181 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:29.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 msgr2=0x7f7d500a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 0x7f7d500a53f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009990 tx=0x7f7d5c005960 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 msgr2=0x7f7d6819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 0x7f7d6819ee40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9c0 tx=0x7f7d5800ecd0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 shutdown_connections 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d681066b0 0x7f7d68198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7d500a2f30 0x7f7d500a53f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68101620 0x7f7d6819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.182+0000 7f7d6d38b700 1 --2- 172.21.15.87:0/3207411394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d68101fd0 0x7f7d6819ee40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.183+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 >> 172.21.15.87:0/3207411394 conn(0x7f7d68078340 msgr2=0x7f7d6806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.183+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 shutdown_connections 2024-06-06T07:48:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:29.183+0000 7f7d6d38b700 1 -- 172.21.15.87:0/3207411394 wait complete. 2024-06-06T07:48:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:29.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:30.222 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:30 smithi181 ceph-mon[36790]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-06-06T07:48:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:30 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3207411394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:30.343 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:30 smithi087 ceph-mon[32276]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-06-06T07:48:30.344 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:30 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3207411394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:30 smithi195 ceph-mon[37040]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-06-06T07:48:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:30 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3207411394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.981+0000 7f7fbe905700 1 -- 172.21.15.87:0/854804136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb81030f0 msgr2=0x7f7fb8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:31.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.981+0000 7f7fbe905700 1 --2- 172.21.15.87:0/854804136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb81030f0 0x7f7fb8107110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0099c0 tx=0x7f7fac009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 -- 172.21.15.87:0/854804136 shutdown_connections 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 --2- 172.21.15.87:0/854804136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8107840 0x7f7fb810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 --2- 172.21.15.87:0/854804136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8102740 0x7f7fb8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 --2- 172.21.15.87:0/854804136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb81030f0 0x7f7fb8107110 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 -- 172.21.15.87:0/854804136 >> 172.21.15.87:0/854804136 conn(0x7f7fb80795f0 msgr2=0x7f7fb8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 -- 172.21.15.87:0/854804136 shutdown_connections 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 -- 172.21.15.87:0/854804136 wait complete. 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 Processor -- start 2024-06-06T07:48:31.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.982+0000 7f7fbe905700 1 -- start start 2024-06-06T07:48:31.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb81030f0 0x7f7fb819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8107840 0x7f7fb8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb8112770 con 0x7f7fb8102740 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81125f0 con 0x7f7fb81030f0 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbe905700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81128f0 con 0x7f7fb8107840 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58564/0 (socket says 172.21.15.87:58564) 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 -- 172.21.15.87:0/1943677562 learned_addr learned my addr 172.21.15.87:0/1943677562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb77fe700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb81030f0 0x7f7fb819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fbcea2700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8107840 0x7f7fb8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 -- 172.21.15.87:0/1943677562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8107840 msgr2=0x7f7fb8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8107840 0x7f7fb8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 -- 172.21.15.87:0/1943677562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb81030f0 msgr2=0x7f7fb819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb81030f0 0x7f7fb819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 -- 172.21.15.87:0/1943677562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb81991b0 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb7fff700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9c0 tx=0x7f7fa800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.983+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800cbd0 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.984+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa800cd70 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.984+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8010430 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.984+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb81994a0 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.984+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb81a5710 con 0x7f7fb8102740 2024-06-06T07:48:31.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.985+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb8069180 con 0x7f7fb8102740 2024-06-06T07:48:31.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.986+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f7fa8004750 con 0x7f7fb8102740 2024-06-06T07:48:31.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.986+0000 7f7fb57fa700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 0x7f7fa00a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:31.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.986+0000 7f7fb77fe700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 0x7f7fa00a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:31.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.986+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7fa80cfd10 con 0x7f7fb8102740 2024-06-06T07:48:31.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.987+0000 7f7fb77fe700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 0x7f7fa00a53f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8102050 tx=0x7f7fac005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:31.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:31.991+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7fa809c310 con 0x7f7fb8102740 2024-06-06T07:48:32.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.230+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fb8199f30 con 0x7f7fb8102740 2024-06-06T07:48:32.231 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:32 smithi087 ceph-mon[32276]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:48:32.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.231+0000 7f7fb57fa700 1 -- 172.21.15.87:0/1943677562 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f7fa809bc10 con 0x7f7fb8102740 2024-06-06T07:48:32.231 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:32.231 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:32.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 msgr2=0x7f7fa00a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:32.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 0x7f7fa00a53f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8102050 tx=0x7f7fac005fb0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 msgr2=0x7f7fb819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:32.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9c0 tx=0x7f7fa800ecd0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 shutdown_connections 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fa00a2f30 0x7f7fa00a53f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fb8107840 0x7f7fb8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fb8102740 0x7f7fb819e840 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 --2- 172.21.15.87:0/1943677562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb81030f0 0x7f7fb819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 >> 172.21.15.87:0/1943677562 conn(0x7f7fb80795f0 msgr2=0x7f7fb8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 shutdown_connections 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:32.233+0000 7f7fbe905700 1 -- 172.21.15.87:0/1943677562 wait complete. 2024-06-06T07:48:32.234 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:32 smithi181 ceph-mon[36790]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:48:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:32 smithi195 ceph-mon[37040]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:48:32.954 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:48:33.195 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:33 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1943677562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:33.317 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:33 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1943677562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:33.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:33 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1943677562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:34 smithi181 ceph-mon[36790]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:34 smithi087 ceph-mon[32276]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:34 smithi195 ceph-mon[37040]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:34.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 -- 172.21.15.87:0/3711882687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf501030f0 msgr2=0x7fdf50107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:34.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 --2- 172.21.15.87:0/3711882687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf501030f0 0x7fdf50107110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdf440099c0 tx=0x7fdf44009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:34.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 -- 172.21.15.87:0/3711882687 shutdown_connections 2024-06-06T07:48:34.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 --2- 172.21.15.87:0/3711882687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50102740 0x7fdf50102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:34.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 --2- 172.21.15.87:0/3711882687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50107840 0x7fdf5010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:34.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 --2- 172.21.15.87:0/3711882687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf501030f0 0x7fdf50107110 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:34.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 -- 172.21.15.87:0/3711882687 >> 172.21.15.87:0/3711882687 conn(0x7fdf500795f0 msgr2=0x7fdf50079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:34.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 -- 172.21.15.87:0/3711882687 shutdown_connections 2024-06-06T07:48:34.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 -- 172.21.15.87:0/3711882687 wait complete. 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.996+0000 7fdf56126700 1 Processor -- start 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 -- start start 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 0x7fdf5019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 0x7fdf5019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50112770 con 0x7fdf50107840 2024-06-06T07:48:34.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501125f0 con 0x7fdf501030f0 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf56126700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501128f0 con 0x7fdf50102740 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf4ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf4ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58576/0 (socket says 172.21.15.87:58576) 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf4ffff700 1 -- 172.21.15.87:0/2468725794 learned_addr learned my addr 172.21.15.87:0/2468725794 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf4f7fe700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 0x7fdf5019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.997+0000 7fdf4effd700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 0x7fdf5019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 -- 172.21.15.87:0/2468725794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 msgr2=0x7fdf5019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 0x7fdf5019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 -- 172.21.15.87:0/2468725794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 msgr2=0x7fdf5019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 0x7fdf5019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 -- 172.21.15.87:0/2468725794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf501990d0 con 0x7fdf50107840 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4effd700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 0x7fdf5019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:35.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4f7fe700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 0x7fdf5019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4ffff700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800ada0 tx=0x7fdf4800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48017020 con 0x7fdf50107840 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501993c0 con 0x7fdf50107840 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf48004bb0 con 0x7fdf50107840 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48005520 con 0x7fdf50107840 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.998+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501a5680 con 0x7fdf50107840 2024-06-06T07:48:35.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:34.999+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf50069180 con 0x7fdf50107840 2024-06-06T07:48:35.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.000+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fdf480056c0 con 0x7fdf50107840 2024-06-06T07:48:35.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.001+0000 7fdf4cff9700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 0x7fdf380a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:35.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.001+0000 7fdf4f7fe700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 0x7fdf380a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:35.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.001+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fdf480cf220 con 0x7fdf50107840 2024-06-06T07:48:35.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.001+0000 7fdf4f7fe700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 0x7fdf380a5370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf50199ab0 tx=0x7fdf4000a680 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:35.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.005+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf48021020 con 0x7fdf50107840 2024-06-06T07:48:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.204+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf50199ea0 con 0x7fdf50107840 2024-06-06T07:48:35.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.206+0000 7fdf4cff9700 1 -- 172.21.15.87:0/2468725794 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25294 (secure 0 0 0) 0x7fdf4809b260 con 0x7fdf50107840 2024-06-06T07:48:35.207 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:35.207 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":100,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T07:48:00.385938+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/3344462507":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/207935511":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/4065374776":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/2153987725":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:48:35.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 msgr2=0x7fdf380a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 0x7fdf380a5370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf50199ab0 tx=0x7fdf4000a680 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 msgr2=0x7fdf50198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800ada0 tx=0x7fdf4800c330 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 shutdown_connections 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdf380a2eb0 0x7fdf380a5370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf50107840 0x7fdf50198970 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50102740 0x7fdf5019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 --2- 172.21.15.87:0/2468725794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf501030f0 0x7fdf5019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.208+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 >> 172.21.15.87:0/2468725794 conn(0x7fdf500795f0 msgr2=0x7fdf50100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.209+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 shutdown_connections 2024-06-06T07:48:35.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:35.209+0000 7fdf56126700 1 -- 172.21.15.87:0/2468725794 wait complete. 2024-06-06T07:48:35.903 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-06T07:48:36.146 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:36.267 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:36 smithi087 ceph-mon[32276]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:36.268 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:36 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2468725794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:36 smithi181 ceph-mon[36790]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:36 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2468725794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:36 smithi195 ceph-mon[37040]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:36 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2468725794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:48:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:38 smithi181 ceph-mon[36790]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:38 smithi087 ceph-mon[32276]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:38.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:38 smithi195 ceph-mon[37040]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:48:38.804 INFO:teuthology.orchestra.run.smithi087.stdout:{"run_dir":"/var/run/ceph"} 2024-06-06T07:48:38.804 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-06T07:48:38.804 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5050 2024-06-06T07:48:38.805 DEBUG:tasks.quiescer:Yielding 2024-06-06T07:48:38.805 INFO:teuthology.run_tasks:Running task workunit... 2024-06-06T07:48:38.824 INFO:tasks.workunit:Pulling workunits from ref 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T07:48:38.825 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-06T07:48:38.825 INFO:tasks.workunit:timeout=3h 2024-06-06T07:48:38.825 INFO:tasks.workunit:cleanup=True 2024-06-06T07:48:38.825 DEBUG:teuthology.orchestra.run.smithi087:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-06T07:48:38.826 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-06-06 07:45:20.005547288 +0000 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-06-06 07:45:20.005547288 +0000 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-06-06 07:47:57.627098815 +0000 2024-06-06T07:48:38.843 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-06-06 07:45:20.005547288 +0000 2024-06-06T07:48:38.843 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-06T07:48:38.844 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-06T07:48:38.916 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T07:48:38.973 INFO:tasks.workunit.client.0.smithi087.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-06T07:48:39.074 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:40 smithi087 ceph-mon[32276]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:40 smithi195 ceph-mon[37040]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:40 smithi181 ceph-mon[36790]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 -- 172.21.15.87:0/1174984098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 msgr2=0x7fadf810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 --2- 172.21.15.87:0/1174984098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf810f330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b0a0 tx=0x7fadf000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 -- 172.21.15.87:0/1174984098 shutdown_connections 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 --2- 172.21.15.87:0/1174984098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf810f330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 --2- 172.21.15.87:0/1174984098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8108510 0x7fadf81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 --2- 172.21.15.87:0/1174984098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8100a80 0x7fadf8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 -- 172.21.15.87:0/1174984098 >> 172.21.15.87:0/1174984098 conn(0x7fadf8077a00 msgr2=0x7fadf80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 -- 172.21.15.87:0/1174984098 shutdown_connections 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.949+0000 7fadfecd2700 1 -- 172.21.15.87:0/1174984098 wait complete. 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 Processor -- start 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 -- start start 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8100a80 0x7fadf819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8108510 0x7fadf8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fadf8112770 con 0x7fadf8101440 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81125f0 con 0x7fadf8108510 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfecd2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81128f0 con 0x7fadf8100a80 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadf7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadf7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38394/0 (socket says 172.21.15.87:38394) 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadf7fff700 1 -- 172.21.15.87:0/2979155263 learned_addr learned my addr 172.21.15.87:0/2979155263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfd26f700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8108510 0x7fadf8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.950+0000 7fadfca6e700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8100a80 0x7fadf819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 -- 172.21.15.87:0/2979155263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8100a80 msgr2=0x7fadf819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8100a80 0x7fadf819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 -- 172.21.15.87:0/2979155263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8108510 msgr2=0x7fadf8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8108510 0x7fadf8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 -- 172.21.15.87:0/2979155263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf81990d0 con 0x7fadf8101440 2024-06-06T07:48:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf7fff700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fade400d950 tx=0x7fade400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4016070 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade400fd70 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf81993c0 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade400e610 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.951+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf8101b90 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.952+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf8199ea0 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.953+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fade4019070 con 0x7fadf8101440 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.954+0000 7fadf5ffb700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 0x7fade80a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.954+0000 7fadfca6e700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 0x7fade80a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.954+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fade40cf4e0 con 0x7fadf8101440 2024-06-06T07:48:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.954+0000 7fadfca6e700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 0x7fade80a5370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fadf8199ab0 tx=0x7fadec0073d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:40.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:40.958+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fade409bb80 con 0x7fadf8101440 2024-06-06T07:48:41.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.199+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadf802de90 con 0x7fadf8101440 2024-06-06T07:48:41.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.200+0000 7fadf5ffb700 1 -- 172.21.15.87:0/2979155263 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7fade409b480 con 0x7fadf8101440 2024-06-06T07:48:41.200 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:41.200 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:41.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 msgr2=0x7fade80a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:41.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 0x7fade80a5370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fadf8199ab0 tx=0x7fadec0073d0 comp rx=0 tx=0).stop 2024-06-06T07:48:41.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 msgr2=0x7fadf819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:41.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fade400d950 tx=0x7fade400dc60 comp rx=0 tx=0).stop 2024-06-06T07:48:41.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 shutdown_connections 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8108510 0x7fadf8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fade80a2eb0 0x7fade80a5370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8100a80 0x7fadf819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 --2- 172.21.15.87:0/2979155263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadf8101440 0x7fadf819ecf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 >> 172.21.15.87:0/2979155263 conn(0x7fadf8077a00 msgr2=0x7fadf80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 shutdown_connections 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:41.202+0000 7fadfecd2700 1 -- 172.21.15.87:0/2979155263 wait complete. 2024-06-06T07:48:41.203 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:42.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T07:44:02.139989+0000', 'modified': '2024-06-06T07:47:58.307983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24467, 'mds_1': 24448, 'mds_2': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/922595027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 922595027}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 922595027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/1343084853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1343084853}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 1343084853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/494555841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 494555841}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 494555841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.181:6837/3077028235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3077028235}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3077028235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.195:6839/203076359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 203076359}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 203076359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/851837058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 851837058}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 851837058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24467, 'qdb_cluster': [24448, 24467, 24446]} 2024-06-06T07:48:42.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T07:48:42.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T07:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.199+0000 7f306f6f5640 1 -- 172.21.15.87:0/1907975144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 msgr2=0x7f3068107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.199+0000 7f306f6f5640 1 --2- 172.21.15.87:0/1907975144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f3068107d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3060009cd0 tx=0x7f306002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- 172.21.15.87:0/1907975144 shutdown_connections 2024-06-06T07:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- 172.21.15.87:0/1907975144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 0x7f306810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- 172.21.15.87:0/1907975144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 0x7f3068101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- 172.21.15.87:0/1907975144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f3068107d80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- 172.21.15.87:0/1907975144 >> 172.21.15.87:0/1907975144 conn(0x7f30680fd460 msgr2=0x7f30680ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- 172.21.15.87:0/1907975144 shutdown_connections 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- 172.21.15.87:0/1907975144 wait complete. 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 Processor -- start 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- start start 2024-06-06T07:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 0x7f3068198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 0x7f3068198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3068111db0 con 0x7f30681079a0 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3068111c30 con 0x7f3068101990 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.200+0000 7f306f6f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3068111f30 con 0x7f3068102350 2024-06-06T07:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38422/0 (socket says 172.21.15.87:38422) 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 -- 172.21.15.87:0/2336795257 learned_addr learned my addr 172.21.15.87:0/2336795257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306d491640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 0x7f3068198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306cc90640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 0x7f3068198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 -- 172.21.15.87:0/2336795257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 msgr2=0x7f3068198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 0x7f3068198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 -- 172.21.15.87:0/2336795257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 msgr2=0x7f3068198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 0x7f3068198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 -- 172.21.15.87:0/2336795257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306819d1d0 con 0x7f30681079a0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306dc92640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f306400d750 tx=0x7f306400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f305e7fc640 1 -- 172.21.15.87:0/2336795257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064017070 con 0x7f30681079a0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f305e7fc640 1 -- 172.21.15.87:0/2336795257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064005500 con 0x7f30681079a0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306819d4c0 con 0x7f30681079a0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f305e7fc640 1 -- 172.21.15.87:0/2336795257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064004d90 con 0x7f30681079a0 2024-06-06T07:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.201+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681a4d50 con 0x7f30681079a0 2024-06-06T07:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.203+0000 7f305e7fc640 1 -- 172.21.15.87:0/2336795257 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f3064004f30 con 0x7f30681079a0 2024-06-06T07:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.204+0000 7f305e7fc640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 0x7f30540a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.204+0000 7f305e7fc640 1 -- 172.21.15.87:0/2336795257 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f30640cf2f0 con 0x7f30681079a0 2024-06-06T07:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.204+0000 7f306d491640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 0x7f30540a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 Processor -- start 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 -- start start 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 0x7f30540a5400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f3060009360 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec9721e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec9724740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55cec967b4f0 con 0x55cec9717170 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cec9619870 con 0x55cec9722350 2024-06-06T07:48:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f3072402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cec96774b0 con 0x55cec9717a90 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38428/0 (socket says 172.21.15.87:38428) 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 -- 172.21.15.87:0/2392902322 learned_addr learned my addr 172.21.15.87:0/2392902322 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306cc90640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec9721e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306dc92640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec9724740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 -- 172.21.15.87:0/2392902322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 msgr2=0x55cec9721e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec9721e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 -- 172.21.15.87:0/2392902322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 msgr2=0x55cec9724740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec9724740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306d491640 1 -- 172.21.15.87:0/2392902322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec96359b0 con 0x55cec9717170 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.205+0000 7f306dc92640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec9724740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.206+0000 7f306d491640 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3060009cd0 tx=0x7f306003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db0b6cea2a1771 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.206+0000 7f30537fe640 1 -- 172.21.15.87:0/2392902322 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060003b70 con 0x55cec9717170 2024-06-06T07:48:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.206+0000 7f30537fe640 1 -- 172.21.15.87:0/2392902322 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060003d10 con 0x55cec9717170 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.206+0000 7f30537fe640 1 -- 172.21.15.87:0/2392902322 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060030c40 con 0x55cec9717170 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- 172.21.15.87:0/2392902322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 msgr2=0x55cec9717550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3060009cd0 tx=0x7f306003e000 comp rx=0 tx=0).stop 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- 172.21.15.87:0/2392902322 shutdown_connections 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec9724740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec9721e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 --2- 172.21.15.87:0/2392902322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9717550 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- 172.21.15.87:0/2392902322 >> 172.21.15.87:0/2392902322 conn(0x55cec9713680 msgr2=0x55cec9713a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- 172.21.15.87:0/2392902322 shutdown_connections 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- 172.21.15.87:0/2392902322 wait complete. 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 -- start start 2024-06-06T07:48:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.207+0000 7f3072402740 1 Processor -- start 2024-06-06T07:48:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec971f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec971fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55cec9724f20 con 0x55cec9717170 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cec9604ae0 con 0x55cec9722350 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cec97250a0 con 0x55cec9717a90 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3072402740 10 client.0 _ll_register_callbacks cb 0x55cec963d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38436/0 (socket says 172.21.15.87:38436) 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 -- 172.21.15.87:0/1566024928 learned_addr learned my addr 172.21.15.87:0/1566024928 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306cc90640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec971f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306dc92640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec971fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 -- 172.21.15.87:0/1566024928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 msgr2=0x55cec971f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec971f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 -- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 msgr2=0x55cec971fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec971fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 -- 172.21.15.87:0/1566024928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec938f210 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306dc92640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec971fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f306d491640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3060005e30 tx=0x7f3060030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f409d0c690afb956 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.208+0000 7f3050ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600040c0 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3060004260 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3060003d10 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3060065050 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cec945b620 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060003d10 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3060030070 con 0x55cec9717170 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 20 client.15525 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec97322c0 0x55cec97346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 4 client.15525 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec968ef10 con 0x55cec97322c0 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.209+0000 7f306cc90640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec97322c0 0x55cec97346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.210+0000 7f306cc90640 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec97322c0 0x55cec97346b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d1faf6022134cd server_cookie=5cd2cbd933409f2f in_seq=0 out_seq=0 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.210+0000 7f3050ff9640 10 client.15525 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3050ff9640 1 -- 172.21.15.87:0/1566024928 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cec968ef10 con 0x55cec97322c0 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3050ff9640 10 client.15525 handle_command_reply: tid=0 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 client.15525 shutdown 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 msgr2=0x55cec971f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3060005e30 tx=0x7f3060030fc0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 shutdown_connections 2024-06-06T07:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec97322c0 0x55cec97346b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9722350 0x55cec971fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9717a90 0x55cec971f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 --2- 172.21.15.87:0/1566024928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec971f2b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 >> 172.21.15.87:0/1566024928 conn(0x55cec9713680 msgr2=0x55cec97193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3050ff9640 0 client.15525 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 shutdown_connections 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 1 -- 172.21.15.87:0/1566024928 wait complete. 2024-06-06T07:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.211+0000 7f3072402740 20 client.15525 trim_cache size 0 max 16384 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 Processor -- start 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 -- start start 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717170 0x55cec970e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9739ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30681b18f0 con 0x55cec9717a90 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30681af1c0 con 0x55cec9717170 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.227+0000 7f3072402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30681ad010 con 0x55cec9722350 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38452/0 (socket says 172.21.15.87:38452) 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 -- 172.21.15.87:0/3716951662 learned_addr learned my addr 172.21.15.87:0/3716951662 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306dc92640 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9739ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306d491640 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717170 0x55cec970e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 -- 172.21.15.87:0/3716951662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 msgr2=0x55cec9739ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9739ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 -- 172.21.15.87:0/3716951662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717170 msgr2=0x55cec970e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717170 0x55cec970e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 -- 172.21.15.87:0/3716951662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec97178c0 con 0x55cec9717a90 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f306cc90640 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55cec9718710 tx=0x7f305800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d70a2d2e5e1cc50 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f3052ffd640 1 -- 172.21.15.87:0/3716951662 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058003d20 con 0x55cec9717a90 2024-06-06T07:48:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f3052ffd640 1 -- 172.21.15.87:0/3716951662 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058005500 con 0x55cec9717a90 2024-06-06T07:48:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.228+0000 7f3052ffd640 1 -- 172.21.15.87:0/3716951662 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580266c0 con 0x55cec9717a90 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- 172.21.15.87:0/3716951662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 msgr2=0x55cec97211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55cec9718710 tx=0x7f305800b390 comp rx=0 tx=0).stop 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- 172.21.15.87:0/3716951662 shutdown_connections 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9739ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717a90 0x55cec97211f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 --2- 172.21.15.87:0/3716951662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717170 0x55cec970e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- 172.21.15.87:0/3716951662 >> 172.21.15.87:0/3716951662 conn(0x55cec9713680 msgr2=0x55cec9720ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- 172.21.15.87:0/3716951662 shutdown_connections 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- 172.21.15.87:0/3716951662 wait complete. 2024-06-06T07:48:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 -- start start 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.229+0000 7f3072402740 1 Processor -- start 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717a90 0x55cec9725fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9794c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30681ad680 con 0x55cec9717170 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30681aeaf0 con 0x55cec9717a90 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30681add50 con 0x55cec9722350 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f3072402740 10 client.0 _ll_register_callbacks cb 0x7f305800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38456/0 (socket says 172.21.15.87:38456) 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 -- 172.21.15.87:0/1217198904 learned_addr learned my addr 172.21.15.87:0/1217198904 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306dc92640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9794c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306cc90640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717a90 0x55cec9725fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 -- 172.21.15.87:0/1217198904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 msgr2=0x55cec9794c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9794c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 -- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717a90 msgr2=0x55cec9725fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717a90 0x55cec9725fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.230+0000 7f306d491640 1 -- 172.21.15.87:0/1217198904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec9795260 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f306d491640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f80 tx=0x7f30600307b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb00e9b4c608acd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060050880 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f30600475f0 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3058005500 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3060071050 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cec979c610 con 0x55cec9717170 2024-06-06T07:48:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058005500 con 0x55cec9717170 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f306003b400 con 0x55cec9717170 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 20 client.15531 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec979ccd0 0x55cec979f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 4 client.15531 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.231+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cec9688c20 con 0x55cec979ccd0 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.232+0000 7f306cc90640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec979ccd0 0x55cec979f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.232+0000 7f3050ff9640 10 client.15531 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.232+0000 7f306cc90640 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec979ccd0 0x55cec979f0f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d919d2e8f8c80e server_cookie=5fb212a9c907cbe6 in_seq=0 out_seq=0 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.240+0000 7f3050ff9640 1 -- 172.21.15.87:0/1217198904 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cec9688c20 con 0x55cec979ccd0 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.240+0000 7f3050ff9640 10 client.15531 handle_command_reply: tid=0 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.240+0000 7f3072402740 1 client.15531 shutdown 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.240+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 msgr2=0x55cec9725a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.240+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f80 tx=0x7f30600307b0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 shutdown_connections 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55cec979ccd0 0x55cec979f0f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec9722350 0x55cec9794c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3050ff9640 0 client.15531 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cec9717a90 0x55cec9725fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 --2- 172.21.15.87:0/1217198904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cec9717170 0x55cec9725a80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 >> 172.21.15.87:0/1217198904 conn(0x55cec9713680 msgr2=0x55cec971fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 shutdown_connections 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 1 -- 172.21.15.87:0/1217198904 wait complete. 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f3072402740 20 client.15531 trim_cache size 0 max 16384 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:48:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 msgr2=0x7f30540a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 0x7f30540a5400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f3060009360 comp rx=0 tx=0).stop 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 msgr2=0x7f306819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f306400d750 tx=0x7f306400dc10 comp rx=0 tx=0).stop 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 shutdown_connections 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f30540a2f40 0x7f30540a5400 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30681079a0 0x7f306819caa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3068102350 0x7f3068198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 --2- 172.21.15.87:0/2336795257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3068101990 0x7f3068198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 >> 172.21.15.87:0/2336795257 conn(0x7f30680fd460 msgr2=0x7f30680fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 shutdown_connections 2024-06-06T07:48:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:42.241+0000 7f306f6f5640 1 -- 172.21.15.87:0/2336795257 wait complete. 2024-06-06T07:48:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:42 smithi087 ceph-mon[32276]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:42 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2979155263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:42.447 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:42 smithi195 ceph-mon[37040]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:42 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2979155263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:42 smithi181 ceph-mon[36790]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T07:48:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:42 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2979155263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:44.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:44 smithi087 ceph-mon[32276]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- 172.21.15.87:0/1916189206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 msgr2=0x7f9d941067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 --2- 172.21.15.87:0/1916189206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941067a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009990 tx=0x7f9d84009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- 172.21.15.87:0/1916189206 shutdown_connections 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 --2- 172.21.15.87:0/1916189206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 --2- 172.21.15.87:0/1916189206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941067a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 --2- 172.21.15.87:0/1916189206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 0x7f9d94102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- 172.21.15.87:0/1916189206 >> 172.21.15.87:0/1916189206 conn(0x7f9d940766c0 msgr2=0x7f9d94076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- 172.21.15.87:0/1916189206 shutdown_connections 2024-06-06T07:48:44.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- 172.21.15.87:0/1916189206 wait complete. 2024-06-06T07:48:44.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 Processor -- start 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.242+0000 7f9d9a712700 1 -- start start 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 0x7f9d94199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941116f0 con 0x7f9d941063c0 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111570 con 0x7f9d94102890 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d9a712700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111870 con 0x7f9d94101ed0 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38460/0 (socket says 172.21.15.87:38460) 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 -- 172.21.15.87:0/2404019761 learned_addr learned my addr 172.21.15.87:0/2404019761 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d93fff700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 0x7f9d94199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d937fe700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 -- 172.21.15.87:0/2404019761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 msgr2=0x7f9d94199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 0x7f9d94199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 -- 172.21.15.87:0/2404019761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 msgr2=0x7f9d9419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d98caf700 1 -- 172.21.15.87:0/2404019761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d941a50b0 con 0x7f9d941063c0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.243+0000 7f9d937fe700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d98caf700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00ada0 tx=0x7f9d8c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d917fa700 1 -- 172.21.15.87:0/2404019761 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c017020 con 0x7f9d941063c0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d941a52e0 con 0x7f9d941063c0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d917fa700 1 -- 172.21.15.87:0/2404019761 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c004bb0 con 0x7f9d941063c0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d917fa700 1 -- 172.21.15.87:0/2404019761 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005520 con 0x7f9d941063c0 2024-06-06T07:48:44.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.244+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941a57a0 con 0x7f9d941063c0 2024-06-06T07:48:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.246+0000 7f9d917fa700 1 -- 172.21.15.87:0/2404019761 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9d8c0056c0 con 0x7f9d941063c0 2024-06-06T07:48:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.246+0000 7f9d917fa700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 0x7f9d7c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9d93fff700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 0x7f9d7c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9d917fa700 1 -- 172.21.15.87:0/2404019761 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9d8c0cf1d0 con 0x7f9d941063c0 2024-06-06T07:48:44.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9d93fff700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 0x7f9d7c0a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8400b540 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9da4f95b80 1 Processor -- start 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9da4f95b80 1 -- start start 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.247+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 0x55e9b29c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e9b291a7c0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e9b28c26d0 con 0x55e9b29c8ed0 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b2916780 con 0x55e9b29cef10 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38470/0 (socket says 172.21.15.87:38470) 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 -- 172.21.15.87:0/3184814433 learned_addr learned my addr 172.21.15.87:0/3184814433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:44.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d937fe700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 0x55e9b29c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d98caf700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 -- 172.21.15.87:0/3184814433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 msgr2=0x55e9b29c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 0x55e9b29c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 -- 172.21.15.87:0/3184814433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 msgr2=0x55e9b29c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 -- 172.21.15.87:0/3184814433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b28b8f00 con 0x55e9b29ce5f0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d937fe700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 0x55e9b29c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d98caf700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400ba60 tx=0x7f9d8401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b30d0a856fd519 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d7a7fc700 1 -- 172.21.15.87:0/3184814433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84003780 con 0x55e9b29ce5f0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d7a7fc700 1 -- 172.21.15.87:0/3184814433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8400f940 con 0x55e9b29ce5f0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.248+0000 7f9d7a7fc700 1 -- 172.21.15.87:0/3184814433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84024970 con 0x55e9b29ce5f0 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3184814433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 msgr2=0x55e9b29ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400ba60 tx=0x7f9d8401f000 comp rx=0 tx=0).stop 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3184814433 shutdown_connections 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29ce9d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3184814433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29cef10 0x55e9b29c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3184814433 >> 172.21.15.87:0/3184814433 conn(0x55e9b28f3c70 msgr2=0x55e9b29c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3184814433 shutdown_connections 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.249+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3184814433 wait complete. 2024-06-06T07:48:44.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 -- start start 2024-06-06T07:48:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 Processor -- start 2024-06-06T07:48:44.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 0x55e9b29d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29d58c0 con 0x55e9b29cef10 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e9b28baba0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29d5a40 con 0x55e9b29c8ed0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9da4f95b80 10 client.0 _ll_register_callbacks cb 0x55e9b28b48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38482/0 (socket says 172.21.15.87:38482) 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.250+0000 7f9d98caf700 1 -- 172.21.15.87:0/3149711088 learned_addr learned my addr 172.21.15.87:0/3149711088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d937fe700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 0x55e9b29d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 -- 172.21.15.87:0/3149711088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 msgr2=0x55e9b29d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 -- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 msgr2=0x55e9b29d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 0x55e9b29d1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 -- 172.21.15.87:0/3149711088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b2869880 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d937fe700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 0x55e9b29d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d98caf700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0975c0 tx=0x7f9d8c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3133b093ac4f068 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0b4060 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9d8c09b3c0 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8400f940 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c09f510 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9da4f95b80 10 client.15543 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9b29e01a0 con 0x55e9b29cef10 2024-06-06T07:48:44.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8400f940 con 0x55e9b29cef10 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8c012070 con 0x55e9b29cef10 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9da4f95b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.251+0000 7f9da4f95b80 10 client.15543 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9da4f95b80 20 client.15543 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b29e27e0 0x55e9b29e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9da4f95b80 4 client.15543 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b2927cc0 con 0x55e9b29e27e0 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b29e27e0 0x55e9b29e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9d737fe700 10 client.15543 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.252+0000 7f9d93fff700 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b29e27e0 0x55e9b29e3c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42ae051ed2ed614f server_cookie=5fb1098f41129abc in_seq=0 out_seq=0 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9d737fe700 1 -- 172.21.15.87:0/3149711088 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e9b2927cc0 con 0x55e9b29e27e0 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9d737fe700 10 client.15543 handle_command_reply: tid=0 2024-06-06T07:48:44.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 client.15543 shutdown 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 msgr2=0x55e9b29d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0975c0 tx=0x7f9d8c007f90 comp rx=0 tx=0).stop 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 shutdown_connections 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b29e27e0 0x55e9b29e3c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29d2490 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/3149711088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29ce5f0 0x55e9b29d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 >> 172.21.15.87:0/3149711088 conn(0x55e9b28f3c70 msgr2=0x55e9b29c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.253+0000 7f9d737fe700 0 client.15543 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.254+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 shutdown_connections 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.254+0000 7f9da4f95b80 1 -- 172.21.15.87:0/3149711088 wait complete. 2024-06-06T07:48:44.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.254+0000 7f9da4f95b80 20 client.15543 trim_cache size 0 max 16384 2024-06-06T07:48:44.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 Processor -- start 2024-06-06T07:48:44.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 -- start start 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29ce5f0 0x55e9b29cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e1f90 con 0x55e9b29cef10 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e2660 con 0x55e9b29c8ed0 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e2d30 con 0x55e9b29ce5f0 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38498/0 (socket says 172.21.15.87:38498) 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 -- 172.21.15.87:0/4014009591 learned_addr learned my addr 172.21.15.87:0/4014009591 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d937fe700 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29ce5f0 0x55e9b29cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d93fff700 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 -- 172.21.15.87:0/4014009591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29ce5f0 msgr2=0x55e9b29cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29ce5f0 0x55e9b29cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 -- 172.21.15.87:0/4014009591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 msgr2=0x55e9b29c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 -- 172.21.15.87:0/4014009591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b29d4890 con 0x55e9b29cef10 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.275+0000 7f9d98caf700 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09ae60 tx=0x7f9d8c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4319578f5d4e600 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9d79ffb700 1 -- 172.21.15.87:0/4014009591 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0a3d40 con 0x55e9b29cef10 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9d79ffb700 1 -- 172.21.15.87:0/4014009591 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c0a38b0 con 0x55e9b29cef10 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9d79ffb700 1 -- 172.21.15.87:0/4014009591 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005c60 con 0x55e9b29cef10 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 -- 172.21.15.87:0/4014009591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 msgr2=0x55e9b29bcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09ae60 tx=0x7f9d8c004f90 comp rx=0 tx=0).stop 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 -- 172.21.15.87:0/4014009591 shutdown_connections 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29cef10 0x55e9b29bcdb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/4014009591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29ce5f0 0x55e9b29cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 -- 172.21.15.87:0/4014009591 >> 172.21.15.87:0/4014009591 conn(0x55e9b28f3c70 msgr2=0x55e9b29cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 -- 172.21.15.87:0/4014009591 shutdown_connections 2024-06-06T07:48:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.276+0000 7f9da4f95b80 1 -- 172.21.15.87:0/4014009591 wait complete. 2024-06-06T07:48:44.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 -- start start 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 Processor -- start 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29cef10 0x55e9b29c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e3ad0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e3400 con 0x55e9b29cef10 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b29e41a0 con 0x55e9b29c8ed0 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9da4f95b80 10 client.0 _ll_register_callbacks cb 0x55e9b28b48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:44.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38514/0 (socket says 172.21.15.87:38514) 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d937fe700 1 -- 172.21.15.87:0/551185985 learned_addr learned my addr 172.21.15.87:0/551185985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d93fff700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d937fe700 1 -- 172.21.15.87:0/551185985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 msgr2=0x55e9b29c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.277+0000 7f9d937fe700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d937fe700 1 -- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29cef10 msgr2=0x55e9b29c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d937fe700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29cef10 0x55e9b29c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d937fe700 1 -- 172.21.15.87:0/551185985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b2a510a0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d937fe700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d94101910 tx=0x7f9d88012900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d6414e4b5a6024 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800f610 con 0x55e9b29ce5f0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9d8800f7b0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c0a38b0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d8800f9b0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 10 client.15549 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9b2a52360 con 0x55e9b29ce5f0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c0a38b0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8801b3c0 con 0x55e9b29ce5f0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 10 client.15549 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 20 client.15549 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b2a57090 0x55e9b2a55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 4 client.15549 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.278+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "status"}) -- 0x55e9b2922240 con 0x55e9b2a57090 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d93fff700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b2a57090 0x55e9b2a55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d737fe700 10 client.15549 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:48:44.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d93fff700 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b2a57090 0x55e9b2a55da0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ae51a3c42a8930 server_cookie=7ecf75d3514513f5 in_seq=0 out_seq=0 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d737fe700 1 -- 172.21.15.87:0/551185985 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55e9b2922240 con 0x55e9b2a57090 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d737fe700 10 client.15549 handle_command_reply: tid=0 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 client.15549 shutdown 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 msgr2=0x55e9b29c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d94101910 tx=0x7f9d88012900 comp rx=0 tx=0).stop 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 shutdown_connections 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e9b29cef10 0x55e9b29c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55e9b2a57090 0x55e9b2a55da0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b29c8ed0 0x55e9b29c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9da4f95b80 1 --2- 172.21.15.87:0/551185985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9b29ce5f0 0x55e9b29c7ec0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 >> 172.21.15.87:0/551185985 conn(0x55e9b28f3c70 msgr2=0x55e9b29cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.279+0000 7f9d737fe700 0 client.15549 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 shutdown_connections 2024-06-06T07:48:44.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9da4f95b80 1 -- 172.21.15.87:0/551185985 wait complete. 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9da4f95b80 20 client.15549 trim_cache size 0 max 16384 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24446, 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 222.05664766999999, 2024-06-06T07:48:44.286 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 27, 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 100, 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 311.47272581800001 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 msgr2=0x7f9d7c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 0x7f9d7c0a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8400b540 comp rx=0 tx=0).stop 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 msgr2=0x7f9d941a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:44.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00ada0 tx=0x7f9d8c00c330 comp rx=0 tx=0).stop 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 shutdown_connections 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9d7c0a2eb0 0x7f9d7c0a5370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d941063c0 0x7f9d941a4b70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d94101ed0 0x7f9d94199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 --2- 172.21.15.87:0/2404019761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94102890 0x7f9d9419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 >> 172.21.15.87:0/2404019761 conn(0x7f9d940766c0 msgr2=0x7f9d940fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 shutdown_connections 2024-06-06T07:48:44.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:44.280+0000 7f9d9a712700 1 -- 172.21.15.87:0/2404019761 wait complete. 2024-06-06T07:48:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:44 smithi195 ceph-mon[37040]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-06-06T07:48:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:44 smithi181 ceph-mon[36790]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-06-06T07:48:45.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T07:48:45.442 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:46.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:46 smithi087 ceph-mon[32276]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T07:48:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:46 smithi195 ceph-mon[37040]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T07:48:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:46 smithi181 ceph-mon[36790]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T07:48:47.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 -- 172.21.15.87:0/335352800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c1030f0 msgr2=0x7f811c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 --2- 172.21.15.87:0/335352800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c1030f0 0x7f811c107110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f810c0099c0 tx=0x7f810c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 -- 172.21.15.87:0/335352800 shutdown_connections 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 --2- 172.21.15.87:0/335352800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c102740 0x7f811c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 --2- 172.21.15.87:0/335352800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c107840 0x7f811c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 --2- 172.21.15.87:0/335352800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c1030f0 0x7f811c107110 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 -- 172.21.15.87:0/335352800 >> 172.21.15.87:0/335352800 conn(0x7f811c0795f0 msgr2=0x7f811c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 -- 172.21.15.87:0/335352800 shutdown_connections 2024-06-06T07:48:47.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.205+0000 7f812244c700 1 -- 172.21.15.87:0/335352800 wait complete. 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 Processor -- start 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 -- start start 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c1030f0 0x7f811c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c107840 0x7f811c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f811c112770 con 0x7f811c102740 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1125f0 con 0x7f811c107840 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.206+0000 7f812244c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1128f0 con 0x7f811c1030f0 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32942/0 (socket says 172.21.15.87:32942) 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 -- 172.21.15.87:0/2108390635 learned_addr learned my addr 172.21.15.87:0/2108390635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:47.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c1030f0 0x7f811c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f81209e9700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c107840 0x7f811c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 -- 172.21.15.87:0/2108390635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c1030f0 msgr2=0x7f811c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c1030f0 0x7f811c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 -- 172.21.15.87:0/2108390635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c107840 msgr2=0x7f811c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c107840 0x7f811c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 -- 172.21.15.87:0/2108390635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c199190 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f811bfff700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f810400c920 tx=0x7f810400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f81197fa700 1 -- 172.21.15.87:0/2108390635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104007b00 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f81197fa700 1 -- 172.21.15.87:0/2108390635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8104004d10 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c199480 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f81197fa700 1 -- 172.21.15.87:0/2108390635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104005680 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.207+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c1a5740 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.209+0000 7f81197fa700 1 -- 172.21.15.87:0/2108390635 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f8104004750 con 0x7f811c102740 2024-06-06T07:48:47.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.210+0000 7f81197fa700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 0x7f81080a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.210+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 0x7f81080a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.210+0000 7f81197fa700 1 -- 172.21.15.87:0/2108390635 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f81040cf660 con 0x7f811c102740 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.210+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 0x7f81080a5370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f811c1070c0 tx=0x7f810c005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 Processor -- start 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 -- start start 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5586299b17c0 con 0x558629a65f10 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5586299596d0 con 0x558629a5fed0 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f812cccfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5586299ad780 con 0x558629a655f0 2024-06-06T07:48:47.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32950/0 (socket says 172.21.15.87:32950) 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 -- 172.21.15.87:0/2874019504 learned_addr learned my addr 172.21.15.87:0/2874019504 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f81209e9700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811bfff700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 -- 172.21.15.87:0/2874019504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 msgr2=0x558629a659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 -- 172.21.15.87:0/2874019504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 msgr2=0x558629a602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811b7fe700 1 -- 172.21.15.87:0/2874019504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862994ff00 con 0x558629a65f10 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f811bfff700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.211+0000 7f81209e9700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f811b7fe700 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f810c015000 tx=0x7f810c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab754397117e8bb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f81127fc700 1 -- 172.21.15.87:0/2874019504 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c00bdb0 con 0x558629a65f10 2024-06-06T07:48:47.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f81127fc700 1 -- 172.21.15.87:0/2874019504 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c00fd50 con 0x558629a65f10 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f81127fc700 1 -- 172.21.15.87:0/2874019504 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c024da0 con 0x558629a65f10 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f812cccfb80 1 -- 172.21.15.87:0/2874019504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 msgr2=0x558629a5f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.212+0000 7f812cccfb80 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f810c015000 tx=0x7f810c020000 comp rx=0 tx=0).stop 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- 172.21.15.87:0/2874019504 shutdown_connections 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- 172.21.15.87:0/2874019504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a5f990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- 172.21.15.87:0/2874019504 >> 172.21.15.87:0/2874019504 conn(0x55862998ac70 msgr2=0x558629a5a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- 172.21.15.87:0/2874019504 shutdown_connections 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- 172.21.15.87:0/2874019504 wait complete. 2024-06-06T07:48:47.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- start start 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 Processor -- start 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a655f0 0x558629a68fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a65f10 0x558629a694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558629a6c8c0 con 0x558629a5fed0 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.213+0000 7f812cccfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558629951ba0 con 0x558629a655f0 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f812cccfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558629a6ca40 con 0x558629a65f10 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32960/0 (socket says 172.21.15.87:32960) 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 -- 172.21.15.87:0/189368581 learned_addr learned my addr 172.21.15.87:0/189368581 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f812cccfb80 10 client.0 _ll_register_callbacks cb 0x55862994b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f81209e9700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a65f10 0x558629a694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811b7fe700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a655f0 0x558629a68fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 -- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a65f10 msgr2=0x558629a694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a65f10 0x558629a694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 -- 172.21.15.87:0/189368581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a655f0 msgr2=0x558629a68fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a655f0 0x558629a68fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 -- 172.21.15.87:0/189368581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558629900880 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f811bfff700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f810400c8f0 tx=0x7f810409b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482a5e3b8a790052 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f80fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040b4060 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f81040a3f40 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810c00fd50 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f810401d030 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f812cccfb80 10 client.15561 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.214+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558629a771c0 con 0x558629a5fed0 2024-06-06T07:48:47.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c00fd50 con 0x558629a5fed0 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f81040ccb40 con 0x558629a5fed0 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 10 client.15561 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 20 client.15561 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629a79720 0x558629a7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 4 client.15561 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586299becc0 con 0x558629a79720 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f811b7fe700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629a79720 0x558629a7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f811b7fe700 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629a79720 0x558629a7ab90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be5ba46b6ba35a06 server_cookie=ba6739e0b477aa6f in_seq=0 out_seq=0 2024-06-06T07:48:47.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.215+0000 7f80fb7fe700 10 client.15561 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f80fb7fe700 1 -- 172.21.15.87:0/189368581 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5586299becc0 con 0x558629a79720 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f80fb7fe700 10 client.15561 handle_command_reply: tid=0 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 client.15561 shutdown 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 msgr2=0x558629a6bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f810400c8f0 tx=0x7f810409b0a0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 shutdown_connections 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a65f10 0x558629a694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a5fed0 0x558629a6bcb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629a79720 0x558629a7ab90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f80fb7fe700 0 client.15561 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.216+0000 7f812cccfb80 1 --2- 172.21.15.87:0/189368581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a655f0 0x558629a68fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.217+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 >> 172.21.15.87:0/189368581 conn(0x55862998ac70 msgr2=0x558629a5e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.217+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 shutdown_connections 2024-06-06T07:48:47.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.217+0000 7f812cccfb80 1 -- 172.21.15.87:0/189368581 wait complete. 2024-06-06T07:48:47.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.217+0000 7f812cccfb80 20 client.15561 trim_cache size 0 max 16384 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 Processor -- start 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 -- start start 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558629a78ed0 con 0x558629a65f10 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558629a795a0 con 0x558629a5fed0 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f812cccfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558629a79c70 con 0x558629a655f0 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f81209e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f81209e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32970/0 (socket says 172.21.15.87:32970) 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.238+0000 7f81209e9700 1 -- 172.21.15.87:0/3921430802 learned_addr learned my addr 172.21.15.87:0/3921430802 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f811bfff700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f811b7fe700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 -- 172.21.15.87:0/3921430802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 msgr2=0x558629a843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a843e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 -- 172.21.15.87:0/3921430802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 msgr2=0x558629a5f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a5f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 -- 172.21.15.87:0/3921430802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558629a67520 con 0x558629a65f10 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f811b7fe700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f811bfff700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a5f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f81209e9700 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8114009dc0 tx=0x7f81140077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ff625aed27b6ea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f8111ffb700 1 -- 172.21.15.87:0/3921430802 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811401b070 con 0x558629a65f10 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f8111ffb700 1 -- 172.21.15.87:0/3921430802 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114017920 con 0x558629a65f10 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.239+0000 7f8111ffb700 1 -- 172.21.15.87:0/3921430802 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81140169c0 con 0x558629a65f10 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- 172.21.15.87:0/3921430802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 msgr2=0x558629a662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8114009dc0 tx=0x7f81140077e0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- 172.21.15.87:0/3921430802 shutdown_connections 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a5f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a662f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 --2- 172.21.15.87:0/3921430802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- 172.21.15.87:0/3921430802 >> 172.21.15.87:0/3921430802 conn(0x55862998ac70 msgr2=0x558629a678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- 172.21.15.87:0/3921430802 shutdown_connections 2024-06-06T07:48:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- 172.21.15.87:0/3921430802 wait complete. 2024-06-06T07:48:47.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.240+0000 7f812cccfb80 1 -- start start 2024-06-06T07:48:47.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 Processor -- start 2024-06-06T07:48:47.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558629a7aa10 con 0x558629a65f10 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558629a7a340 con 0x558629a5fed0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558629a7b0e0 con 0x558629a655f0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f812cccfb80 10 client.0 _ll_register_callbacks cb 0x55862994b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32972/0 (socket says 172.21.15.87:32972) 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 -- 172.21.15.87:0/642267281 learned_addr learned my addr 172.21.15.87:0/642267281 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f811bfff700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f811b7fe700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 -- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 msgr2=0x558629a75ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a75ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 -- 172.21.15.87:0/642267281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 msgr2=0x558629a75990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a75990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f81209e9700 1 -- 172.21.15.87:0/642267281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558629a83dd0 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.241+0000 7f811b7fe700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a75ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f81209e9700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8114017dd0 tx=0x7f8114017e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a99c883a2114829 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811401b030 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8114017920 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f81140165c0 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8114030050 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f812cccfb80 10 client.15567 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558629ae9bd0 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114017920 con 0x558629a65f10 2024-06-06T07:48:47.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.242+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8114020970 con 0x558629a65f10 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 10 client.15567 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 20 client.15567 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629aee430 0x558629aed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 4 client.15567 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "status"}) -- 0x5586299b9240 con 0x558629aee430 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f811bfff700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629aee430 0x558629aed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f811bfff700 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629aee430 0x558629aed140 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe5ac33df82900f6 server_cookie=ebf1d5cdf453fbb4 in_seq=0 out_seq=0 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f80fb7fe700 10 client.15567 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f80fb7fe700 1 -- 172.21.15.87:0/642267281 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5586299b9240 con 0x558629aee430 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f80fb7fe700 10 client.15567 handle_command_reply: tid=0 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.243+0000 7f812cccfb80 1 client.15567 shutdown 2024-06-06T07:48:47.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 msgr2=0x558629a835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8114017dd0 tx=0x7f8114017e00 comp rx=0 tx=0).stop 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 shutdown_connections 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x558629aee430 0x558629aed140 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558629a5fed0 0x558629a75990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f80fb7fe700 0 client.15567 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558629a65f10 0x558629a835d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 --2- 172.21.15.87:0/642267281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558629a655f0 0x558629a75ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 >> 172.21.15.87:0/642267281 conn(0x55862998ac70 msgr2=0x558629a53db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 shutdown_connections 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 1 -- 172.21.15.87:0/642267281 wait complete. 2024-06-06T07:48:47.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812cccfb80 20 client.15567 trim_cache size 0 max 16384 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24448, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 226.03077293600001, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 27, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 100, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 311.58013609900001 2024-06-06T07:48:47.252 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 msgr2=0x7f81080a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 0x7f81080a5370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f811c1070c0 tx=0x7f810c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 msgr2=0x7f811c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.244+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f810400c920 tx=0x7f810400cc30 comp rx=0 tx=0).stop 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 shutdown_connections 2024-06-06T07:48:47.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c107840 0x7f811c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81080a2eb0 0x7f81080a5370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f811c102740 0x7f811c19e870 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 --2- 172.21.15.87:0/2108390635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f811c1030f0 0x7f811c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 >> 172.21.15.87:0/2108390635 conn(0x7f811c0795f0 msgr2=0x7f811c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 shutdown_connections 2024-06-06T07:48:47.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:47.245+0000 7f812244c700 1 -- 172.21.15.87:0/2108390635 wait complete. 2024-06-06T07:48:47.963 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:48:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:48 smithi087 ceph-mon[32276]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:48.202 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:48.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:48 smithi195 ceph-mon[37040]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:48 smithi181 ceph-mon[36790]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4137767729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c1024d0 msgr2=0x7f3f7c10e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4137767729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c10e250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400b0a0 tx=0x7f3f7400b3b0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4137767729 shutdown_connections 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4137767729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c107480 0x7f3f7c107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4137767729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c10e250 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4137767729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c101b10 0x7f3f7c101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4137767729 >> 172.21.15.87:0/4137767729 conn(0x7f3f7c077990 msgr2=0x7f3f7c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4137767729 shutdown_connections 2024-06-06T07:48:50.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.049+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4137767729 wait complete. 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.050+0000 7f3f81f0e700 1 Processor -- start 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.050+0000 7f3f81f0e700 1 -- start start 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.050+0000 7f3f81f0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c101b10 0x7f3f7c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.050+0000 7f3f81f0e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.050+0000 7f3f81f0e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f81f0e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c111760 con 0x7f3f7c107480 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f81f0e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1115e0 con 0x7f3f7c101b10 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f81f0e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1118e0 con 0x7f3f7c1024d0 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32990/0 (socket says 172.21.15.87:32990) 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 -- 172.21.15.87:0/4282970070 learned_addr learned my addr 172.21.15.87:0/4282970070 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7affd700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c101b10 0x7f3f7c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 -- 172.21.15.87:0/4282970070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c1024d0 msgr2=0x7f3f7c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 -- 172.21.15.87:0/4282970070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c101b10 msgr2=0x7f3f7c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c101b10 0x7f3f7c199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 -- 172.21.15.87:0/4282970070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1a5060 con 0x7f3f7c107480 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3f740062a0 tx=0x7f3f74004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f78ff9700 1 -- 172.21.15.87:0/4282970070 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74021070 con 0x7f3f7c107480 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1a5230 con 0x7f3f7c107480 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f78ff9700 1 -- 172.21.15.87:0/4282970070 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74003c00 con 0x7f3f7c107480 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f78ff9700 1 -- 172.21.15.87:0/4282970070 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7401d450 con 0x7f3f7c107480 2024-06-06T07:48:50.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.051+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a5740 con 0x7f3f7c107480 2024-06-06T07:48:50.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.054+0000 7f3f78ff9700 1 -- 172.21.15.87:0/4282970070 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f3f74004020 con 0x7f3f7c107480 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.054+0000 7f3f78ff9700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 0x7f3f640a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.054+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 0x7f3f640a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.054+0000 7f3f78ff9700 1 -- 172.21.15.87:0/4282970070 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3f7400f020 con 0x7f3f7c107480 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 0x7f3f640a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c006fd0 tx=0x7f3f6c008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 Processor -- start 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 -- start start 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca336210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55faca228f10 con 0x55faca336750 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55faca287800 con 0x55faca33c830 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55faca1ef410 con 0x55faca33bf10 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32996/0 (socket says 172.21.15.87:32996) 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.055+0000 7f3f7bfff700 1 -- 172.21.15.87:0/3764164677 learned_addr learned my addr 172.21.15.87:0/3764164677 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 -- 172.21.15.87:0/3764164677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 msgr2=0x55faca33c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca336210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 -- 172.21.15.87:0/3764164677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 msgr2=0x55faca336210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca336210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 -- 172.21.15.87:0/3764164677 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faca1d6880 con 0x55faca336750 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400b830 tx=0x7f3f7409c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd284cd58d1f955 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f61ffb700 1 -- 172.21.15.87:0/3764164677 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f740a0f80 con 0x55faca336750 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f61ffb700 1 -- 172.21.15.87:0/3764164677 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f740a57b0 con 0x55faca336750 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.056+0000 7f3f61ffb700 1 -- 172.21.15.87:0/3764164677 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74007980 con 0x55faca336750 2024-06-06T07:48:50.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3764164677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 msgr2=0x55faca336b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400b830 tx=0x7f3f7409c530 comp rx=0 tx=0).stop 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3764164677 shutdown_connections 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca336b70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3764164677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca336210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3764164677 >> 172.21.15.87:0/3764164677 conn(0x55faca263030 msgr2=0x55faca3311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3764164677 shutdown_connections 2024-06-06T07:48:50.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3764164677 wait complete. 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 -- start start 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.057+0000 7f3f8c791b80 1 Processor -- start 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca336750 0x55faca342580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55faca227f70 con 0x55faca33c830 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55faca2837c0 con 0x55faca336750 2024-06-06T07:48:50.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55faca342fc0 con 0x55faca33bf10 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 10 client.0 _ll_register_callbacks cb 0x55faca21d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33010/0 (socket says 172.21.15.87:33010) 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 -- 172.21.15.87:0/532768435 learned_addr learned my addr 172.21.15.87:0/532768435 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca336750 0x55faca342580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7affd700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 -- 172.21.15.87:0/532768435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 msgr2=0x55faca33f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 -- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca336750 msgr2=0x55faca342580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca336750 0x55faca342580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 -- 172.21.15.87:0/532768435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faca24d700 con 0x55faca33c830 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007fa0 tx=0x7f3f74098750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad063df03582760b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74080060 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3f7409ce30 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.058+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f740a57b0 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f740bb070 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 10 client.15579 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55faca34d6a0 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f740a57b0 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3f74011070 con 0x55faca33c830 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 10 client.15579 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 20 client.15579 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca34fa70 0x55faca350ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 4 client.15579 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:50.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faca294d00 con 0x55faca34fa70 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.059+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca34fa70 0x55faca350ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.060+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca34fa70 0x55faca350ee0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9407d3e1cbb248b7 server_cookie=63c0439ab82d48b9 in_seq=0 out_seq=0 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.060+0000 7f3f5affd700 10 client.15579 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.060+0000 7f3f5affd700 1 -- 172.21.15.87:0/532768435 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55faca294d00 con 0x55faca34fa70 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.060+0000 7f3f5affd700 10 client.15579 handle_command_reply: tid=0 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 client.15579 shutdown 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 msgr2=0x55faca33fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007fa0 tx=0x7f3f74098750 comp rx=0 tx=0).stop 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 shutdown_connections 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca34fa70 0x55faca350ee0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca336750 0x55faca342580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33c830 0x55faca33fdc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/532768435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33bf10 0x55faca33f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 >> 172.21.15.87:0/532768435 conn(0x55faca263030 msgr2=0x55faca3306b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f5affd700 0 client.15579 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 shutdown_connections 2024-06-06T07:48:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 1 -- 172.21.15.87:0/532768435 wait complete. 2024-06-06T07:48:50.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.061+0000 7f3f8c791b80 20 client.15579 trim_cache size 0 max 16384 2024-06-06T07:48:50.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 Processor -- start 2024-06-06T07:48:50.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 -- start start 2024-06-06T07:48:50.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca336750 0x55faca336210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca331370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.083+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55faca34f220 con 0x55faca33bf10 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55faca34f8f0 con 0x55faca33c830 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55faca34ffc0 con 0x55faca336750 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33016/0 (socket says 172.21.15.87:33016) 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 -- 172.21.15.87:0/3161757310 learned_addr learned my addr 172.21.15.87:0/3161757310 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca331370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca336750 0x55faca336210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 -- 172.21.15.87:0/3161757310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca336750 msgr2=0x55faca336210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca336750 0x55faca336210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 -- 172.21.15.87:0/3161757310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 msgr2=0x55faca331370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca331370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 -- 172.21.15.87:0/3161757310 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faca332970 con 0x55faca33bf10 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f70004610 tx=0x7f3f7000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c79a4627a49516d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f617fa700 1 -- 172.21.15.87:0/3161757310 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70003680 con 0x55faca33bf10 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f617fa700 1 -- 172.21.15.87:0/3161757310 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70003d00 con 0x55faca33bf10 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.084+0000 7f3f617fa700 1 -- 172.21.15.87:0/3161757310 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011440 con 0x55faca33bf10 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3161757310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 msgr2=0x55faca330e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f70004610 tx=0x7f3f7000b9e0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3161757310 shutdown_connections 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33c830 0x55faca331370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca336750 0x55faca336210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3161757310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca33bf10 0x55faca330e30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3161757310 >> 172.21.15.87:0/3161757310 conn(0x55faca263030 msgr2=0x55faca33e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3161757310 shutdown_connections 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3161757310 wait complete. 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.085+0000 7f3f8c791b80 1 -- start start 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 Processor -- start 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 0x55faca3303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 0x55faca330920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55faca350d60 con 0x55faca336750 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55faca350690 con 0x55faca33bf10 2024-06-06T07:48:50.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55faca351430 con 0x55faca33c830 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f8c791b80 10 client.0 _ll_register_callbacks cb 0x55faca21d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33026/0 (socket says 172.21.15.87:33026) 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 -- 172.21.15.87:0/3059998759 learned_addr learned my addr 172.21.15.87:0/3059998759 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 0x55faca330920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 0x55faca3303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 -- 172.21.15.87:0/3059998759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 msgr2=0x55faca330920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 0x55faca330920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 -- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 msgr2=0x55faca3303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 0x55faca3303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 -- 172.21.15.87:0/3059998759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faca3310a0 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7bfff700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 0x55faca330920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 0x55faca3303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f7b7fe700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009730 tx=0x7f3f6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42599e0689c64c09 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.086+0000 7f3f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c02e020 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3f6c00f040 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70003d00 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f6c039050 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55faca3bb300 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70003d00 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3f6c013070 con 0x55faca336750 2024-06-06T07:48:50.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 10 client.15585 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 20 client.15585 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca3bff30 0x55faca3bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 4 client.15585 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.087+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "status"}) -- 0x55faca28f280 con 0x55faca3bff30 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca3bff30 0x55faca3bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f7affd700 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca3bff30 0x55faca3bec40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97474d40b726fe82 server_cookie=bc012259eceaca0d in_seq=0 out_seq=0 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f5affd700 10 client.15585 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:50.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f5affd700 1 -- 172.21.15.87:0/3059998759 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55faca28f280 con 0x55faca3bff30 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f5affd700 10 client.15585 handle_command_reply: tid=0 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 client.15585 shutdown 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 msgr2=0x55faca35a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009730 tx=0x7f3f6c004750 comp rx=0 tx=0).stop 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 shutdown_connections 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55faca3bff30 0x55faca3bec40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55faca33c830 0x55faca330920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55faca336750 0x55faca35a9b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f5affd700 0 client.15585 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 --2- 172.21.15.87:0/3059998759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55faca33bf10 0x55faca3303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.088+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 >> 172.21.15.87:0/3059998759 conn(0x55faca263030 msgr2=0x55faca33cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 shutdown_connections 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f8c791b80 1 -- 172.21.15.87:0/3059998759 wait complete. 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f8c791b80 20 client.15585 trim_cache size 0 max 16384 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "3f55109c-23d7-11ef-bc9b-c7b262605968", 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24467, 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-06-06T07:48:50.097 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 287.757993446, 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 27, 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 100, 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 304.51691224199999 2024-06-06T07:48:50.098 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 msgr2=0x7f3f640a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 0x7f3f640a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c006fd0 tx=0x7f3f6c008040 comp rx=0 tx=0).stop 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 msgr2=0x7f3f7c1a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3f740062a0 tx=0x7f3f74004ce0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 shutdown_connections 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7c101b10 0x7f3f7c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f640a2eb0 0x7f3f640a5370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f7c107480 0x7f3f7c1a4b20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 --2- 172.21.15.87:0/4282970070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f7c1024d0 0x7f3f7c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:50.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 >> 172.21.15.87:0/4282970070 conn(0x7f3f7c077990 msgr2=0x7f3f7c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:50.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 shutdown_connections 2024-06-06T07:48:50.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:50.089+0000 7f3f81f0e700 1 -- 172.21.15.87:0/4282970070 wait complete. 2024-06-06T07:48:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:50 smithi087 ceph-mon[32276]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:50 smithi195 ceph-mon[37040]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:50 smithi181 ceph-mon[36790]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:51.013 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-06T07:48:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:52 smithi087 ceph-mon[32276]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:52 smithi195 ceph-mon[37040]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:52 smithi181 ceph-mon[36790]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:48:53.014 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:48:53.266 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:54 smithi087 ceph-mon[32276]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:54 smithi195 ceph-mon[37040]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:54 smithi181 ceph-mon[36790]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-06T07:48:55.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 -- 172.21.15.87:0/4076295210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c101390 msgr2=0x7f144c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 --2- 172.21.15.87:0/4076295210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c101390 0x7f144c101810 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1440009990 tx=0x7f1440009ca0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 -- 172.21.15.87:0/4076295210 shutdown_connections 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 --2- 172.21.15.87:0/4076295210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c107400 0x7f144c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 --2- 172.21.15.87:0/4076295210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101d50 0x7f144c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 --2- 172.21.15.87:0/4076295210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c101390 0x7f144c101810 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 -- 172.21.15.87:0/4076295210 >> 172.21.15.87:0/4076295210 conn(0x7f144c078340 msgr2=0x7f144c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 -- 172.21.15.87:0/4076295210 shutdown_connections 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.066+0000 7f1454301700 1 -- 172.21.15.87:0/4076295210 wait complete. 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 Processor -- start 2024-06-06T07:48:55.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 -- start start 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101390 0x7f144c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 0x7f144c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f144c111720 con 0x7f144c107400 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1115a0 con 0x7f144c101d50 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f1454301700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1118a0 con 0x7f144c101390 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33048/0 (socket says 172.21.15.87:33048) 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 -- 172.21.15.87:0/2630417019 learned_addr learned my addr 172.21.15.87:0/2630417019 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145209d700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101390 0x7f144c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145189c700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 0x7f144c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 -- 172.21.15.87:0/2630417019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101390 msgr2=0x7f144c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101390 0x7f144c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.067+0000 7f145289e700 1 -- 172.21.15.87:0/2630417019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 msgr2=0x7f144c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f145289e700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 0x7f144c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f145289e700 1 -- 172.21.15.87:0/2630417019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c1a7160 con 0x7f144c107400 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f145189c700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 0x7f144c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f145289e700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f143c007ee0 tx=0x7f143c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f144b7fe700 1 -- 172.21.15.87:0/2630417019 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00dbd0 con 0x7f144c107400 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f144b7fe700 1 -- 172.21.15.87:0/2630417019 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c00dd70 con 0x7f144c107400 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1a7450 con 0x7f144c107400 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f144b7fe700 1 -- 172.21.15.87:0/2630417019 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0133f0 con 0x7f144c107400 2024-06-06T07:48:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.068+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a7940 con 0x7f144c107400 2024-06-06T07:48:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.070+0000 7f144b7fe700 1 -- 172.21.15.87:0/2630417019 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f143c01a020 con 0x7f144c107400 2024-06-06T07:48:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.071+0000 7f144b7fe700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 0x7f14340a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.071+0000 7f145209d700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 0x7f14340a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.071+0000 7f144b7fe700 1 -- 172.21.15.87:0/2630417019 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f143c0cf300 con 0x7f144c107400 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.071+0000 7f145209d700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 0x7f14340a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1444006fd0 tx=0x7f1444008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 Processor -- start 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 -- start start 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460967a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x559460962340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5594608b3810 con 0x559460967f60 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55946085b720 con 0x559460967640 2024-06-06T07:48:55.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145eb84b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594608af7d0 con 0x559460961f20 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145189c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145189c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33052/0 (socket says 172.21.15.87:33052) 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145189c700 1 -- 172.21.15.87:0/305380053 learned_addr learned my addr 172.21.15.87:0/305380053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.072+0000 7f145289e700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x559460962340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145209d700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460967a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 -- 172.21.15.87:0/305380053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 msgr2=0x559460962340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x559460962340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 -- 172.21.15.87:0/305380053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 msgr2=0x559460967a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460967a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 -- 172.21.15.87:0/305380053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559460851f50 con 0x559460967f60 2024-06-06T07:48:55.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145209d700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460967a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f145189c700 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1440009750 tx=0x7f1440005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2eb5c34285ffbc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f1448d70700 1 -- 172.21.15.87:0/305380053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440026070 con 0x559460967f60 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f1448d70700 1 -- 172.21.15.87:0/305380053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1440003e40 con 0x559460967f60 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.073+0000 7f1448d70700 1 -- 172.21.15.87:0/305380053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144000bb70 con 0x559460967f60 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- 172.21.15.87:0/305380053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 msgr2=0x5594609619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1440009750 tx=0x7f1440005e80 comp rx=0 tx=0).stop 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- 172.21.15.87:0/305380053 shutdown_connections 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460967a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x559460962340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 --2- 172.21.15.87:0/305380053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x5594609619e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- 172.21.15.87:0/305380053 >> 172.21.15.87:0/305380053 conn(0x55946088ccc0 msgr2=0x55946095c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- 172.21.15.87:0/305380053 shutdown_connections 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- 172.21.15.87:0/305380053 wait complete. 2024-06-06T07:48:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 -- start start 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 Processor -- start 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.074+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460961f20 0x55946096dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946096afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55946096e970 con 0x559460967f60 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559460853bf0 con 0x559460961f20 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55946096eaf0 con 0x559460967640 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 10 client.0 _ll_register_callbacks cb 0x55946084d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33066/0 (socket says 172.21.15.87:33066) 2024-06-06T07:48:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145189c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946096afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 -- 172.21.15.87:0/1584408564 learned_addr learned my addr 172.21.15.87:0/1584408564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145209d700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460961f20 0x55946096dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 -- 172.21.15.87:0/1584408564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 msgr2=0x55946096afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946096afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 -- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460961f20 msgr2=0x55946096dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460961f20 0x55946096dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 -- 172.21.15.87:0/1584408564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594608028d0 con 0x559460967f60 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145289e700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ccc80 tx=0x7f143c09ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16daf5e7a7212629 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f1431ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0bb2c0 con 0x559460967f60 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594608c0d10 con 0x559460967f60 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.075+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f143c09b460 con 0x559460967f60 2024-06-06T07:48:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f143c09b200 con 0x559460967f60 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559460979240 con 0x559460967f60 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5594608c0d10 con 0x559460967f60 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f143c09f120 con 0x559460967f60 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 20 client.15597 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x55946097dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 4 client.15597 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594608bb290 con 0x55946097b7d0 2024-06-06T07:48:55.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.076+0000 7f145209d700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x55946097dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.077+0000 7f1431ffb700 10 client.15597 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.077+0000 7f145209d700 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x55946097dbc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c61e367abd61c5 server_cookie=ff4deaed2bc1492b in_seq=0 out_seq=0 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.077+0000 7f1431ffb700 1 -- 172.21.15.87:0/1584408564 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5594608bb290 con 0x55946097b7d0 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.077+0000 7f1431ffb700 10 client.15597 handle_command_reply: tid=0 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.077+0000 7f145eb84b80 1 client.15597 shutdown 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 msgr2=0x55946096b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ccc80 tx=0x7f143c09ac50 comp rx=0 tx=0).stop 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 shutdown_connections 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x55946097dbc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f1431ffb700 0 client.15597 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460961f20 0x55946096dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x55946096b4e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1584408564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946096afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 >> 172.21.15.87:0/1584408564 conn(0x55946088ccc0 msgr2=0x5594609609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 shutdown_connections 2024-06-06T07:48:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 1 -- 172.21.15.87:0/1584408564 wait complete. 2024-06-06T07:48:55.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.078+0000 7f145eb84b80 20 client.15597 trim_cache size 0 max 16384 2024-06-06T07:48:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 Processor -- start 2024-06-06T07:48:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 -- start start 2024-06-06T07:48:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946095c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967f60 0x559460968a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1b3080 con 0x559460961f20 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1b0910 con 0x559460967f60 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145eb84b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1ae760 con 0x559460967640 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.098+0000 7f145209d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33068/0 (socket says 172.21.15.87:33068) 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 -- 172.21.15.87:0/1528500089 learned_addr learned my addr 172.21.15.87:0/1528500089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145289e700 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967f60 0x559460968a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145189c700 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946095c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 -- 172.21.15.87:0/1528500089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 msgr2=0x55946095c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946095c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 -- 172.21.15.87:0/1528500089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967f60 msgr2=0x559460968a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967f60 0x559460968a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 -- 172.21.15.87:0/1528500089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946096d8e0 con 0x559460961f20 2024-06-06T07:48:55.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f145209d700 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f144400d440 tx=0x7f1444013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52993cbd4a61e29a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f1433fff700 1 -- 172.21.15.87:0/1528500089 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144402f070 con 0x559460961f20 2024-06-06T07:48:55.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f1433fff700 1 -- 172.21.15.87:0/1528500089 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144400de70 con 0x559460961f20 2024-06-06T07:48:55.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.099+0000 7f1433fff700 1 -- 172.21.15.87:0/1528500089 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444022700 con 0x559460961f20 2024-06-06T07:48:55.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- 172.21.15.87:0/1528500089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 msgr2=0x5594609619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f144400d440 tx=0x7f1444013000 comp rx=0 tx=0).stop 2024-06-06T07:48:55.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- 172.21.15.87:0/1528500089 shutdown_connections 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967f60 0x559460968a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460961f20 0x5594609619e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1528500089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460967640 0x55946095c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- 172.21.15.87:0/1528500089 >> 172.21.15.87:0/1528500089 conn(0x55946088ccc0 msgr2=0x559460968340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- 172.21.15.87:0/1528500089 shutdown_connections 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- 172.21.15.87:0/1528500089 wait complete. 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 -- start start 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.100+0000 7f145eb84b80 1 Processor -- start 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x5594609831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460983720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1aedd0 con 0x559460967f60 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1b0240 con 0x559460967640 2024-06-06T07:48:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1af4a0 con 0x559460961f20 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 10 client.0 _ll_register_callbacks cb 0x55946084d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33084/0 (socket says 172.21.15.87:33084) 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 -- 172.21.15.87:0/1911050286 learned_addr learned my addr 172.21.15.87:0/1911050286 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145209d700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x5594609831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145189c700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460983720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 -- 172.21.15.87:0/1911050286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 msgr2=0x5594609831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x5594609831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 -- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 msgr2=0x559460983720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460983720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 -- 172.21.15.87:0/1911050286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594609e54f0 con 0x559460967f60 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145189c700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460983720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145289e700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f143c097b80 tx=0x7f143c0ccb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8272db087c06b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f1431ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0bb2b0 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f143c09b460 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.101+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144400de70 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f143c065050 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 10 client.15603 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594609e67e0 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144400de70 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f143c09a650 con 0x559460967f60 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:55.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 20 client.15603 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x5594609e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 4 client.15603 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559460735250 con 0x55946097b7d0 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.102+0000 7f145209d700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x5594609e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145209d700 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x5594609e8d70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34031c81207cf8f0 server_cookie=e50c9309a4e41a54 in_seq=0 out_seq=0 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f1431ffb700 10 client.15603 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f1431ffb700 1 -- 172.21.15.87:0/1911050286 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559460735250 con 0x55946097b7d0 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f1431ffb700 10 client.15603 handle_command_reply: tid=0 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 client.15603 shutdown 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 msgr2=0x559460983c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f143c097b80 tx=0x7f143c0ccb40 comp rx=0 tx=0).stop 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 shutdown_connections 2024-06-06T07:48:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55946097b7d0 0x5594609e8d70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559460967f60 0x559460983c60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559460961f20 0x5594609831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 --2- 172.21.15.87:0/1911050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559460967640 0x559460983720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 >> 172.21.15.87:0/1911050286 conn(0x55946088ccc0 msgr2=0x559460968a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f1431ffb700 0 client.15603 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 shutdown_connections 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 1 -- 172.21.15.87:0/1911050286 wait complete. 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.103+0000 7f145eb84b80 20 client.15603 trim_cache size 0 max 16384 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:48:55.109 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-06-06T07:48:55.110 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-06-06T07:48:55.110 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:48:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 msgr2=0x7f14340a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 0x7f14340a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1444006fd0 tx=0x7f1444008040 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 msgr2=0x7f144c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f143c007ee0 tx=0x7f143c00d3b0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 shutdown_connections 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f14340a2eb0 0x7f14340a5370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f144c107400 0x7f144c198370 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c101390 0x7f144c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 --2- 172.21.15.87:0/2630417019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f144c101d50 0x7f144c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 >> 172.21.15.87:0/2630417019 conn(0x7f144c078340 msgr2=0x7f144c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 shutdown_connections 2024-06-06T07:48:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:55.104+0000 7f1454301700 1 -- 172.21.15.87:0/2630417019 wait complete. 2024-06-06T07:48:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/2727507452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 msgr2=0x7f0f0810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/2727507452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f0810e860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009cd0 tx=0x7f0f0002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/2727507452 shutdown_connections 2024-06-06T07:48:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/2727507452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f0810e860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/2727507452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0806a180 0x7f0f0806a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/2727507452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f081079a0 0x7f0f08107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/2727507452 >> 172.21.15.87:0/2727507452 conn(0x7f0f08077680 msgr2=0x7f0f08077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/2727507452 shutdown_connections 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.329+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/2727507452 wait complete. 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 Processor -- start 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 -- start start 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0806a180 0x7f0f081981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f081079a0 0x7f0f0819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08111db0 con 0x7f0f0806ab40 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08111c30 con 0x7f0f0806a180 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0f6a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08111f30 con 0x7f0f081079a0 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0cc40640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0cc40640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33108/0 (socket says 172.21.15.87:33108) 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.330+0000 7f0f0cc40640 1 -- 172.21.15.87:0/1698655618 learned_addr learned my addr 172.21.15.87:0/1698655618 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 -- 172.21.15.87:0/1698655618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f081079a0 msgr2=0x7f0f0819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f081079a0 0x7f0f0819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 -- 172.21.15.87:0/1698655618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0806a180 msgr2=0x7f0f081981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0806a180 0x7f0f081981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 -- 172.21.15.87:0/1698655618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0819d190 con 0x7f0f0806ab40 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00d950 tx=0x7f0efc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f067fc640 1 -- 172.21.15.87:0/1698655618 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc017070 con 0x7f0f0806ab40 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f067fc640 1 -- 172.21.15.87:0/1698655618 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0efc009c30 con 0x7f0f0806ab40 2024-06-06T07:48:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0819d480 con 0x7f0f0806ab40 2024-06-06T07:48:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f067fc640 1 -- 172.21.15.87:0/1698655618 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00f6b0 con 0x7f0f0806ab40 2024-06-06T07:48:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.331+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a4d10 con 0x7f0f0806ab40 2024-06-06T07:48:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.333+0000 7f0f067fc640 1 -- 172.21.15.87:0/1698655618 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f0efc0109d0 con 0x7f0f0806ab40 2024-06-06T07:48:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.333+0000 7f0f123bb740 1 Processor -- start 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.333+0000 7f0f123bb740 1 -- start start 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259170 0x56343b259550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b266740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56343b1bd4f0 con 0x56343b259a90 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56343b15b870 con 0x56343b264350 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f123bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56343b1b94b0 con 0x56343b259170 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f067fc640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 0x7f0ef00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33120/0 (socket says 172.21.15.87:33120) 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 -- 172.21.15.87:0/119465263 learned_addr learned my addr 172.21.15.87:0/119465263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259170 0x56343b259550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 0x7f0ef00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f067fc640 1 -- 172.21.15.87:0/1698655618 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0efc0cf7d0 con 0x7f0f0806ab40 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b266740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 -- 172.21.15.87:0/119465263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259170 msgr2=0x56343b259550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259170 0x56343b259550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 -- 172.21.15.87:0/119465263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 msgr2=0x56343b266740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b266740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 -- 172.21.15.87:0/119465263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343b1779b0 con 0x56343b259a90 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0d441640 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800ba80 tx=0x7f0ef800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1585c30e2b7b256c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 0x7f0ef00a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009ca0 tx=0x7f0f000095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.334+0000 7f0eef7fe640 1 -- 172.21.15.87:0/119465263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8016070 con 0x56343b259a90 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0eef7fe640 1 -- 172.21.15.87:0/119465263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8005500 con 0x56343b259a90 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0eef7fe640 1 -- 172.21.15.87:0/119465263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8010460 con 0x56343b259a90 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 -- 172.21.15.87:0/119465263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 msgr2=0x56343b263e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800ba80 tx=0x7f0ef800bf40 comp rx=0 tx=0).stop 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 -- 172.21.15.87:0/119465263 shutdown_connections 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b266740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259a90 0x56343b263e10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 --2- 172.21.15.87:0/119465263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259170 0x56343b259550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 -- 172.21.15.87:0/119465263 >> 172.21.15.87:0/119465263 conn(0x56343b255680 msgr2=0x56343b255a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 -- 172.21.15.87:0/119465263 shutdown_connections 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.335+0000 7f0f123bb740 1 -- 172.21.15.87:0/119465263 wait complete. 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 -- start start 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 Processor -- start 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b2618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b261de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56343b266f80 con 0x56343b259170 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56343b146ae0 con 0x56343b264350 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56343b267100 con 0x56343b259a90 2024-06-06T07:48:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f123bb740 10 client.0 _ll_register_callbacks cb 0x56343b17f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33130/0 (socket says 172.21.15.87:33130) 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 -- 172.21.15.87:0/3832450995 learned_addr learned my addr 172.21.15.87:0/3832450995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b2618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 -- 172.21.15.87:0/3832450995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 msgr2=0x56343b2618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b2618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 -- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 msgr2=0x56343b261de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b261de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0d441640 1 -- 172.21.15.87:0/3832450995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343aed1210 con 0x56343b259170 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.336+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b2618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f0d441640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d770 tx=0x7f0ef80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ab936a3c0be85e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8016070 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0ef800dca0 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef8005500 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ef8004ee0 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56343af9d620 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8005500 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ef8022020 con 0x56343b259170 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 20 client.15615 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2744a0 0x56343b276890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 4 client.15615 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.337+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56343b1d0f10 con 0x56343b2744a0 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.338+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2744a0 0x56343b276890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.338+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2744a0 0x56343b276890 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3a41696252a3e0 server_cookie=c03fefa0ca1dbb95 in_seq=0 out_seq=0 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.338+0000 7f0eecff9640 10 client.15615 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0eecff9640 1 -- 172.21.15.87:0/3832450995 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56343b1d0f10 con 0x56343b2744a0 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0eecff9640 10 client.15615 handle_command_reply: tid=0 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 client.15615 shutdown 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 msgr2=0x56343b261360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d770 tx=0x7f0ef80049d0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 shutdown_connections 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2744a0 0x56343b276890 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b264350 0x56343b261de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b2618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3832450995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b261360 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0eecff9640 0 client.15615 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 >> 172.21.15.87:0/3832450995 conn(0x56343b255680 msgr2=0x56343b25b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 shutdown_connections 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 1 -- 172.21.15.87:0/3832450995 wait complete. 2024-06-06T07:48:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.339+0000 7f0f123bb740 20 client.15615 trim_cache size 0 max 16384 2024-06-06T07:48:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 Processor -- start 2024-06-06T07:48:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 -- start start 2024-06-06T07:48:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259170 0x56343b25b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b27bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081b1810 con 0x56343b264350 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081af0e0 con 0x56343b259170 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f123bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081acf30 con 0x56343b259a90 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33140/0 (socket says 172.21.15.87:33140) 2024-06-06T07:48:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 -- 172.21.15.87:0/3354352575 learned_addr learned my addr 172.21.15.87:0/3354352575 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b27bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0d441640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259170 0x56343b25b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 -- 172.21.15.87:0/3354352575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 msgr2=0x56343b27bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b27bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 -- 172.21.15.87:0/3354352575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259170 msgr2=0x56343b25b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259170 0x56343b25b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0dc42640 1 -- 172.21.15.87:0/3354352575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343b25f8f0 con 0x56343b264350 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.354+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b27bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:48:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009430 tx=0x7f0f00005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b0c5e952616edc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0eeeffd640 1 -- 172.21.15.87:0/3354352575 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0004e020 con 0x56343b264350 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0eeeffd640 1 -- 172.21.15.87:0/3354352575 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f00036070 con 0x56343b264350 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0eeeffd640 1 -- 172.21.15.87:0/3354352575 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00030c40 con 0x56343b264350 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 -- 172.21.15.87:0/3354352575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 msgr2=0x56343b27c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009430 tx=0x7f0f00005d80 comp rx=0 tx=0).stop 2024-06-06T07:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 -- 172.21.15.87:0/3354352575 shutdown_connections 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b264350 0x56343b27c4f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b259a90 0x56343b27bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 --2- 172.21.15.87:0/3354352575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259170 0x56343b25b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 -- 172.21.15.87:0/3354352575 >> 172.21.15.87:0/3354352575 conn(0x56343b255680 msgr2=0x56343b25ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 -- 172.21.15.87:0/3354352575 shutdown_connections 2024-06-06T07:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.355+0000 7f0f123bb740 1 -- 172.21.15.87:0/3354352575 wait complete. 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 -- start start 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 Processor -- start 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 0x56343b26c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 0x56343b25da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ad5a0 con 0x56343b259170 2024-06-06T07:48:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081aea10 con 0x56343b259a90 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081adc70 con 0x56343b264350 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f123bb740 10 client.0 _ll_register_callbacks cb 0x7f0f00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33154/0 (socket says 172.21.15.87:33154) 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 -- 172.21.15.87:0/2359932495 learned_addr learned my addr 172.21.15.87:0/2359932495 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 0x56343b26c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 -- 172.21.15.87:0/2359932495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 msgr2=0x56343b25da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 0x56343b25da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 0x56343b25da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 -- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 msgr2=0x56343b26c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.356+0000 7f0f0d441640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 0x56343b26c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f0d441640 1 -- 172.21.15.87:0/2359932495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343b25df60 con 0x56343b259170 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 0x56343b26c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f0dc42640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 0x56343b25da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f0d441640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8003dc0 tx=0x7f0ef8003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39324439419bafd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8016040 con 0x56343b259170 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0ef8004040 con 0x56343b259170 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f00036070 con 0x56343b259170 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ef8004240 con 0x56343b259170 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56343b2de710 con 0x56343b259170 2024-06-06T07:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f00036070 con 0x56343b259170 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ef8022050 con 0x56343b259170 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 20 client.15621 populate_metadata read hostname 'smithi087' 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2decc0 0x56343b2e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 4 client.15621 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.357+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56343b1cac20 con 0x56343b2decc0 2024-06-06T07:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.358+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2decc0 0x56343b2e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.358+0000 7f0eecff9640 10 client.15621 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.358+0000 7f0f0cc40640 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2decc0 0x56343b2e10e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d281cc1344f6608e server_cookie=4fe123f14c7a50d8 in_seq=0 out_seq=0 2024-06-06T07:48:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0eecff9640 1 -- 172.21.15.87:0/2359932495 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56343b1cac20 con 0x56343b2decc0 2024-06-06T07:48:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0eecff9640 10 client.15621 handle_command_reply: tid=0 2024-06-06T07:48:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 client.15621 shutdown 2024-06-06T07:48:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 msgr2=0x56343b26c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8003dc0 tx=0x7f0ef8003df0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 shutdown_connections 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56343b2decc0 0x56343b2e10e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56343b264350 0x56343b25da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0eecff9640 0 client.15621 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56343b259a90 0x56343b26c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 --2- 172.21.15.87:0/2359932495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56343b259170 0x56343b26c350 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 >> 172.21.15.87:0/2359932495 conn(0x56343b255680 msgr2=0x56343b25b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 shutdown_connections 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 1 -- 172.21.15.87:0/2359932495 wait complete. 2024-06-06T07:48:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.364+0000 7f0f123bb740 20 client.15621 trim_cache size 0 max 16384 2024-06-06T07:48:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:48:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:48:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 msgr2=0x7f0ef00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 0x7f0ef00a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009ca0 tx=0x7f0f000095c0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 msgr2=0x7f0f08198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00d950 tx=0x7f0efc00de10 comp rx=0 tx=0).stop 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 shutdown_connections 2024-06-06T07:48:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0ef00a2ec0 0x7f0ef00a5380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f081079a0 0x7f0f0819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f0806ab40 0x7f0f08198730 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 --2- 172.21.15.87:0/1698655618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0806a180 0x7f0f081981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 >> 172.21.15.87:0/1698655618 conn(0x7f0f08077680 msgr2=0x7f0f080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 shutdown_connections 2024-06-06T07:48:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:48:55.365+0000 7f0f0f6a5640 1 -- 172.21.15.87:0/1698655618 wait complete. 2024-06-06T07:48:55.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:48:55.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:48:56.090 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:56.349 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:56 smithi087 ceph-mon[32276]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:48:56.358 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:56 smithi181 ceph-mon[36790]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:48:56.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:56 smithi195 ceph-mon[37040]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:48:57.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 -- 172.21.15.87:0/1499132101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c341084e0 msgr2=0x7f2c341088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:57.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 --2- 172.21.15.87:0/1499132101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c341084e0 0x7f2c341088c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2c300099c0 tx=0x7f2c30009cd0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 -- 172.21.15.87:0/1499132101 shutdown_connections 2024-06-06T07:48:57.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 --2- 172.21.15.87:0/1499132101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c3406ab10 0x7f2c3410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 --2- 172.21.15.87:0/1499132101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c341084e0 0x7f2c341088c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 --2- 172.21.15.87:0/1499132101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 0x7f2c3406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 -- 172.21.15.87:0/1499132101 >> 172.21.15.87:0/1499132101 conn(0x7f2c340782c0 msgr2=0x7f2c340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 -- 172.21.15.87:0/1499132101 shutdown_connections 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 -- 172.21.15.87:0/1499132101 wait complete. 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.879+0000 7f2c3c632700 1 Processor -- start 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 -- start start 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 0x7f2c34101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c341084e0 0x7f2c34101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:57.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34112740 con 0x7f2c3406ab10 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341125c0 con 0x7f2c3406a170 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341128c0 con 0x7f2c341084e0 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56072/0 (socket says 172.21.15.87:56072) 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 -- 172.21.15.87:0/3506773956 learned_addr learned my addr 172.21.15.87:0/3506773956 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3abcf700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c341084e0 0x7f2c34101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3a3ce700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 0x7f2c34101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 -- 172.21.15.87:0/3506773956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c341084e0 msgr2=0x7f2c34101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c341084e0 0x7f2c34101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 -- 172.21.15.87:0/3506773956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 msgr2=0x7f2c34101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 0x7f2c34101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 -- 172.21.15.87:0/3506773956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c341a7110 con 0x7f2c3406ab10 2024-06-06T07:48:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c39bcd700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400d980 tx=0x7f2c2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24016070 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c341a7400 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.880+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c24004500 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.881+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24004d10 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.881+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c341a7990 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.881+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c34069180 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.883+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f2c2400bbf0 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.883+0000 7f2c2b7fe700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 0x7f2c200a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.883+0000 7f2c3a3ce700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 0x7f2c200a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.883+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f2c240cfe90 con 0x7f2c3406ab10 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.884+0000 7f2c3a3ce700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 0x7f2c200a5370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000c00 tx=0x7f2c30005fd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:48:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:57.887+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c2401f020 con 0x7f2c3406ab10 2024-06-06T07:48:58.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.127+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c3402dca0 con 0x7f2c3406ab10 2024-06-06T07:48:58.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.128+0000 7f2c2b7fe700 1 -- 172.21.15.87:0/3506773956 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f2c2409c490 con 0x7f2c3406ab10 2024-06-06T07:48:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:48:58.129 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:48:58.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 msgr2=0x7f2c200a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 0x7f2c200a5370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000c00 tx=0x7f2c30005fd0 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 msgr2=0x7f2c341019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400d980 tx=0x7f2c2400dd40 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 shutdown_connections 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c3406a170 0x7f2c34101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2c200a2eb0 0x7f2c200a5370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c341084e0 0x7f2c34101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 --2- 172.21.15.87:0/3506773956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c3406ab10 0x7f2c341019a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 >> 172.21.15.87:0/3506773956 conn(0x7f2c340782c0 msgr2=0x7f2c340ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.130+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 shutdown_connections 2024-06-06T07:48:58.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:48:58.131+0000 7f2c3c632700 1 -- 172.21.15.87:0/3506773956 wait complete. 2024-06-06T07:48:58.135 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:48:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:58 smithi087 ceph-mon[32276]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-06T07:48:58.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:58 smithi195 ceph-mon[37040]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-06T07:48:58.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:58 smithi181 ceph-mon[36790]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-06T07:48:58.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:48:59.087 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:48:59.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:48:59 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3506773956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:59.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:48:59 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3506773956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:48:59.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:48:59 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3506773956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:00.355 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-mon[32276]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:00.356 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:49:00.128+0000 7f57956bd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:49:00.356 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:49:00.128+0000 7f57956bd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:49:00.356 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:49:00.128+0000 7f57956bd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:49:00.356 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:49:00.128+0000 7f57956bd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:49:00.356 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 07:49:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T07:49:00.128+0000 7f57956bd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:49:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:00 smithi195 ceph-mon[37040]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:00 smithi181 ceph-mon[36790]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 -- 172.21.15.87:0/181199539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc103120 msgr2=0x7fabbc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 --2- 172.21.15.87:0/181199539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc103120 0x7fabbc107130 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fabac0099c0 tx=0x7fabac009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 -- 172.21.15.87:0/181199539 shutdown_connections 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 --2- 172.21.15.87:0/181199539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc102770 0x7fabbc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 --2- 172.21.15.87:0/181199539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc107860 0x7fabbc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 --2- 172.21.15.87:0/181199539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc103120 0x7fabbc107130 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 -- 172.21.15.87:0/181199539 >> 172.21.15.87:0/181199539 conn(0x7fabbc079600 msgr2=0x7fabbc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 -- 172.21.15.87:0/181199539 shutdown_connections 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.905+0000 7fabc458f700 1 -- 172.21.15.87:0/181199539 wait complete. 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 Processor -- start 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 -- start start 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc103120 0x7fabbc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc107860 0x7fabbc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc112790 con 0x7fabbc102770 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc112610 con 0x7fabbc107860 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc458f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc112910 con 0x7fabbc103120 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56104/0 (socket says 172.21.15.87:56104) 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.906+0000 7fabc232b700 1 -- 172.21.15.87:0/3299556766 learned_addr learned my addr 172.21.15.87:0/3299556766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc107860 0x7fabbc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc103120 0x7fabbc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 -- 172.21.15.87:0/3299556766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc103120 msgr2=0x7fabbc074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc103120 0x7fabbc074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 -- 172.21.15.87:0/3299556766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc107860 msgr2=0x7fabbc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc107860 0x7fabbc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 -- 172.21.15.87:0/3299556766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc1a7090 con 0x7fabbc102770 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc232b700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c9b0 tx=0x7fabb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabb37fe700 1 -- 172.21.15.87:0/3299556766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8007b00 con 0x7fabbc102770 2024-06-06T07:49:00.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabb37fe700 1 -- 172.21.15.87:0/3299556766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8007ca0 con 0x7fabbc102770 2024-06-06T07:49:00.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1a7380 con 0x7fabbc102770 2024-06-06T07:49:00.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabb37fe700 1 -- 172.21.15.87:0/3299556766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8018780 con 0x7fabbc102770 2024-06-06T07:49:00.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.907+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a7890 con 0x7fabbc102770 2024-06-06T07:49:00.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.909+0000 7fabb37fe700 1 -- 172.21.15.87:0/3299556766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fabb800f450 con 0x7fabbc102770 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.910+0000 7fabb37fe700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 0x7faba80a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.910+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 0x7faba80a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.910+0000 7fabb37fe700 1 -- 172.21.15.87:0/3299556766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fabb80cfb70 con 0x7fabbc102770 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.910+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 0x7faba80a5370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabbc076220 tx=0x7fabac005fd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 Processor -- start 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 -- start start 2024-06-06T07:49:00.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 0x55eabe5d9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eabe52a810 con 0x55eabe5de640 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eabe4d2720 con 0x55eabe5d8f20 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabcee12b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eabe5267d0 con 0x55eabe5def60 2024-06-06T07:49:00.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56120/0 (socket says 172.21.15.87:56120) 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.911+0000 7fabc232b700 1 -- 172.21.15.87:0/3871828449 learned_addr learned my addr 172.21.15.87:0/3871828449 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 0x55eabe5d9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 -- 172.21.15.87:0/3871828449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 msgr2=0x55eabe5d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5d89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 -- 172.21.15.87:0/3871828449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 msgr2=0x55eabe5d9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 0x55eabe5d9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 -- 172.21.15.87:0/3871828449 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabe4c8f50 con 0x55eabe5de640 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 0x55eabe5d9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:00.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabc232b700 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fabb8000c00 tx=0x7fabb809b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcee236bb935f61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabb0ff9700 1 -- 172.21.15.87:0/3871828449 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80a46e0 con 0x55eabe5de640 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabb0ff9700 1 -- 172.21.15.87:0/3871828449 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8099d70 con 0x55eabe5de640 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.912+0000 7fabb0ff9700 1 -- 172.21.15.87:0/3871828449 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80a4250 con 0x55eabe5de640 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- 172.21.15.87:0/3871828449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 msgr2=0x55eabe5dea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fabb8000c00 tx=0x7fabb809b970 comp rx=0 tx=0).stop 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- 172.21.15.87:0/3871828449 shutdown_connections 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5d8f20 0x55eabe5d9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5de640 0x55eabe5dea20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3871828449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- 172.21.15.87:0/3871828449 >> 172.21.15.87:0/3871828449 conn(0x55eabe503cc0 msgr2=0x55eabe5d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- 172.21.15.87:0/3871828449 shutdown_connections 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- 172.21.15.87:0/3871828449 wait complete. 2024-06-06T07:49:00.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- start start 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 Processor -- start 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eabe5e59d0 con 0x55eabe5def60 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eabe4cabf0 con 0x55eabe5de640 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.913+0000 7fabcee12b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eabe5e5b50 con 0x55eabe5d8f20 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabcee12b80 10 client.0 _ll_register_callbacks cb 0x55eabe4c4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56132/0 (socket says 172.21.15.87:56132) 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc232b700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 -- 172.21.15.87:0/2674553513 learned_addr learned my addr 172.21.15.87:0/2674553513 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 -- 172.21.15.87:0/2674553513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 msgr2=0x55eabe5e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 -- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 msgr2=0x55eabe5e20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5e20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 -- 172.21.15.87:0/2674553513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabe4798d0 con 0x55eabe5def60 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fabb4008ad0 tx=0x7fabb4008de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ee02db6c4485b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7faba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004bb0 con 0x55eabe5def60 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fabb4004d50 con 0x55eabe5def60 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb8099d70 con 0x55eabe5def60 2024-06-06T07:49:00.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.914+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabb4022050 con 0x55eabe5def60 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 10 client.15645 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eabe5f03b0 con 0x55eabe5def60 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8099d70 con 0x55eabe5def60 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fabb40159b0 con 0x55eabe5def60 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 10 client.15645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 20 client.15645 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe5f29f0 0x55eabe5f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 4 client.15645 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:00.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eabe537d10 con 0x55eabe5f29f0 2024-06-06T07:49:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.915+0000 7fabc232b700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe5f29f0 0x55eabe5f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.916+0000 7fabc232b700 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe5f29f0 0x55eabe5f4e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9461d3e2f3edeed server_cookie=fe3cb19c8c72854e in_seq=0 out_seq=0 2024-06-06T07:49:00.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.916+0000 7faba5ffb700 10 client.15645 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:00.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.916+0000 7faba5ffb700 1 -- 172.21.15.87:0/2674553513 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eabe537d10 con 0x55eabe5f29f0 2024-06-06T07:49:00.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.916+0000 7faba5ffb700 10 client.15645 handle_command_reply: tid=0 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.916+0000 7fabcee12b80 1 client.15645 shutdown 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 msgr2=0x55eabe5e2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fabb4008ad0 tx=0x7fabb4008de0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 shutdown_connections 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe5f29f0 0x55eabe5f4e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5e2620 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7faba5ffb700 0 client.15645 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 --2- 172.21.15.87:0/2674553513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 >> 172.21.15.87:0/2674553513 conn(0x55eabe503cc0 msgr2=0x55eabe5d79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 shutdown_connections 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 1 -- 172.21.15.87:0/2674553513 wait complete. 2024-06-06T07:49:00.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.917+0000 7fabcee12b80 20 client.15645 trim_cache size 0 max 16384 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 Processor -- start 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 -- start start 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b31f0 con 0x55eabe5d8f20 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b0a80 con 0x55eabe5de640 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabcee12b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1ae8d0 con 0x55eabe5def60 2024-06-06T07:49:00.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56138/0 (socket says 172.21.15.87:56138) 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 -- 172.21.15.87:0/224089597 learned_addr learned my addr 172.21.15.87:0/224089597 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 -- 172.21.15.87:0/224089597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 msgr2=0x55eabe5e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 -- 172.21.15.87:0/224089597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 msgr2=0x55eabe5eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 -- 172.21.15.87:0/224089597 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabe5f5470 con 0x55eabe5d8f20 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.932+0000 7fabc232b700 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fabb80cd4f0 tx=0x7fabb8097d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f879cc8226dfb63c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7faba7fff700 1 -- 172.21.15.87:0/224089597 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb800de00 con 0x55eabe5d8f20 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7faba7fff700 1 -- 172.21.15.87:0/224089597 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8099d70 con 0x55eabe5d8f20 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7faba7fff700 1 -- 172.21.15.87:0/224089597 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb809bf30 con 0x55eabe5d8f20 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 -- 172.21.15.87:0/224089597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 msgr2=0x55eabe5d89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fabb80cd4f0 tx=0x7fabb8097d10 comp rx=0 tx=0).stop 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 -- 172.21.15.87:0/224089597 shutdown_connections 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5def60 0x55eabe5e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5d8f20 0x55eabe5d89e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 --2- 172.21.15.87:0/224089597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 -- 172.21.15.87:0/224089597 >> 172.21.15.87:0/224089597 conn(0x55eabe503cc0 msgr2=0x55eabe5df340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 -- 172.21.15.87:0/224089597 shutdown_connections 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.933+0000 7fabcee12b80 1 -- 172.21.15.87:0/224089597 wait complete. 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 -- start start 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 Processor -- start 2024-06-06T07:49:00.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1aef40 con 0x55eabe5def60 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b03b0 con 0x55eabe5de640 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1af610 con 0x55eabe5d8f20 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabcee12b80 10 client.0 _ll_register_callbacks cb 0x55eabe4c4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabc2b2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabc232b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabc2b2c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56140/0 (socket says 172.21.15.87:56140) 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabc2b2c700 1 -- 172.21.15.87:0/3357087406 learned_addr learned my addr 172.21.15.87:0/3357087406 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:00.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.934+0000 7fabc1b2a700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 -- 172.21.15.87:0/3357087406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 msgr2=0x55eabe5eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 -- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 msgr2=0x55eabe5fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 -- 172.21.15.87:0/3357087406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabe5fae60 con 0x55eabe5def60 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabc2b2c700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabb4017040 tx=0x7fabb4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c946469351b0e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7faba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb40145e0 con 0x55eabe5def60 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fabb4005c60 con 0x55eabe5def60 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb8099d70 con 0x55eabe5def60 2024-06-06T07:49:00.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabb4034050 con 0x55eabe5def60 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabcee12b80 10 client.15651 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eabe65d6f0 con 0x55eabe5def60 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.935+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8099d70 con 0x55eabe5def60 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fabb400f8a0 con 0x55eabe5def60 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 20 client.15651 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe65de20 0x55eabe660240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 4 client.15651 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eabe532290 con 0x55eabe65de20 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabc232b700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe65de20 0x55eabe660240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7faba5ffb700 10 client.15651 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:00.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.936+0000 7fabc232b700 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe65de20 0x55eabe660240 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38eb8d65fd72cf2b server_cookie=b40f980b07525a11 in_seq=0 out_seq=0 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7faba5ffb700 1 -- 172.21.15.87:0/3357087406 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eabe532290 con 0x55eabe65de20 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7faba5ffb700 10 client.15651 handle_command_reply: tid=0 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 client.15651 shutdown 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 msgr2=0x55eabe5fa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabb4017040 tx=0x7fabb4000f40 comp rx=0 tx=0).stop 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 shutdown_connections 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55eabe65de20 0x55eabe660240 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eabe5def60 0x55eabe5fa5f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eabe5d8f20 0x55eabe5eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 --2- 172.21.15.87:0/3357087406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eabe5de640 0x55eabe5fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 >> 172.21.15.87:0/3357087406 conn(0x55eabe503cc0 msgr2=0x55eabe5e41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7faba5ffb700 0 client.15651 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 shutdown_connections 2024-06-06T07:49:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 1 -- 172.21.15.87:0/3357087406 wait complete. 2024-06-06T07:49:00.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabcee12b80 20 client.15651 trim_cache size 0 max 16384 2024-06-06T07:49:00.945 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 msgr2=0x7faba80a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 0x7faba80a5370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabbc076220 tx=0x7fabac005fd0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 msgr2=0x7fabbc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.937+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c9b0 tx=0x7fabb800cd70 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 shutdown_connections 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc107860 0x7fabbc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7faba80a2eb0 0x7faba80a5370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabbc102770 0x7fabbc074790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 --2- 172.21.15.87:0/3299556766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabbc103120 0x7fabbc074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 >> 172.21.15.87:0/3299556766 conn(0x7fabbc079600 msgr2=0x7fabbc100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:00.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 shutdown_connections 2024-06-06T07:49:00.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:00.938+0000 7fabc458f700 1 -- 172.21.15.87:0/3299556766 wait complete. 2024-06-06T07:49:01.641 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:49:01.879 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:02.259 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:02 smithi087 ceph-mon[32276]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:02.259 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:02 smithi087 ceph-mon[32276]: mgrmap e25: x(active, since 60s), standbys: y, z 2024-06-06T07:49:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:02 smithi195 ceph-mon[37040]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:02 smithi195 ceph-mon[37040]: mgrmap e25: x(active, since 60s), standbys: y, z 2024-06-06T07:49:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:02 smithi181 ceph-mon[36790]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T07:49:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:02 smithi181 ceph-mon[36790]: mgrmap e25: x(active, since 60s), standbys: y, z 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 -- 172.21.15.87:0/1633974783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4108520 msgr2=0x7f27e4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 --2- 172.21.15.87:0/1633974783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4108520 0x7f27e4108900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009990 tx=0x7f27cc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 -- 172.21.15.87:0/1633974783 shutdown_connections 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 --2- 172.21.15.87:0/1633974783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4102ed0 0x7f27e410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 --2- 172.21.15.87:0/1633974783 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4108520 0x7f27e4108900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 --2- 172.21.15.87:0/1633974783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102510 0x7f27e4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 -- 172.21.15.87:0/1633974783 >> 172.21.15.87:0/1633974783 conn(0x7f27e4079560 msgr2=0x7f27e4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 -- 172.21.15.87:0/1633974783 shutdown_connections 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.695+0000 7f27e97bc700 1 -- 172.21.15.87:0/1633974783 wait complete. 2024-06-06T07:49:03.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 Processor -- start 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 -- start start 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102ed0 0x7f27e4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4108520 0x7f27e4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41127b0 con 0x7f27e4102510 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112630 con 0x7f27e4108520 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.696+0000 7f27e97bc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112930 con 0x7f27e4102ed0 2024-06-06T07:49:03.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56168/0 (socket says 172.21.15.87:56168) 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 -- 172.21.15.87:0/319008960 learned_addr learned my addr 172.21.15.87:0/319008960 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e37fe700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4108520 0x7f27e4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e27fc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102ed0 0x7f27e4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 -- 172.21.15.87:0/319008960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102ed0 msgr2=0x7f27e4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102ed0 0x7f27e4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 -- 172.21.15.87:0/319008960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4108520 msgr2=0x7f27e4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4108520 0x7f27e4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 -- 172.21.15.87:0/319008960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41a70b0 con 0x7f27e4102510 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27dbfff700 1 -- 172.21.15.87:0/319008960 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc027070 con 0x7f27e4102510 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27dbfff700 1 -- 172.21.15.87:0/319008960 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc003e40 con 0x7f27e4102510 2024-06-06T07:49:03.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e41a7340 con 0x7f27e4102510 2024-06-06T07:49:03.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27dbfff700 1 -- 172.21.15.87:0/319008960 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc022d10 con 0x7f27e4102510 2024-06-06T07:49:03.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.697+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a7790 con 0x7f27e4102510 2024-06-06T07:49:03.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.700+0000 7f27dbfff700 1 -- 172.21.15.87:0/319008960 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f27cc02a070 con 0x7f27e4102510 2024-06-06T07:49:03.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.700+0000 7f27dbfff700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 0x7f27d00a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.700+0000 7f27e27fc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 0x7f27d00a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.700+0000 7f27dbfff700 1 -- 172.21.15.87:0/319008960 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f27cc0d0900 con 0x7f27e4102510 2024-06-06T07:49:03.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27e27fc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 0x7f27d00a5400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27e4079b60 tx=0x7f27d400b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 Processor -- start 2024-06-06T07:49:03.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 -- start start 2024-06-06T07:49:03.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f02340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4e53810 con 0x55c6e4f07640 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4dfb720 con 0x55c6e4f01f20 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.701+0000 7f27f403fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4e4f7d0 con 0x55c6e4f07f60 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56178/0 (socket says 172.21.15.87:56178) 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4125698570 learned_addr learned my addr 172.21.15.87:0/4125698570 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e37fe700 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f02340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e27fc700 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4125698570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 msgr2=0x55c6e4f019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4125698570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 msgr2=0x55c6e4f02340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f02340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4125698570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e4df1f50 con 0x55c6e4f07640 2024-06-06T07:49:03.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0cdb30 tx=0x7f27cc09c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9871877c2985a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27d97fa700 1 -- 172.21.15.87:0/4125698570 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0a5480 con 0x55c6e4f07640 2024-06-06T07:49:03.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27d97fa700 1 -- 172.21.15.87:0/4125698570 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc09aba0 con 0x55c6e4f07640 2024-06-06T07:49:03.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.702+0000 7f27d97fa700 1 -- 172.21.15.87:0/4125698570 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc004150 con 0x55c6e4f07640 2024-06-06T07:49:03.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- 172.21.15.87:0/4125698570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 msgr2=0x55c6e4f07a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0cdb30 tx=0x7f27cc09c2f0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- 172.21.15.87:0/4125698570 shutdown_connections 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f02340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f07a20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4125698570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- 172.21.15.87:0/4125698570 >> 172.21.15.87:0/4125698570 conn(0x55c6e4e2ccc0 msgr2=0x55c6e4efc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- 172.21.15.87:0/4125698570 shutdown_connections 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- 172.21.15.87:0/4125698570 wait complete. 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.703+0000 7f27f403fb80 1 -- start start 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 Processor -- start 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4f0e6f0 con 0x55c6e4f01f20 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4df3bf0 con 0x55c6e4f07640 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4f0e870 con 0x55c6e4f07f60 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27f403fb80 10 client.0 _ll_register_callbacks cb 0x55c6e4ded920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56186/0 (socket says 172.21.15.87:56186) 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4145412273 learned_addr learned my addr 172.21.15.87:0/4145412273 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e27fc700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e37fe700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 msgr2=0x55c6e4f0b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f0b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4145412273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 msgr2=0x55c6e4f0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 -- 172.21.15.87:0/4145412273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e4da28d0 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e27fc700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0227e0 tx=0x7f27cc09adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0155604515d019 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.704+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0ba060 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f27cc0870b0 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27cc09aba0 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27cc0b4030 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 10 client.15663 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6e4f0e6f0 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc09cb00 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27cc0a4ff0 con 0x55c6e4f01f20 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 10 client.15663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 20 client.15663 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f1b320 0x55c6e4f1d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 4 client.15663 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.705+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f27cc09aba0 con 0x55c6e4f1b320 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.706+0000 7f27e27fc700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f1b320 0x55c6e4f1d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.706+0000 7f27ca7fc700 10 client.15663 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.706+0000 7f27e27fc700 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f1b320 0x55c6e4f1d7a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fb6e68022d47ff5 server_cookie=49d8645513e5d1ea in_seq=0 out_seq=0 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27ca7fc700 1 -- 172.21.15.87:0/4145412273 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f27cc09aba0 con 0x55c6e4f1b320 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27ca7fc700 10 client.15663 handle_command_reply: tid=0 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 client.15663 shutdown 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 msgr2=0x55c6e4f0dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0227e0 tx=0x7f27cc09adb0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 shutdown_connections 2024-06-06T07:49:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f0b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f0dc70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f1b320 0x55c6e4f1d7a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 --2- 172.21.15.87:0/4145412273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27ca7fc700 0 client.15663 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 >> 172.21.15.87:0/4145412273 conn(0x55c6e4e2ccc0 msgr2=0x55c6e4efc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 shutdown_connections 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 1 -- 172.21.15.87:0/4145412273 wait complete. 2024-06-06T07:49:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.707+0000 7f27f403fb80 20 client.15663 trim_cache size 0 max 16384 2024-06-06T07:49:03.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 Processor -- start 2024-06-06T07:49:03.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 -- start start 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41b2ff0 con 0x55c6e4f07640 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41b0880 con 0x55c6e4f01f20 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27f403fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41ae6d0 con 0x55c6e4f07f60 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56190/0 (socket says 172.21.15.87:56190) 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 -- 172.21.15.87:0/1369618064 learned_addr learned my addr 172.21.15.87:0/1369618064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e37fe700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 -- 172.21.15.87:0/1369618064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 msgr2=0x55c6e4f23070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 -- 172.21.15.87:0/1369618064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 msgr2=0x55c6e4f019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e27fc700 1 -- 172.21.15.87:0/1369618064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e4f0d8b0 con 0x55c6e4f07640 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.722+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27e27fc700 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27d40046e0 tx=0x7f27d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c4e147fe324707 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27d8ff9700 1 -- 172.21.15.87:0/1369618064 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4020110 con 0x55c6e4f07640 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27d8ff9700 1 -- 172.21.15.87:0/1369618064 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d4004d10 con 0x55c6e4f07640 2024-06-06T07:49:03.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27d8ff9700 1 -- 172.21.15.87:0/1369618064 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d40207a0 con 0x55c6e4f07640 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 -- 172.21.15.87:0/1369618064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 msgr2=0x55c6e4f08a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27d40046e0 tx=0x7f27d4013000 comp rx=0 tx=0).stop 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 -- 172.21.15.87:0/1369618064 shutdown_connections 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f01f20 0x55c6e4f019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 --2- 172.21.15.87:0/1369618064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f07640 0x55c6e4f08a50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 -- 172.21.15.87:0/1369618064 >> 172.21.15.87:0/1369618064 conn(0x55c6e4e2ccc0 msgr2=0x55c6e4f08340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 -- 172.21.15.87:0/1369618064 shutdown_connections 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.723+0000 7f27f403fb80 1 -- 172.21.15.87:0/1369618064 wait complete. 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 -- start start 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 Processor -- start 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41aed40 con 0x55c6e4f01f20 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41b01b0 con 0x55c6e4f07640 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41af410 con 0x55c6e4f07f60 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27f403fb80 10 client.0 _ll_register_callbacks cb 0x55c6e4ded920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56202/0 (socket says 172.21.15.87:56202) 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27e2ffd700 1 -- 172.21.15.87:0/2097671515 learned_addr learned my addr 172.21.15.87:0/2097671515 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.724+0000 7f27e27fc700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e37fe700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 -- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 msgr2=0x55c6e4f23020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 -- 172.21.15.87:0/2097671515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 msgr2=0x55c6e4f22aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f22aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 -- 172.21.15.87:0/2097671515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e4f23860 con 0x55c6e4f01f20 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27e2ffd700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09ab50 tx=0x7f27cc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7831be3232b695a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0ba070 con 0x55c6e4f01f20 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f27cc0870b0 con 0x55c6e4f01f20 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d4004d10 con 0x55c6e4f01f20 2024-06-06T07:49:03.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27cc09cfa0 con 0x55c6e4f01f20 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27f403fb80 10 client.15669 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6e4f23860 con 0x55c6e4f01f20 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d4004d10 con 0x55c6e4f01f20 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27cc09cb00 con 0x55c6e4f01f20 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27f403fb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.725+0000 7f27f403fb80 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 20 client.15669 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f86380 0x55c6e4f887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 4 client.15669 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6e4e60d10 con 0x55c6e4f86380 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27e27fc700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f86380 0x55c6e4f887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27ca7fc700 10 client.15669 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27e27fc700 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f86380 0x55c6e4f887a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a91beef92d5f39a1 server_cookie=8dd4c4135fd9052a in_seq=0 out_seq=0 2024-06-06T07:49:03.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27ca7fc700 1 -- 172.21.15.87:0/2097671515 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6e4e60d10 con 0x55c6e4f86380 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27ca7fc700 10 client.15669 handle_command_reply: tid=0 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 1 client.15669 shutdown 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 msgr2=0x55c6e4f14310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.726+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09ab50 tx=0x7f27cc009310 comp rx=0 tx=0).stop 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 shutdown_connections 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c6e4f07f60 0x55c6e4f23020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c6e4f01f20 0x55c6e4f14310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55c6e4f86380 0x55c6e4f887a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 --2- 172.21.15.87:0/2097671515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6e4f07640 0x55c6e4f22aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27ca7fc700 0 client.15669 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 >> 172.21.15.87:0/2097671515 conn(0x55c6e4e2ccc0 msgr2=0x55c6e4ef5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 shutdown_connections 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 1 -- 172.21.15.87:0/2097671515 wait complete. 2024-06-06T07:49:03.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27f403fb80 20 client.15669 trim_cache size 0 max 16384 2024-06-06T07:49:03.733 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 msgr2=0x7f27d00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 0x7f27d00a5400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27e4079b60 tx=0x7f27d400b3c0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 msgr2=0x7f27e4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc0047b0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 shutdown_connections 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4108520 0x7f27e4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27d00a2f40 0x7f27d00a5400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e4102510 0x7f27e4074790 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 --2- 172.21.15.87:0/319008960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27e4102ed0 0x7f27e4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 >> 172.21.15.87:0/319008960 conn(0x7f27e4079560 msgr2=0x7f27e4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.727+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 shutdown_connections 2024-06-06T07:49:03.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:03.728+0000 7f27e97bc700 1 -- 172.21.15.87:0/319008960 wait complete. 2024-06-06T07:49:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:04 smithi087 ceph-mon[32276]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:04 smithi195 ceph-mon[37040]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:04.550 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:49:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:04 smithi181 ceph-mon[36790]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:06.014 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.4 2024-06-06T07:49:06.014 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:49:06.335 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:06 smithi087 ceph-mon[32276]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-06-06T07:49:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:06 smithi195 ceph-mon[37040]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-06-06T07:49:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:06 smithi181 ceph-mon[36790]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-06-06T07:49:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 -- 172.21.15.87:0/3644794361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0101440 msgr2=0x7f04a010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 --2- 172.21.15.87:0/3644794361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0101440 0x7f04a010f330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f049800b0a0 tx=0x7f049800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 -- 172.21.15.87:0/3644794361 shutdown_connections 2024-06-06T07:49:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 --2- 172.21.15.87:0/3644794361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0108510 0x7f04a01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 --2- 172.21.15.87:0/3644794361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0101440 0x7f04a010f330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 --2- 172.21.15.87:0/3644794361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0100a80 0x7f04a0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 -- 172.21.15.87:0/3644794361 >> 172.21.15.87:0/3644794361 conn(0x7f04a0077a00 msgr2=0x7f04a00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 -- 172.21.15.87:0/3644794361 shutdown_connections 2024-06-06T07:49:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.586+0000 7f04a6537700 1 -- 172.21.15.87:0/3644794361 wait complete. 2024-06-06T07:49:06.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 Processor -- start 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 -- start start 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 0x7f04a0103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 0x7f04a01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0112770 con 0x7f04a0108510 2024-06-06T07:49:06.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01125f0 con 0x7f04a0100a80 2024-06-06T07:49:06.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a6537700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01128f0 con 0x7f04a0101440 2024-06-06T07:49:06.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f0497fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f04a4ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 0x7f04a01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f0497fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58050/0 (socket says 172.21.15.87:58050) 2024-06-06T07:49:06.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f0497fff700 1 -- 172.21.15.87:0/882146307 learned_addr learned my addr 172.21.15.87:0/882146307 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.587+0000 7f049ffff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 0x7f04a0103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 -- 172.21.15.87:0/882146307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 msgr2=0x7f04a0103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 0x7f04a0103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 -- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 msgr2=0x7f04a01022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 0x7f04a01022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 -- 172.21.15.87:0/882146307 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a01ab5f0 con 0x7f04a0101440 2024-06-06T07:49:06.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f049ffff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 0x7f04a0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 0x7f04a01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f0497fff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f048800d980 tx=0x7f048800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f049dffb700 1 -- 172.21.15.87:0/882146307 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488016070 con 0x7f04a0101440 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a01ab880 con 0x7f04a0101440 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f049dffb700 1 -- 172.21.15.87:0/882146307 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0488004d10 con 0x7f04a0101440 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f049dffb700 1 -- 172.21.15.87:0/882146307 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048800f690 con 0x7f04a0101440 2024-06-06T07:49:06.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.588+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a01abdc0 con 0x7f04a0101440 2024-06-06T07:49:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.591+0000 7f049dffb700 1 -- 172.21.15.87:0/882146307 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f0488004750 con 0x7f04a0101440 2024-06-06T07:49:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f049dffb700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 0x7f048c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f04b0dbab80 1 Processor -- start 2024-06-06T07:49:06.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f04b0dbab80 1 -- start start 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f049ffff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 0x7f048c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f049dffb700 1 -- 172.21.15.87:0/882146307 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f04880cf7a0 con 0x7f04a0101440 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.592+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba3880 0x55d2b7b9d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7a8ff60 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7aee850 con 0x55d2b7ba2f60 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7a56460 con 0x55d2b7ba3880 2024-06-06T07:49:06.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f0497fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba3880 0x55d2b7b9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56228/0 (socket says 172.21.15.87:56228) 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 -- 172.21.15.87:0/882510291 learned_addr learned my addr 172.21.15.87:0/882510291 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f049ffff700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 0x7f048c0a5480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0490009fd0 tx=0x7f0490009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f049ffff700 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 -- 172.21.15.87:0/882510291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba3880 msgr2=0x55d2b7b9d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba3880 0x55d2b7b9d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 -- 172.21.15.87:0/882510291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba2f60 msgr2=0x55d2b7ba3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 -- 172.21.15.87:0/882510291 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b7a3d8d0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d60 tx=0x7f0498009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3473e955f9e92df6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.593+0000 7f04967fc700 1 -- 172.21.15.87:0/882510291 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498021070 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04967fc700 1 -- 172.21.15.87:0/882510291 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498006060 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04967fc700 1 -- 172.21.15.87:0/882510291 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980256b0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 -- 172.21.15.87:0/882510291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 msgr2=0x55d2b7b9dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d60 tx=0x7f0498009d90 comp rx=0 tx=0).stop 2024-06-06T07:49:06.604 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 -- 172.21.15.87:0/882510291 shutdown_connections 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7b9dbc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/882510291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba3880 0x55d2b7b9d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 -- 172.21.15.87:0/882510291 >> 172.21.15.87:0/882510291 conn(0x55d2b7aca080 msgr2=0x55d2b7b98220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 -- 172.21.15.87:0/882510291 shutdown_connections 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.594+0000 7f04b0dbab80 1 -- 172.21.15.87:0/882510291 wait complete. 2024-06-06T07:49:06.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 -- start start 2024-06-06T07:49:06.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 Processor -- start 2024-06-06T07:49:06.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7a8efc0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7aea810 con 0x55d2b7ba3880 2024-06-06T07:49:06.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.595+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2b7baa200 con 0x55d2b7ba2f60 2024-06-06T07:49:06.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f04b0dbab80 10 client.0 _ll_register_callbacks cb 0x55d2b7a845b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:06.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56236/0 (socket says 172.21.15.87:56236) 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 -- 172.21.15.87:0/4280924745 learned_addr learned my addr 172.21.15.87:0/4280924745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f0497fff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 -- 172.21.15.87:0/4280924745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 msgr2=0x55d2b7ba68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 -- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 msgr2=0x55d2b7ba6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 -- 172.21.15.87:0/4280924745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b7ab4750 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f049ffff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0490003870 tx=0x7f04900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f16bf4e40aaaa0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f047f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:06.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049001f720 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f049001f8c0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.596+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0498006060 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f049001fac0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 10 client.15681 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2b7bb49f0 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498006060 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0490026070 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:06.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:06.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 20 client.15681 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:06.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7bb7080 0x55d2b7bb94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 4 client.15681 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:06.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2b7afbd50 con 0x55d2b7bb7080 2024-06-06T07:49:06.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.597+0000 7f0497fff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7bb7080 0x55d2b7bb94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.598+0000 7f0497fff700 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7bb7080 0x55d2b7bb94d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95112fe8e27e78cd server_cookie=6e846b92355487cf in_seq=0 out_seq=0 2024-06-06T07:49:06.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.598+0000 7f047f7fe700 10 client.15681 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f047f7fe700 1 -- 172.21.15.87:0/4280924745 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2b7afbd50 con 0x55d2b7bb7080 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f047f7fe700 10 client.15681 handle_command_reply: tid=0 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 client.15681 shutdown 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 msgr2=0x55d2b7ba95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0490003870 tx=0x7f04900047c0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 shutdown_connections 2024-06-06T07:49:06.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7bb7080 0x55d2b7bb94d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7ba95c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/4280924745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7ba68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 >> 172.21.15.87:0/4280924745 conn(0x55d2b7aca080 msgr2=0x55d2b7b97700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f047f7fe700 0 client.15681 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:06.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 shutdown_connections 2024-06-06T07:49:06.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 1 -- 172.21.15.87:0/4280924745 wait complete. 2024-06-06T07:49:06.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.599+0000 7f04b0dbab80 20 client.15681 trim_cache size 0 max 16384 2024-06-06T07:49:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 Processor -- start 2024-06-06T07:49:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 -- start start 2024-06-06T07:49:06.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7bb3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b74a0 con 0x55d2b7ba2f60 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b4d30 con 0x55d2b7ba3880 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.615+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b2b80 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56244/0 (socket says 172.21.15.87:56244) 2024-06-06T07:49:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 -- 172.21.15.87:0/2466879552 learned_addr learned my addr 172.21.15.87:0/2466879552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f049ffff700 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7bb3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 -- 172.21.15.87:0/2466879552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7b9d7a0 msgr2=0x55d2b7bb3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7bb3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 -- 172.21.15.87:0/2466879552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 msgr2=0x55d2b7ba44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 -- 172.21.15.87:0/2466879552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b7ba7180 con 0x55d2b7ba2f60 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0497fff700 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0488098080 tx=0x7f048809be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366b93d6c7c07c45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0495ffb700 1 -- 172.21.15.87:0/2466879552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048809fa60 con 0x55d2b7ba2f60 2024-06-06T07:49:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0495ffb700 1 -- 172.21.15.87:0/2466879552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04880a4320 con 0x55d2b7ba2f60 2024-06-06T07:49:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.616+0000 7f0495ffb700 1 -- 172.21.15.87:0/2466879552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488004a40 con 0x55d2b7ba2f60 2024-06-06T07:49:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- 172.21.15.87:0/2466879552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 msgr2=0x55d2b7b9d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0488098080 tx=0x7f048809be00 comp rx=0 tx=0).stop 2024-06-06T07:49:06.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- 172.21.15.87:0/2466879552 shutdown_connections 2024-06-06T07:49:06.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7bb3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/2466879552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba2f60 0x55d2b7b9d260 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- 172.21.15.87:0/2466879552 >> 172.21.15.87:0/2466879552 conn(0x55d2b7aca080 msgr2=0x55d2b7ba51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- 172.21.15.87:0/2466879552 shutdown_connections 2024-06-06T07:49:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- 172.21.15.87:0/2466879552 wait complete. 2024-06-06T07:49:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.617+0000 7f04b0dbab80 1 -- start start 2024-06-06T07:49:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 Processor -- start 2024-06-06T07:49:06.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7baf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7bafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b31f0 con 0x55d2b7ba3880 2024-06-06T07:49:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b4660 con 0x55d2b7b9d7a0 2024-06-06T07:49:06.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01b38c0 con 0x55d2b7ba2f60 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04a4ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04a4ad4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56248/0 (socket says 172.21.15.87:56248) 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f049ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7baf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04a4ad4700 1 -- 172.21.15.87:0/845336937 learned_addr learned my addr 172.21.15.87:0/845336937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f04b0dbab80 10 client.0 _ll_register_callbacks cb 0x55d2b7a845b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:06.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.618+0000 7f0497fff700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7bafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 -- 172.21.15.87:0/845336937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 msgr2=0x55d2b7bafa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7bafa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 -- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 msgr2=0x55d2b7baf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:06.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7baf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 -- 172.21.15.87:0/845336937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b7ba5b10 con 0x55d2b7ba3880 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f049ffff700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7baf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04a4ad4700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0498009a20 tx=0x7f0498003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebb2c87c9e83e54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f047f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498021040 con 0x55d2b7ba3880 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f04980078b0 con 0x55d2b7ba3880 2024-06-06T07:49:06.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04880a4320 con 0x55d2b7ba3880 2024-06-06T07:49:06.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0498036050 con 0x55d2b7ba3880 2024-06-06T07:49:06.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04b0dbab80 10 client.15687 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:06.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2b7c21510 con 0x55d2b7ba3880 2024-06-06T07:49:06.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.619+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498029030 con 0x55d2b7ba3880 2024-06-06T07:49:06.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0498011070 con 0x55d2b7ba3880 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 20 client.15687 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7c21790 0x55d2b7c23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 4 client.15687 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:06.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f04880a4320 con 0x55d2b7c21790 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f049ffff700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7c21790 0x55d2b7c23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f047f7fe700 10 client.15687 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.620+0000 7f049ffff700 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7c21790 0x55d2b7c23b80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df171208cae946ed server_cookie=51305e1d38256542 in_seq=0 out_seq=0 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f047f7fe700 1 -- 172.21.15.87:0/845336937 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f04880a4320 con 0x55d2b7c21790 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f047f7fe700 10 client.15687 handle_command_reply: tid=0 2024-06-06T07:49:06.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 client.15687 shutdown 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 msgr2=0x55d2b7ba55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0498009a20 tx=0x7f0498003c30 comp rx=0 tx=0).stop 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 shutdown_connections 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d2b7c21790 0x55d2b7c23b80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2b7b9d7a0 0x55d2b7baf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f047f7fe700 0 client.15687 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:06.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b7ba3880 0x55d2b7ba55d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 --2- 172.21.15.87:0/845336937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2b7ba2f60 0x55d2b7bafa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 >> 172.21.15.87:0/845336937 conn(0x55d2b7aca080 msgr2=0x55d2b7ba4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 shutdown_connections 2024-06-06T07:49:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 1 -- 172.21.15.87:0/845336937 wait complete. 2024-06-06T07:49:06.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.621+0000 7f04b0dbab80 20 client.15687 trim_cache size 0 max 16384 2024-06-06T07:49:06.643 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:06.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 msgr2=0x7f048c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 0x7f048c0a5480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0490009fd0 tx=0x7f0490009260 comp rx=0 tx=0).stop 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 msgr2=0x7f04a0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f048800d980 tx=0x7f048800dc90 comp rx=0 tx=0).stop 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 shutdown_connections 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0100a80 0x7f04a0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f048c0a2fc0 0x7f048c0a5480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04a0108510 0x7f04a01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 --2- 172.21.15.87:0/882146307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04a0101440 0x7f04a0101c60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:06.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 >> 172.21.15.87:0/882146307 conn(0x7f04a0077a00 msgr2=0x7f04a00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:06.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 shutdown_connections 2024-06-06T07:49:06.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:06.622+0000 7f04a6537700 1 -- 172.21.15.87:0/882146307 wait complete. 2024-06-06T07:49:06.646 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:08.441 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:49:08.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:49:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 -- 172.21.15.87:0/2684933502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c101990 msgr2=0x7ff64c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 --2- 172.21.15.87:0/2684933502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c101990 0x7ff64c101e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff638009930 tx=0x7ff63802a420 comp rx=0 tx=0).stop 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 -- 172.21.15.87:0/2684933502 shutdown_connections 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 --2- 172.21.15.87:0/2684933502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 0x7ff64c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 --2- 172.21.15.87:0/2684933502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c101990 0x7ff64c101e10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 --2- 172.21.15.87:0/2684933502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c1079a0 0x7ff64c107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 -- 172.21.15.87:0/2684933502 >> 172.21.15.87:0/2684933502 conn(0x7ff64c0fd460 msgr2=0x7ff64c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 -- 172.21.15.87:0/2684933502 shutdown_connections 2024-06-06T07:49:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.456+0000 7ff653064640 1 -- 172.21.15.87:0/2684933502 wait complete. 2024-06-06T07:49:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 Processor -- start 2024-06-06T07:49:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 -- start start 2024-06-06T07:49:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c101990 0x7ff64c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 0x7ff64c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c111db0 con 0x7ff64c1079a0 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c111c30 con 0x7ff64c101990 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff653064640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c111f30 con 0x7ff64c102350 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff651601640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff651601640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38046/0 (socket says 172.21.15.87:38046) 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.457+0000 7ff651601640 1 -- 172.21.15.87:0/1158117204 learned_addr learned my addr 172.21.15.87:0/1158117204 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff650e00640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c101990 0x7ff64c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff64bfff640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 0x7ff64c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 -- 172.21.15.87:0/1158117204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 msgr2=0x7ff64c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 0x7ff64c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 -- 172.21.15.87:0/1158117204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c101990 msgr2=0x7ff64c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c101990 0x7ff64c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 -- 172.21.15.87:0/1158117204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19d1c0 con 0x7ff64c1079a0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff651601640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff64400cad0 tx=0x7ff64400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff649ffb640 1 -- 172.21.15.87:0/1158117204 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400ad60 con 0x7ff64c1079a0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c19d4b0 con 0x7ff64c1079a0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff649ffb640 1 -- 172.21.15.87:0/1158117204 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff644005020 con 0x7ff64c1079a0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff649ffb640 1 -- 172.21.15.87:0/1158117204 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400f6b0 con 0x7ff64c1079a0 2024-06-06T07:49:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.458+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1a4d40 con 0x7ff64c1079a0 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.460+0000 7ff649ffb640 1 -- 172.21.15.87:0/1158117204 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7ff644005560 con 0x7ff64c1079a0 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.461+0000 7ff649ffb640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 0x7ff63c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.461+0000 7ff650e00640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 0x7ff63c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.461+0000 7ff649ffb640 1 -- 172.21.15.87:0/1158117204 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff6440cfb10 con 0x7ff64c1079a0 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff650e00640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 0x7ff63c0a5410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff64000ab60 tx=0x7ff64000a2d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 Processor -- start 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 -- start start 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.464 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b44ba90 0x55778b455e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b456350 0x55778b458740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55778b3af4f0 con 0x55778b44b170 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55778b34d870 con 0x55778b456350 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff655d75740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55778b3ab4b0 con 0x55778b44ba90 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38060/0 (socket says 172.21.15.87:38060) 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff650e00640 1 -- 172.21.15.87:0/4202789342 learned_addr learned my addr 172.21.15.87:0/4202789342 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff64bfff640 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b44ba90 0x55778b455e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.462+0000 7ff651601640 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b456350 0x55778b458740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 -- 172.21.15.87:0/4202789342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b44ba90 msgr2=0x55778b455e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b44ba90 0x55778b455e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 -- 172.21.15.87:0/4202789342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b456350 msgr2=0x55778b458740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b456350 0x55778b458740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 -- 172.21.15.87:0/4202789342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778b3699b0 con 0x55778b44b170 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff650e00640 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff64000ff70 tx=0x7ff640012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5427792586689242 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff636ffd640 1 -- 172.21.15.87:0/4202789342 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640016070 con 0x55778b44b170 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff636ffd640 1 -- 172.21.15.87:0/4202789342 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.463+0000 7ff636ffd640 1 -- 172.21.15.87:0/4202789342 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64001c8b0 con 0x55778b44b170 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- 172.21.15.87:0/4202789342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 msgr2=0x55778b44b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff64000ff70 tx=0x7ff640012000 comp rx=0 tx=0).stop 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- 172.21.15.87:0/4202789342 shutdown_connections 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b456350 0x55778b458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b44ba90 0x55778b455e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 --2- 172.21.15.87:0/4202789342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b44b550 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- 172.21.15.87:0/4202789342 >> 172.21.15.87:0/4202789342 conn(0x55778b447680 msgr2=0x55778b447a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- 172.21.15.87:0/4202789342 shutdown_connections 2024-06-06T07:49:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- 172.21.15.87:0/4202789342 wait complete. 2024-06-06T07:49:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 -- start start 2024-06-06T07:49:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.464+0000 7ff655d75740 1 Processor -- start 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b453830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b453d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55778b458f80 con 0x55778b44b170 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55778b338ae0 con 0x55778b44ba90 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55778b459100 con 0x55778b456350 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 10 client.0 _ll_register_callbacks cb 0x55778b3719b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38070/0 (socket says 172.21.15.87:38070) 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 -- 172.21.15.87:0/1171842884 learned_addr learned my addr 172.21.15.87:0/1171842884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff651601640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b453d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff64bfff640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b453830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 -- 172.21.15.87:0/1171842884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 msgr2=0x55778b453d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b453d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 -- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 msgr2=0x55778b453830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b453830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 -- 172.21.15.87:0/1171842884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778b0c3210 con 0x55778b44b170 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff651601640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b453d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff64bfff640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b453830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff650e00640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff640010c90 tx=0x7ff64001bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363a96ea282a5a36 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff62bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640033060 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff64001b620 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.465+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff64002a040 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55778b18f620 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff64002d030 con 0x55778b44b170 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 20 client.15699 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b466430 0x55778b468820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 4 client.15699 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778b3c2f10 con 0x55778b466430 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.466+0000 7ff64bfff640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b466430 0x55778b468820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.467+0000 7ff62bfff640 10 client.15699 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.467+0000 7ff64bfff640 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b466430 0x55778b468820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c149774a7a2691 server_cookie=36be63a64007bd15 in_seq=0 out_seq=0 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.467+0000 7ff62bfff640 1 -- 172.21.15.87:0/1171842884 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55778b3c2f10 con 0x55778b466430 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff62bfff640 10 client.15699 handle_command_reply: tid=0 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 client.15699 shutdown 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 msgr2=0x55778b4532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff640010c90 tx=0x7ff64001bd70 comp rx=0 tx=0).stop 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 shutdown_connections 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b466430 0x55778b468820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b453d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b453830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 --2- 172.21.15.87:0/1171842884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4532f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.474 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:08 smithi087 ceph-mon[32276]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 >> 172.21.15.87:0/1171842884 conn(0x55778b447680 msgr2=0x55778b44d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff62bfff640 0 client.15699 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 shutdown_connections 2024-06-06T07:49:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 1 -- 172.21.15.87:0/1171842884 wait complete. 2024-06-06T07:49:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.468+0000 7ff655d75740 20 client.15699 trim_cache size 0 max 16384 2024-06-06T07:49:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.483+0000 7ff655d75740 1 Processor -- start 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.483+0000 7ff655d75740 1 -- start start 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b46df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b46e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1b1810 con 0x55778b44b170 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1af0e0 con 0x55778b44ba90 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff655d75740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1acf30 con 0x55778b456350 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38086/0 (socket says 172.21.15.87:38086) 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 -- 172.21.15.87:0/747973801 learned_addr learned my addr 172.21.15.87:0/747973801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff651601640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b46e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff64bfff640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b46df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 -- 172.21.15.87:0/747973801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 msgr2=0x55778b46e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b46e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 -- 172.21.15.87:0/747973801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 msgr2=0x55778b46df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b46df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 -- 172.21.15.87:0/747973801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778b458e60 con 0x55778b44b170 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff651601640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b46e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff650e00640 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff64000f8f0 tx=0x7ff64000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa10264f6c487d1d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.484+0000 7ff6367fc640 1 -- 172.21.15.87:0/747973801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640016070 con 0x55778b44b170 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff6367fc640 1 -- 172.21.15.87:0/747973801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff6367fc640 1 -- 172.21.15.87:0/747973801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640033040 con 0x55778b44b170 2024-06-06T07:49:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 -- 172.21.15.87:0/747973801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 msgr2=0x55778b452a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff64000f8f0 tx=0x7ff64000a7b0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 -- 172.21.15.87:0/747973801 shutdown_connections 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b46e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b46df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 --2- 172.21.15.87:0/747973801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b452a60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 -- 172.21.15.87:0/747973801 >> 172.21.15.87:0/747973801 conn(0x55778b447680 msgr2=0x55778b451bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 -- 172.21.15.87:0/747973801 shutdown_connections 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.485+0000 7ff655d75740 1 -- 172.21.15.87:0/747973801 wait complete. 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 -- start start 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 Processor -- start 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b4c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b4c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1ad5a0 con 0x55778b44b170 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aea10 con 0x55778b44ba90 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1adc70 con 0x55778b456350 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff655d75740 10 client.0 _ll_register_callbacks cb 0x7ff64000fe40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff650e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38088/0 (socket says 172.21.15.87:38088) 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff650e00640 1 -- 172.21.15.87:0/3753609854 learned_addr learned my addr 172.21.15.87:0/3753609854 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.486+0000 7ff64bfff640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b4c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff651601640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b4c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 -- 172.21.15.87:0/3753609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 msgr2=0x55778b4c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b4c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 -- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 msgr2=0x55778b4c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b4c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 -- 172.21.15.87:0/3753609854 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778b45d340 con 0x55778b44b170 2024-06-06T07:49:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff651601640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b4c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff64bfff640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b4c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff650e00640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff64001c540 tx=0x7ff640010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b00d353dd720 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640033070 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff640005000 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff640016040 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55778b45e650 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64001b900 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff64002d070 con 0x55778b44b170 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 20 client.15705 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b45ed00 0x55778b4d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 4 client.15705 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.487+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55778b3bcc20 con 0x55778b45ed00 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.488+0000 7ff64bfff640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b45ed00 0x55778b4d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.488+0000 7ff64bfff640 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b45ed00 0x55778b4d7300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cecd2854c57ee1 server_cookie=d4253fd7627324e4 in_seq=0 out_seq=0 2024-06-06T07:49:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.488+0000 7ff62bfff640 10 client.15705 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.495+0000 7ff62bfff640 1 -- 172.21.15.87:0/3753609854 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55778b3bcc20 con 0x55778b45ed00 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.495+0000 7ff62bfff640 10 client.15705 handle_command_reply: tid=0 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.495+0000 7ff655d75740 1 client.15705 shutdown 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 msgr2=0x55778b4cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff64001c540 tx=0x7ff640010d00 comp rx=0 tx=0).stop 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 shutdown_connections 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55778b45ed00 0x55778b4d7300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55778b456350 0x55778b4c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff62bfff640 0 client.15705 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55778b44ba90 0x55778b4c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 --2- 172.21.15.87:0/3753609854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55778b44b170 0x55778b4cc930 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 >> 172.21.15.87:0/3753609854 conn(0x55778b447680 msgr2=0x55778b453290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 shutdown_connections 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 1 -- 172.21.15.87:0/3753609854 wait complete. 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff655d75740 20 client.15705 trim_cache size 0 max 16384 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:49:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:49:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 msgr2=0x7ff63c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 0x7ff63c0a5410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff64000ab60 tx=0x7ff64000a2d0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 msgr2=0x7ff64c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.496+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff64400cad0 tx=0x7ff64400ced0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 shutdown_connections 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff63c0a2f50 0x7ff63c0a5410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c1079a0 0x7ff64c19ca90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff64c102350 0x7ff64c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 --2- 172.21.15.87:0/1158117204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64c101990 0x7ff64c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 >> 172.21.15.87:0/1158117204 conn(0x7ff64c0fd460 msgr2=0x7ff64c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 shutdown_connections 2024-06-06T07:49:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:08.497+0000 7ff653064640 1 -- 172.21.15.87:0/1158117204 wait complete. 2024-06-06T07:49:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:08 smithi195 ceph-mon[37040]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:08 smithi181 ceph-mon[36790]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:49:08.710 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 -- 172.21.15.87:0/3544458564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c1086a0 msgr2=0x7ff72c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 --2- 172.21.15.87:0/3544458564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c1086a0 0x7ff72c108a80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff7200099c0 tx=0x7ff720009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 -- 172.21.15.87:0/3544458564 shutdown_connections 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 --2- 172.21.15.87:0/3544458564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 0x7ff72c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 --2- 172.21.15.87:0/3544458564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c1086a0 0x7ff72c108a80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 --2- 172.21.15.87:0/3544458564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c06abc0 0x7ff72c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 -- 172.21.15.87:0/3544458564 >> 172.21.15.87:0/3544458564 conn(0x7ff72c078160 msgr2=0x7ff72c078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 -- 172.21.15.87:0/3544458564 shutdown_connections 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 -- 172.21.15.87:0/3544458564 wait complete. 2024-06-06T07:49:09.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.420+0000 7ff732ecc700 1 Processor -- start 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 -- start start 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 0x7ff72c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 0x7ff72c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1127b0 con 0x7ff72c06abc0 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c112630 con 0x7ff72c1086a0 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff732ecc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c112930 con 0x7ff72c06b580 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38110/0 (socket says 172.21.15.87:38110) 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 -- 172.21.15.87:0/409956640 learned_addr learned my addr 172.21.15.87:0/409956640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:09.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff731469700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 0x7ff72c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff72bfff700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 0x7ff72c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 -- 172.21.15.87:0/409956640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 msgr2=0x7ff72c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 0x7ff72c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 -- 172.21.15.87:0/409956640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 msgr2=0x7ff72c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 0x7ff72c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 -- 172.21.15.87:0/409956640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c199150 con 0x7ff72c06abc0 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff731469700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 0x7ff72c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:09.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff730c68700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff720006010 tx=0x7ff720009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:09.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.421+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720027070 con 0x7ff72c06abc0 2024-06-06T07:49:09.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.422+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff720004050 con 0x7ff72c06abc0 2024-06-06T07:49:09.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.422+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c1993e0 con 0x7ff72c06abc0 2024-06-06T07:49:09.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.422+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720022e10 con 0x7ff72c06abc0 2024-06-06T07:49:09.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.422+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1a56b0 con 0x7ff72c06abc0 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.423+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72c069180 con 0x7ff72c06abc0 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.425+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7ff72002a070 con 0x7ff72c06abc0 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.425+0000 7ff729ffb700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 0x7ff71c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.425+0000 7ff72bfff700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 0x7ff71c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.425+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff7200d09e0 con 0x7ff72c06abc0 2024-06-06T07:49:09.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.426+0000 7ff72bfff700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 0x7ff71c0a5480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff71800a560 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:09.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.429+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff720013070 con 0x7ff72c06abc0 2024-06-06T07:49:09.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.675+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff72c199ed0 con 0x7ff72c06abc0 2024-06-06T07:49:09.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.676+0000 7ff729ffb700 1 -- 172.21.15.87:0/409956640 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7ff72009d080 con 0x7ff72c06abc0 2024-06-06T07:49:09.676 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:49:09.677 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 msgr2=0x7ff71c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 0x7ff71c0a5480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff71800a560 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 msgr2=0x7ff72c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff720006010 tx=0x7ff720009f80 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 shutdown_connections 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff72c1086a0 0x7ff72c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff71c0a2fc0 0x7ff71c0a5480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff72c06abc0 0x7ff72c19e7e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 --2- 172.21.15.87:0/409956640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff72c06b580 0x7ff72c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:09.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 >> 172.21.15.87:0/409956640 conn(0x7ff72c078160 msgr2=0x7ff72c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:09.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 shutdown_connections 2024-06-06T07:49:09.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:09.679+0000 7ff732ecc700 1 -- 172.21.15.87:0/409956640 wait complete. 2024-06-06T07:49:09.680 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:49:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:10 smithi195 ceph-mon[37040]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:10 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/409956640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:10 smithi181 ceph-mon[36790]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:10 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/409956640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:10 smithi087 ceph-mon[32276]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:10 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/409956640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:10.659 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0' 2024-06-06T07:49:10.660 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0 2024-06-06T07:49:11.674 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 -- 172.21.15.87:0/2966891621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d8103150 msgr2=0x7fb7d8107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/2966891621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d8103150 0x7fb7d8107140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0099c0 tx=0x7fb7cc009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 -- 172.21.15.87:0/2966891621 shutdown_connections 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/2966891621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8107870 0x7fb7d810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/2966891621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d81027a0 0x7fb7d8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/2966891621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d8103150 0x7fb7d8107140 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 -- 172.21.15.87:0/2966891621 >> 172.21.15.87:0/2966891621 conn(0x7fb7d8079520 msgr2=0x7fb7d8079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 -- 172.21.15.87:0/2966891621 shutdown_connections 2024-06-06T07:49:11.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.680+0000 7fb7dcec4700 1 -- 172.21.15.87:0/2966891621 wait complete. 2024-06-06T07:49:11.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.681+0000 7fb7dcec4700 1 Processor -- start 2024-06-06T07:49:11.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.681+0000 7fb7dcec4700 1 -- start start 2024-06-06T07:49:11.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8103150 0x7fb7d819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d8107870 0x7fb7d8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81127f0 con 0x7fb7d81027a0 2024-06-06T07:49:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8112670 con 0x7fb7d8103150 2024-06-06T07:49:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7dcec4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8112970 con 0x7fb7d8107870 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38136/0 (socket says 172.21.15.87:38136) 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 -- 172.21.15.87:0/1390272733 learned_addr learned my addr 172.21.15.87:0/1390272733 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8103150 0x7fb7d819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d6d9d700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d8107870 0x7fb7d8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 -- 172.21.15.87:0/1390272733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d8107870 msgr2=0x7fb7d8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d8107870 0x7fb7d8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 -- 172.21.15.87:0/1390272733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8103150 msgr2=0x7fb7d819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8103150 0x7fb7d819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.682+0000 7fb7d659c700 1 -- 172.21.15.87:0/1390272733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8199190 con 0x7fb7d81027a0 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7d659c700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800d980 tx=0x7fb7c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7c77fe700 1 -- 172.21.15.87:0/1390272733 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8017070 con 0x7fb7d81027a0 2024-06-06T07:49:11.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7c77fe700 1 -- 172.21.15.87:0/1390272733 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c8005020 con 0x7fb7d81027a0 2024-06-06T07:49:11.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7c77fe700 1 -- 172.21.15.87:0/1390272733 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800fbf0 con 0x7fb7d81027a0 2024-06-06T07:49:11.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d8199480 con 0x7fb7d81027a0 2024-06-06T07:49:11.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.683+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d81a56f0 con 0x7fb7d81027a0 2024-06-06T07:49:11.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.685+0000 7fb7c77fe700 1 -- 172.21.15.87:0/1390272733 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fb7c80051d0 con 0x7fb7d81027a0 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.686+0000 7fb7c77fe700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 0x7fb7c00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 0x7fb7c00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7c77fe700 1 -- 172.21.15.87:0/1390272733 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb7c80cf8c0 con 0x7fb7d81027a0 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7e7747b80 1 Processor -- start 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7e7747b80 1 -- start start 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 0x7fb7c00a5480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8100fc0 tx=0x7fb7cc005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d3979eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.687+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563d39690fe0 con 0x563d397a2080 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563d396ef8d0 con 0x563d3979e820 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563d396574e0 con 0x563d397b0f40 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38140/0 (socket says 172.21.15.87:38140) 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 -- 172.21.15.87:0/373335997 learned_addr learned my addr 172.21.15.87:0/373335997 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d6d9d700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d3979eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 -- 172.21.15.87:0/373335997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 msgr2=0x563d397b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 -- 172.21.15.87:0/373335997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 msgr2=0x563d3979eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d3979eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 -- 172.21.15.87:0/373335997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d3963e950 con 0x563d397a2080 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d659c700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7c809b4d0 tx=0x7fb7c809c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d46ccb0d1366fe8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d3979eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7c4ff9700 1 -- 172.21.15.87:0/373335997 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80a4440 con 0x563d397a2080 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7c4ff9700 1 -- 172.21.15.87:0/373335997 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c8005540 con 0x563d397a2080 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.688+0000 7fb7c4ff9700 1 -- 172.21.15.87:0/373335997 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80092e0 con 0x563d397a2080 2024-06-06T07:49:11.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 -- 172.21.15.87:0/373335997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 msgr2=0x563d3979e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7c809b4d0 tx=0x7fb7c809c020 comp rx=0 tx=0).stop 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 -- 172.21.15.87:0/373335997 shutdown_connections 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d3979e2e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/373335997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d3979eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.689+0000 7fb7e7747b80 1 -- 172.21.15.87:0/373335997 >> 172.21.15.87:0/373335997 conn(0x563d396cb100 msgr2=0x563d397992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 -- 172.21.15.87:0/373335997 shutdown_connections 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 -- 172.21.15.87:0/373335997 wait complete. 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 -- start start 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 Processor -- start 2024-06-06T07:49:11.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d397ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.690+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563d39690040 con 0x563d397a2080 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563d396eb890 con 0x563d3979e820 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563d397b3b20 con 0x563d397b0f40 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:48122/0 (socket says 172.21.15.87:48122) 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 -- 172.21.15.87:0/3316968103 learned_addr learned my addr 172.21.15.87:0/3316968103 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7e7747b80 10 client.0 _ll_register_callbacks cb 0x563d39685630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d6d9d700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d397ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 -- 172.21.15.87:0/3316968103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 msgr2=0x563d397adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 -- 172.21.15.87:0/3316968103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 msgr2=0x563d397ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d397ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 -- 172.21.15.87:0/3316968103 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d396b57d0 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7d659c700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7c809b710 tx=0x7fb7c809bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b568846eefd406 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.691+0000 7fb7bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b9070 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c8005540 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb7c80a41c0 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7c80092a0 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7e7747b80 10 client.25142 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d397be090 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c8005540 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb7c80ccd10 con 0x563d3979e820 2024-06-06T07:49:11.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7e7747b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.692+0000 7fb7e7747b80 10 client.25142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7e7747b80 20 client.25142 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d397c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7e7747b80 4 client.25142 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d396fcdd0 con 0x563d397c0560 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d397c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7bdffb700 10 client.25142 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.693+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d397c2950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e892ba204fcf0fd server_cookie=d16723834efff5cc in_seq=0 out_seq=0 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7bdffb700 1 -- 172.21.15.87:0/3316968103 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d396fcdd0 con 0x563d397c0560 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7bdffb700 10 client.25142 handle_command_reply: tid=0 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 client.25142 shutdown 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 msgr2=0x563d397b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7c809b710 tx=0x7fb7c809bd70 comp rx=0 tx=0).stop 2024-06-06T07:49:11.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 shutdown_connections 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d397c2950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7bdffb700 0 client.25142 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397b0780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/3316968103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397a2080 0x563d397ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.694+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 >> 172.21.15.87:0/3316968103 conn(0x563d396cb100 msgr2=0x563d39798780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.695+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 shutdown_connections 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.695+0000 7fb7e7747b80 1 -- 172.21.15.87:0/3316968103 wait complete. 2024-06-06T07:49:11.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.695+0000 7fb7e7747b80 20 client.25142 trim_cache size 0 max 16384 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.719+0000 7fb7e7747b80 1 Processor -- start 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.719+0000 7fb7e7747b80 1 -- start start 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.719+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d397a2080 0x563d39799440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397c5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81aed20 con 0x563d3979e820 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81ac5b0 con 0x563d397a2080 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81aa400 con 0x563d397b0f40 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38160/0 (socket says 172.21.15.87:38160) 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 -- 172.21.15.87:0/4104157347 learned_addr learned my addr 172.21.15.87:0/4104157347 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d6d9d700 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d397a2080 0x563d39799440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 -- 172.21.15.87:0/4104157347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 msgr2=0x563d397c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 -- 172.21.15.87:0/4104157347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d397a2080 msgr2=0x563d39799440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d397a2080 0x563d39799440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 -- 172.21.15.87:0/4104157347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d397ac7f0 con 0x563d3979e820 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7d659c700 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80981a0 tx=0x7fb7c80cca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8238228f5deb7b73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.720+0000 7fb7bffff700 1 -- 172.21.15.87:0/4104157347 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b9070 con 0x563d3979e820 2024-06-06T07:49:11.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7bffff700 1 -- 172.21.15.87:0/4104157347 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c80a41c0 con 0x563d3979e820 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7bffff700 1 -- 172.21.15.87:0/4104157347 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8097520 con 0x563d3979e820 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 -- 172.21.15.87:0/4104157347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 msgr2=0x563d39798f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80981a0 tx=0x7fb7c80cca20 comp rx=0 tx=0).stop 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 -- 172.21.15.87:0/4104157347 shutdown_connections 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397b0f40 0x563d397c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d3979e820 0x563d39798f00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/4104157347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d397a2080 0x563d39799440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.721+0000 7fb7e7747b80 1 -- 172.21.15.87:0/4104157347 >> 172.21.15.87:0/4104157347 conn(0x563d396cb100 msgr2=0x563d397afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- 172.21.15.87:0/4104157347 shutdown_connections 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- 172.21.15.87:0/4104157347 wait complete. 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- start start 2024-06-06T07:49:11.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 Processor -- start 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397a2080 0x563d397c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 0x563d39823f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81aaa70 con 0x563d397b0f40 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81abee0 con 0x563d3979e820 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81ab140 con 0x563d397a2080 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.722+0000 7fb7e7747b80 10 client.0 _ll_register_callbacks cb 0x563d39685630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 0x563d39823f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:48140/0 (socket says 172.21.15.87:48140) 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 -- 172.21.15.87:0/2689147362 learned_addr learned my addr 172.21.15.87:0/2689147362 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397a2080 0x563d397c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 -- 172.21.15.87:0/2689147362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397a2080 msgr2=0x563d397c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397a2080 0x563d397c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 -- 172.21.15.87:0/2689147362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 msgr2=0x563d39823f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 0x563d39823f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 -- 172.21.15.87:0/2689147362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d39824590 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d6d9d700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 0x563d39823f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7d659c700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80058b0 tx=0x7fb7c8097ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3e04355c165ea0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b9070 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb7c80ccd10 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.723+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c80a41c0 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7c809ac20 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 10 client.25148 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d39824590 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c809adc0 con 0x563d3979e820 2024-06-06T07:49:11.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb7c8004a30 con 0x563d3979e820 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 20 client.25148 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d3982dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 4 client.25148 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb7c80a41c0 con 0x563d397c0560 2024-06-06T07:49:11.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.724+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d3982dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.725+0000 7fb7d5d9b700 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d3982dca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de1ad8adf0b0df4 server_cookie=411a2b9435de9e28 in_seq=0 out_seq=0 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.725+0000 7fb7bdffb700 10 client.25148 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7bdffb700 1 -- 172.21.15.87:0/2689147362 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb7c80a41c0 con 0x563d397c0560 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7bdffb700 10 client.25148 handle_command_reply: tid=0 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 client.25148 shutdown 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 msgr2=0x563d397c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80058b0 tx=0x7fb7c8097ba0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 shutdown_connections 2024-06-06T07:49:11.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563d397c0560 0x563d3982dca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d3979e820 0x563d397c40c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563d397b0f40 0x563d39823f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 --2- 172.21.15.87:0/2689147362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d397a2080 0x563d397c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7bdffb700 0 client.25148 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.726+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 >> 172.21.15.87:0/2689147362 conn(0x563d396cb100 msgr2=0x563d397ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 shutdown_connections 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7e7747b80 1 -- 172.21.15.87:0/2689147362 wait complete. 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7e7747b80 20 client.25148 trim_cache size 0 max 16384 2024-06-06T07:49:11.730 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "a7292f85-17d6-46fa-8ceb-33c61169eb81", 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 msgr2=0x7fb7c00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 0x7fb7c00a5480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8100fc0 tx=0x7fb7cc005fb0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 msgr2=0x7fb7d819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800d980 tx=0x7fb7c800dc90 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 shutdown_connections 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb7c00a2fc0 0x7fb7c00a5480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7d8107870 0x7fb7d8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7d81027a0 0x7fb7d819e820 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.727+0000 7fb7dcec4700 1 --2- 172.21.15.87:0/1390272733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d8103150 0x7fb7d819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.728+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 >> 172.21.15.87:0/1390272733 conn(0x7fb7d8079520 msgr2=0x7fb7d8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.728+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 shutdown_connections 2024-06-06T07:49:11.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:11.728+0000 7fb7dcec4700 1 -- 172.21.15.87:0/1390272733 wait complete. 2024-06-06T07:49:12.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:12 smithi087 ceph-mon[32276]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:12 smithi195 ceph-mon[37040]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:12 smithi181 ceph-mon[36790]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T07:49:13.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:13 smithi087 ceph-mon[32276]: scrub queued for path: / 2024-06-06T07:49:13.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:13 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:13 smithi087 ceph-mon[32276]: scrub summary: active paths [/] 2024-06-06T07:49:13.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:13 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:13 smithi195 ceph-mon[37040]: scrub queued for path: / 2024-06-06T07:49:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:13 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:13 smithi195 ceph-mon[37040]: scrub summary: active paths [/] 2024-06-06T07:49:13.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:13 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:13 smithi181 ceph-mon[36790]: scrub queued for path: / 2024-06-06T07:49:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:13 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:13 smithi181 ceph-mon[36790]: scrub summary: active paths [/] 2024-06-06T07:49:13.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:13 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:13.865 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:49:14.400 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:14 smithi087 ceph-mon[32276]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-06-06T07:49:14.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.428+0000 7fd508d08700 1 -- 172.21.15.87:0/4280107031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 msgr2=0x7fd50410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.428+0000 7fd508d08700 1 --2- 172.21.15.87:0/4280107031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50410e220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b0a0 tx=0x7fd4fc00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 -- 172.21.15.87:0/4280107031 shutdown_connections 2024-06-06T07:49:14.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 --2- 172.21.15.87:0/4280107031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50410e220 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 --2- 172.21.15.87:0/4280107031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd504107470 0x7fd504107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 --2- 172.21.15.87:0/4280107031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5041013c0 0x7fd504101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 -- 172.21.15.87:0/4280107031 >> 172.21.15.87:0/4280107031 conn(0x7fd5040783d0 msgr2=0x7fd5040787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 -- 172.21.15.87:0/4280107031 shutdown_connections 2024-06-06T07:49:14.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.429+0000 7fd508d08700 1 -- 172.21.15.87:0/4280107031 wait complete. 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 Processor -- start 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 -- start start 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5041013c0 0x7fd50419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 0x7fd504197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041116a0 con 0x7fd504101d80 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd504111520 con 0x7fd504107470 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd508d08700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd504111820 con 0x7fd5041013c0 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38202/0 (socket says 172.21.15.87:38202) 2024-06-06T07:49:14.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd501d9b700 1 -- 172.21.15.87:0/1881890454 learned_addr learned my addr 172.21.15.87:0/1881890454 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 0x7fd504197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.430+0000 7fd50259c700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5041013c0 0x7fd50419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 -- 172.21.15.87:0/1881890454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5041013c0 msgr2=0x7fd50419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5041013c0 0x7fd50419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 -- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 msgr2=0x7fd50419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 -- 172.21.15.87:0/1881890454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd504198040 con 0x7fd504107470 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 0x7fd504197760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b0a0 tx=0x7fd4fc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd4f37fe700 1 -- 172.21.15.87:0/1881890454 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc021070 con 0x7fd504107470 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041982d0 con 0x7fd504107470 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd4f37fe700 1 -- 172.21.15.87:0/1881890454 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4fc007b80 con 0x7fd504107470 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd4f37fe700 1 -- 172.21.15.87:0/1881890454 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc005040 con 0x7fd504107470 2024-06-06T07:49:14.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.431+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50406aa00 con 0x7fd504107470 2024-06-06T07:49:14.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.434+0000 7fd4f37fe700 1 -- 172.21.15.87:0/1881890454 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fd4fc024070 con 0x7fd504107470 2024-06-06T07:49:14.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.435+0000 7fd51358bb80 1 Processor -- start 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 -- start start 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd4f37fe700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 0x7fd4ec0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x558889573bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889574510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5588894620f0 con 0x5588895799a0 2024-06-06T07:49:14.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588894c09e0 con 0x5588895740f0 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd51358bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588894285f0 con 0x55888957a2c0 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 0x7fd4ec0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd4f37fe700 1 -- 172.21.15.87:0/1881890454 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd4fc00f020 con 0x7fd504107470 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38210/0 (socket says 172.21.15.87:38210) 2024-06-06T07:49:14.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 -- 172.21.15.87:0/2649610857 learned_addr learned my addr 172.21.15.87:0/2649610857 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 -- 172.21.15.87:0/2649610857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 msgr2=0x558889573bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x558889573bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd502d9d700 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889574510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 -- 172.21.15.87:0/2649610857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 msgr2=0x558889574510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889574510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd50259c700 1 -- 172.21.15.87:0/2649610857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888940fa60 con 0x5588895799a0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.436+0000 7fd502d9d700 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889574510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.437+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 0x7fd4ec0a5530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd504198f20 tx=0x7fd4f8011920 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.437+0000 7fd50259c700 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009db0 tx=0x7fd4f4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba2f0bb5e4bf3e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.437+0000 7fd4f0ff9700 1 -- 172.21.15.87:0/2649610857 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f401e070 con 0x5588895799a0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.437+0000 7fd4f0ff9700 1 -- 172.21.15.87:0/2649610857 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f4019b10 con 0x5588895799a0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.437+0000 7fd4f0ff9700 1 -- 172.21.15.87:0/2649610857 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400ec30 con 0x5588895799a0 2024-06-06T07:49:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- 172.21.15.87:0/2649610857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 msgr2=0x558889579d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009db0 tx=0x7fd4f4011720 comp rx=0 tx=0).stop 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- 172.21.15.87:0/2649610857 shutdown_connections 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889574510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x558889579d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2649610857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x558889573bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- 172.21.15.87:0/2649610857 >> 172.21.15.87:0/2649610857 conn(0x55888949c210 msgr2=0x55888956aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- 172.21.15.87:0/2649610857 shutdown_connections 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- 172.21.15.87:0/2649610857 wait complete. 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 -- start start 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.438+0000 7fd51358bb80 1 Processor -- start 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 0x5588895800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 0x55888957d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558889461150 con 0x5588895799a0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588894bc9a0 con 0x55888957a2c0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558889580ca0 con 0x5588895740f0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd51358bb80 10 client.0 _ll_register_callbacks cb 0x558889456740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38224/0 (socket says 172.21.15.87:38224) 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd501d9b700 1 -- 172.21.15.87:0/2111587528 learned_addr learned my addr 172.21.15.87:0/2111587528 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd502d9d700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 0x55888957d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 0x5588895800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 -- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 msgr2=0x55888957d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 0x55888957d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 -- 172.21.15.87:0/2111587528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 msgr2=0x55888957d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd50259c700 1 -- 172.21.15.87:0/2111587528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588894868e0 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd502d9d700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 0x55888957d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.439+0000 7fd501d9b700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd50259c700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 0x5588895800a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009db0 tx=0x7fd4f4004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82dcb585c4b21add server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd4e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4027020 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f4019b10 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd4f4019840 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4f4032050 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd51358bb80 10 client.25225 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55888958b230 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.440+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f4019b10 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd4f4005350 con 0x5588895740f0 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 10 client.25225 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 10 client.25225 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 20 client.25225 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55888958d700 0x55888958faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 4 client.25225 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588894cdee0 con 0x55888958d700 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd501d9b700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55888958d700 0x55888958faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd501d9b700 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55888958d700 0x55888958faf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898b190bf008ba19 server_cookie=5de2c627aef52677 in_seq=0 out_seq=0 2024-06-06T07:49:14.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.441+0000 7fd4e9ffb700 10 client.25225 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/2111587528 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588894cdee0 con 0x55888958d700 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd4e9ffb700 10 client.25225 handle_command_reply: tid=0 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 client.25225 shutdown 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 msgr2=0x5588895800a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 0x5588895800a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009db0 tx=0x7fd4f4004510 comp rx=0 tx=0).stop 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 shutdown_connections 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55888958d700 0x55888958faf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55888957a2c0 0x55888957d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd4e9ffb700 0 client.25225 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:14.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588895740f0 0x5588895800a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 --2- 172.21.15.87:0/2111587528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 >> 172.21.15.87:0/2111587528 conn(0x55888949c210 msgr2=0x558889569c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.442+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 shutdown_connections 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.443+0000 7fd51358bb80 1 -- 172.21.15.87:0/2111587528 wait complete. 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.443+0000 7fd51358bb80 20 client.25225 trim_cache size 0 max 16384 2024-06-06T07:49:14.448 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 Processor -- start 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 -- start start 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x55888957c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x55888956a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041aeb80 con 0x5588895799a0 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041ac410 con 0x5588895740f0 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd51358bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041aa260 con 0x55888957a2c0 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38230/0 (socket says 172.21.15.87:38230) 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.466+0000 7fd501d9b700 1 -- 172.21.15.87:0/1989091928 learned_addr learned my addr 172.21.15.87:0/1989091928 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:14.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd50259c700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x55888957c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x55888956a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 -- 172.21.15.87:0/1989091928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 msgr2=0x55888956a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x55888956a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 -- 172.21.15.87:0/1989091928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 msgr2=0x55888957c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x55888957c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 -- 172.21.15.87:0/1989091928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558889574850 con 0x5588895799a0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd50259c700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x55888957c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55888957e970 tx=0x7fd4f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59217b4ce5c76d48 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd4ebfff700 1 -- 172.21.15.87:0/1989091928 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017030 con 0x5588895799a0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd4ebfff700 1 -- 172.21.15.87:0/1989091928 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8019050 con 0x5588895799a0 2024-06-06T07:49:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.467+0000 7fd4ebfff700 1 -- 172.21.15.87:0/1989091928 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8004d50 con 0x5588895799a0 2024-06-06T07:49:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 -- 172.21.15.87:0/1989091928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 msgr2=0x55888957f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55888957e970 tx=0x7fd4f801a000 comp rx=0 tx=0).stop 2024-06-06T07:49:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 -- 172.21.15.87:0/1989091928 shutdown_connections 2024-06-06T07:49:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x55888957c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x55888956a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1989091928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588895799a0 0x55888957f2e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 -- 172.21.15.87:0/1989091928 >> 172.21.15.87:0/1989091928 conn(0x55888949c210 msgr2=0x55888957ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 -- 172.21.15.87:0/1989091928 shutdown_connections 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.468+0000 7fd51358bb80 1 -- 172.21.15.87:0/1989091928 wait complete. 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 -- start start 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 Processor -- start 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889585f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 0x558889570e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041aa8d0 con 0x558889586a10 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041abd40 con 0x5588895740f0 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041aafa0 con 0x55888957a2c0 2024-06-06T07:49:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd501d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:56778/0 (socket says 172.21.15.87:56778) 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 0x558889570e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd501d9b700 1 -- 172.21.15.87:0/1188643641 learned_addr learned my addr 172.21.15.87:0/1188643641 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd50259c700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889585f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.469+0000 7fd51358bb80 10 client.0 _ll_register_callbacks cb 0x558889456740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 -- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 msgr2=0x558889585f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889585f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 -- 172.21.15.87:0/1188643641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 msgr2=0x558889570e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 0x558889570e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 -- 172.21.15.87:0/1188643641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558889571510 con 0x55888957a2c0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd502d9d700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 0x558889570e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd501d9b700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80046f0 tx=0x7fd4f800fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70bb1b13d4f2cba9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd4e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f801e9c0 con 0x55888957a2c0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd4f801eb60 con 0x55888957a2c0 2024-06-06T07:49:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.470+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f8019050 con 0x55888957a2c0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4f803f050 con 0x55888957a2c0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 10 client.25231 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588895fd3e0 con 0x55888957a2c0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8019050 con 0x55888957a2c0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd4f80303c0 con 0x55888957a2c0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 10 client.25231 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 10 client.25231 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 20 client.25231 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588895fdad0 0x5588895ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 4 client.25231 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}] 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.471+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}) -- 0x5588894c8460 con 0x5588895fdad0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.472+0000 7fd50259c700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588895fdad0 0x5588895ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.472+0000 7fd4e9ffb700 10 client.25231 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.472+0000 7fd50259c700 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588895fdad0 0x5588895ffef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5079320347cefc7 server_cookie=6154ff36212c13b1 in_seq=0 out_seq=0 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.485+0000 7fd4e9ffb700 1 -- 172.21.15.87:0/1188643641 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5588894c8460 con 0x5588895fdad0 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.485+0000 7fd4e9ffb700 10 client.25231 handle_command_reply: tid=0 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.485+0000 7fd51358bb80 1 client.25231 shutdown 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 msgr2=0x5588895864d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80046f0 tx=0x7fd4f800fec0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 shutdown_connections 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588895fdad0 0x5588895ffef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588895740f0 0x558889585f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558889586a10 0x558889570e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 --2- 172.21.15.87:0/1188643641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55888957a2c0 0x5588895864d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd4e9ffb700 0 client.25231 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 >> 172.21.15.87:0/1188643641 conn(0x55888949c210 msgr2=0x558889578470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 shutdown_connections 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 1 -- 172.21.15.87:0/1188643641 wait complete. 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.486+0000 7fd51358bb80 20 client.25231 trim_cache size 0 max 16384 2024-06-06T07:49:14.487 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 27, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 2, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "544e979e": { 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 2, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:49:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 154.4, 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:14.489 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 msgr2=0x7fd4ec0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 0x7fd4ec0a5530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd504198f20 tx=0x7fd4f8011920 comp rx=0 tx=0).stop 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 msgr2=0x7fd504197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 0x7fd504197760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b0a0 tx=0x7fd4fc004d00 comp rx=0 tx=0).stop 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 shutdown_connections 2024-06-06T07:49:14.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd504107470 0x7fd504197760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4ec0a3070 0x7fd4ec0a5530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5041013c0 0x7fd50419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 --2- 172.21.15.87:0/1881890454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd504101d80 0x7fd50419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 >> 172.21.15.87:0/1881890454 conn(0x7fd5040783d0 msgr2=0x7fd5040ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 shutdown_connections 2024-06-06T07:49:14.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:14.487+0000 7fd508d08700 1 -- 172.21.15.87:0/1881890454 wait complete. 2024-06-06T07:49:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:14 smithi195 ceph-mon[37040]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-06-06T07:49:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:14 smithi181 ceph-mon[36790]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-06-06T07:49:15.451 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:15 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:49:15.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:15 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:49:15.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:15 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:49:15.872 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '544e979e', quiesce took 9.9 seconds. Will release after: 24.5 2024-06-06T07:49:16.498 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:16 smithi087 ceph-mon[32276]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:49:16.498 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:16 smithi087 ceph-mon[32276]: scrub summary: idle 2024-06-06T07:49:16.498 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:16 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:49:16.498 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:16 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:49:16.498 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:16 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:49:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:16 smithi195 ceph-mon[37040]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:49:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:16 smithi195 ceph-mon[37040]: scrub summary: idle 2024-06-06T07:49:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:16 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:49:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:16 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:49:16.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:16 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:49:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:16 smithi181 ceph-mon[36790]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T07:49:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:16 smithi181 ceph-mon[36790]: scrub summary: idle 2024-06-06T07:49:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:16 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:49:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:16 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:49:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:16 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:49:16.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 -- 172.21.15.87:0/2338151533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 msgr2=0x7f419006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 --2- 172.21.15.87:0/2338151533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419006afe0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f418000b0a0 tx=0x7f418000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 -- 172.21.15.87:0/2338151533 shutdown_connections 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 --2- 172.21.15.87:0/2338151533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 --2- 172.21.15.87:0/2338151533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 0x7f419010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 --2- 172.21.15.87:0/2338151533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419006afe0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 -- 172.21.15.87:0/2338151533 >> 172.21.15.87:0/2338151533 conn(0x7f41900783d0 msgr2=0x7f41900787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.881+0000 7f4196b7c700 1 -- 172.21.15.87:0/2338151533 shutdown_connections 2024-06-06T07:49:16.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.882+0000 7f4196b7c700 1 -- 172.21.15.87:0/2338151533 wait complete. 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.882+0000 7f4196b7c700 1 Processor -- start 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.882+0000 7f4196b7c700 1 -- start start 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 0x7f419019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4190112790 con 0x7f419006ab60 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4190112610 con 0x7f4190108530 2024-06-06T07:49:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4196b7c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4190112910 con 0x7f419006b520 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38258/0 (socket says 172.21.15.87:38258) 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 -- 172.21.15.87:0/339447927 learned_addr learned my addr 172.21.15.87:0/339447927 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f418ffff700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 0x7f419019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4195119700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 -- 172.21.15.87:0/339447927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 msgr2=0x7f419019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 0x7f419019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 -- 172.21.15.87:0/339447927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 msgr2=0x7f4190198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 -- 172.21.15.87:0/339447927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4190199190 con 0x7f419006ab60 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4195119700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.883+0000 7f4194918700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f418800c9b0 tx=0x7f418800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.884+0000 7f418dffb700 1 -- 172.21.15.87:0/339447927 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188007b00 con 0x7f419006ab60 2024-06-06T07:49:16.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.884+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4190199480 con 0x7f419006ab60 2024-06-06T07:49:16.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.884+0000 7f418dffb700 1 -- 172.21.15.87:0/339447927 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4188007ca0 con 0x7f419006ab60 2024-06-06T07:49:16.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.884+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901a56f0 con 0x7f419006ab60 2024-06-06T07:49:16.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.884+0000 7f418dffb700 1 -- 172.21.15.87:0/339447927 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188018730 con 0x7f419006ab60 2024-06-06T07:49:16.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.886+0000 7f418dffb700 1 -- 172.21.15.87:0/339447927 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f41880188d0 con 0x7f419006ab60 2024-06-06T07:49:16.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.887+0000 7f418dffb700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 0x7f417c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.887+0000 7f418ffff700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 0x7f417c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.887+0000 7f418dffb700 1 -- 172.21.15.87:0/339447927 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f41880cf9d0 con 0x7f419006ab60 2024-06-06T07:49:16.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.887+0000 7f418ffff700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 0x7f417c0a5480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4190078ab0 tx=0x7f418000b7b0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 Processor -- start 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 -- start start 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114f60 0x5586fd10e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd10ef20 0x5586fd10f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5586fd060810 con 0x5586fd114640 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5586fd008720 con 0x5586fd10ef20 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5586fd05c7d0 con 0x5586fd114f60 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38268/0 (socket says 172.21.15.87:38268) 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f4194918700 1 -- 172.21.15.87:0/314311486 learned_addr learned my addr 172.21.15.87:0/314311486 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:16.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.888+0000 7f4195119700 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd10ef20 0x5586fd10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f418ffff700 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114f60 0x5586fd10e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 -- 172.21.15.87:0/314311486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114f60 msgr2=0x5586fd10e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114f60 0x5586fd10e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 -- 172.21.15.87:0/314311486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd10ef20 msgr2=0x5586fd10f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd10ef20 0x5586fd10f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 -- 172.21.15.87:0/314311486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fcffef50 con 0x5586fd114640 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4194918700 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f418809b5e0 tx=0x7f4188097cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a68cd2c637d91 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4186ffd700 1 -- 172.21.15.87:0/314311486 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418809fc90 con 0x5586fd114640 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4186ffd700 1 -- 172.21.15.87:0/314311486 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41880a4550 con 0x5586fd114640 2024-06-06T07:49:16.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.889+0000 7f4186ffd700 1 -- 172.21.15.87:0/314311486 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418809bda0 con 0x5586fd114640 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- 172.21.15.87:0/314311486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 msgr2=0x5586fd114a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f418809b5e0 tx=0x7f4188097cb0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- 172.21.15.87:0/314311486 shutdown_connections 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd10ef20 0x5586fd10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd114a20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/314311486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114f60 0x5586fd10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- 172.21.15.87:0/314311486 >> 172.21.15.87:0/314311486 conn(0x5586fd039cc0 msgr2=0x5586fd109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- 172.21.15.87:0/314311486 shutdown_connections 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- 172.21.15.87:0/314311486 wait complete. 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 -- start start 2024-06-06T07:49:16.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.890+0000 7f41a13ffb80 1 Processor -- start 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd11acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd118510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5586fd11b970 con 0x5586fd114640 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5586fd000bf0 con 0x5586fd114f60 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5586fd11baf0 con 0x5586fd10ef20 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f41a13ffb80 10 client.0 _ll_register_callbacks cb 0x5586fcffa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38284/0 (socket says 172.21.15.87:38284) 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 -- 172.21.15.87:0/4051292936 learned_addr learned my addr 172.21.15.87:0/4051292936 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f4194918700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd11acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f4195119700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd118510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 -- 172.21.15.87:0/4051292936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 msgr2=0x5586fd11acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd11acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 -- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 msgr2=0x5586fd118510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd118510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.891+0000 7f418ffff700 1 -- 172.21.15.87:0/4051292936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fcfaf8d0 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f418ffff700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4180015000 tx=0x7f41800099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c2df9c8ec260b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418002d030 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4180023040 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41880a4550 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f418003a050 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 10 client.15759 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586fd12a680 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41880a4550 con 0x5586fd114640 2024-06-06T07:49:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4180011070 con 0x5586fd114640 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 20 client.15759 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd12f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 4 client.15759 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.892+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586fd06dd10 con 0x5586fd12cc60 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.893+0000 7f4194918700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd12f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.893+0000 7f416ffff700 10 client.15759 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.893+0000 7f4194918700 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd12f050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba0b99cfb74d2a0 server_cookie=4775374123298baf in_seq=0 out_seq=0 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f416ffff700 1 -- 172.21.15.87:0/4051292936 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5586fd06dd10 con 0x5586fd12cc60 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f416ffff700 10 client.15759 handle_command_reply: tid=0 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 client.15759 shutdown 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 msgr2=0x5586fd117fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4180015000 tx=0x7f41800099e0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 shutdown_connections 2024-06-06T07:49:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd12f050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f416ffff700 0 client.15759 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd118510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd11acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/4051292936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114640 0x5586fd117fd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 >> 172.21.15.87:0/4051292936 conn(0x5586fd039cc0 msgr2=0x5586fd1095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 shutdown_connections 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 1 -- 172.21.15.87:0/4051292936 wait complete. 2024-06-06T07:49:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.894+0000 7f41a13ffb80 20 client.15759 trim_cache size 0 max 16384 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 Processor -- start 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 -- start start 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 0x5586fd103690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd134900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41901aece0 con 0x5586fd10ef20 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41901ac570 con 0x5586fd114f60 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41901aa3c0 con 0x5586fd114640 2024-06-06T07:49:16.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38296/0 (socket says 172.21.15.87:38296) 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 -- 172.21.15.87:0/1006412902 learned_addr learned my addr 172.21.15.87:0/1006412902 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4195119700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f418ffff700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 0x5586fd103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 -- 172.21.15.87:0/1006412902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 msgr2=0x5586fd103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 0x5586fd103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 -- 172.21.15.87:0/1006412902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 msgr2=0x5586fd134900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd134900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4194918700 1 -- 172.21.15.87:0/1006412902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fd116560 con 0x5586fd10ef20 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f418ffff700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 0x5586fd103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.915+0000 7f4195119700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.916+0000 7f4194918700 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4188086010 tx=0x7f4188000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8d0a2b38fb4b17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.916+0000 7f41867fc700 1 -- 172.21.15.87:0/1006412902 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880b8070 con 0x5586fd10ef20 2024-06-06T07:49:16.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.916+0000 7f41867fc700 1 -- 172.21.15.87:0/1006412902 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418800de00 con 0x5586fd10ef20 2024-06-06T07:49:16.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.916+0000 7f41867fc700 1 -- 172.21.15.87:0/1006412902 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418809bd80 con 0x5586fd10ef20 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.916+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1006412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 msgr2=0x5586fd10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4188086010 tx=0x7f4188000bc0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1006412902 shutdown_connections 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114f60 0x5586fd134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd10ef20 0x5586fd10e9e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1006412902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd114640 0x5586fd103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1006412902 >> 172.21.15.87:0/1006412902 conn(0x5586fd039cc0 msgr2=0x5586fd1159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1006412902 shutdown_connections 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1006412902 wait complete. 2024-06-06T07:49:16.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 -- start start 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 Processor -- start 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.917+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd1345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114640 0x5586fd134b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41901aaa30 con 0x5586fd114f60 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41901abea0 con 0x5586fd114640 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41901ab100 con 0x5586fd10ef20 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 10 client.0 _ll_register_callbacks cb 0x5586fcffa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38312/0 (socket says 172.21.15.87:38312) 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 -- 172.21.15.87:0/1643447466 learned_addr learned my addr 172.21.15.87:0/1643447466 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4194918700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd1345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f418ffff700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114640 0x5586fd134b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 -- 172.21.15.87:0/1643447466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 msgr2=0x5586fd1345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd1345e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 -- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114640 msgr2=0x5586fd134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114640 0x5586fd134b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 -- 172.21.15.87:0/1643447466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fd1969a0 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f4195119700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f417800fa40 tx=0x7f417800fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf93a3310d32a493 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f416ffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178020060 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f417800ebb0 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.918+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418800de00 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f417800edf0 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 10 client.15765 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586fd197c60 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417801b8d0 con 0x5586fd114f60 2024-06-06T07:49:16.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f417801a030 con 0x5586fd114f60 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 20 client.15765 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd19a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 4 client.15765 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f418800de00 con 0x5586fd12cc60 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.919+0000 7f4194918700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f4194918700 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd19a410 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48d2fb3d03b9c1 server_cookie=59ebfbb04003755b in_seq=0 out_seq=0 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f416ffff700 10 client.15765 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f416ffff700 1 -- 172.21.15.87:0/1643447466 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f418800de00 con 0x5586fd12cc60 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f416ffff700 10 client.15765 handle_command_reply: tid=0 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 client.15765 shutdown 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 msgr2=0x5586fd135060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f417800fa40 tx=0x7f417800fd50 comp rx=0 tx=0).stop 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 shutdown_connections 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5586fd12cc60 0x5586fd19a410 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5586fd114f60 0x5586fd135060 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586fd10ef20 0x5586fd1345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 --2- 172.21.15.87:0/1643447466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586fd114640 0x5586fd134b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 >> 172.21.15.87:0/1643447466 conn(0x5586fd039cc0 msgr2=0x5586fd1168e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f416ffff700 0 client.15765 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 shutdown_connections 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 1 -- 172.21.15.87:0/1643447466 wait complete. 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.920+0000 7f41a13ffb80 20 client.15765 trim_cache size 0 max 16384 2024-06-06T07:49:16.925 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 msgr2=0x7f417c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 0x7f417c0a5480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4190078ab0 tx=0x7f418000b7b0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 msgr2=0x7f419019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f418800c9b0 tx=0x7f418800ccc0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 shutdown_connections 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190108530 0x7f4190198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f417c0a2fc0 0x7f417c0a5480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f419006ab60 0x7f419019e820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 --2- 172.21.15.87:0/339447927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419006b520 0x7f419019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 >> 172.21.15.87:0/339447927 conn(0x7f41900783d0 msgr2=0x7f41901000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 shutdown_connections 2024-06-06T07:49:16.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:16.921+0000 7f4196b7c700 1 -- 172.21.15.87:0/339447927 wait complete. 2024-06-06T07:49:17.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:49:17.668 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:49:17.918 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:18.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:18 smithi087 ceph-mon[32276]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:18 smithi195 ceph-mon[37040]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:18 smithi181 ceph-mon[36790]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.758+0000 7f4521568700 1 -- 172.21.15.87:0/1323642408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c1085d0 msgr2=0x7f451c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.758+0000 7f4521568700 1 --2- 172.21.15.87:0/1323642408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c1085d0 0x7f451c1089b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f450c0099c0 tx=0x7f450c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.758+0000 7f4521568700 1 -- 172.21.15.87:0/1323642408 shutdown_connections 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.758+0000 7f4521568700 1 --2- 172.21.15.87:0/1323642408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 0x7f451c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.758+0000 7f4521568700 1 --2- 172.21.15.87:0/1323642408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c1085d0 0x7f451c1089b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.759+0000 7f4521568700 1 --2- 172.21.15.87:0/1323642408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c102530 0x7f451c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.759+0000 7f4521568700 1 -- 172.21.15.87:0/1323642408 >> 172.21.15.87:0/1323642408 conn(0x7f451c079640 msgr2=0x7f451c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.759+0000 7f4521568700 1 -- 172.21.15.87:0/1323642408 shutdown_connections 2024-06-06T07:49:19.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.759+0000 7f4521568700 1 -- 172.21.15.87:0/1323642408 wait complete. 2024-06-06T07:49:19.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.759+0000 7f4521568700 1 Processor -- start 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 -- start start 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 0x7f451c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c1085d0 0x7f451c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f451c112890 con 0x7f451c102530 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f451c112710 con 0x7f451c102ef0 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f4521568700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f451c112a10 con 0x7f451c1085d0 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49482/0 (socket says 172.21.15.87:49482) 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 -- 172.21.15.87:0/788989202 learned_addr learned my addr 172.21.15.87:0/788989202 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:19.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451a7fc700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 0x7f451c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451b7fe700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c1085d0 0x7f451c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 -- 172.21.15.87:0/788989202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c1085d0 msgr2=0x7f451c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c1085d0 0x7f451c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 -- 172.21.15.87:0/788989202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 msgr2=0x7f451c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 0x7f451c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.760+0000 7f451affd700 1 -- 172.21.15.87:0/788989202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c1991b0 con 0x7f451c102530 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f451affd700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f450c009fd0 tx=0x7f450c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c027070 con 0x7f451c102530 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c199440 con 0x7f451c102530 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c0040d0 con 0x7f451c102530 2024-06-06T07:49:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1a5760 con 0x7f451c102530 2024-06-06T07:49:19.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.761+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f460 con 0x7f451c102530 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.762+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c103c40 con 0x7f451c102530 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.763+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f450c02a070 con 0x7f451c102530 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.764+0000 7f4503fff700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 0x7f45040a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.765+0000 7f451a7fc700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 0x7f45040a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.765+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f450c0d0af0 con 0x7f451c102530 2024-06-06T07:49:19.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.765+0000 7f451a7fc700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 0x7f45040a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f451c079bb0 tx=0x7f451000a300 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:19.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:19.769+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f450c05e850 con 0x7f451c102530 2024-06-06T07:49:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.051+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451c1a5930 con 0x7f451c102530 2024-06-06T07:49:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.052+0000 7f4503fff700 1 -- 172.21.15.87:0/788989202 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f450c013070 con 0x7f451c102530 2024-06-06T07:49:20.052 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:49:20.052 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:49:20.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 msgr2=0x7f45040a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:20.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 0x7f45040a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f451c079bb0 tx=0x7f451000a300 comp rx=0 tx=0).stop 2024-06-06T07:49:20.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 msgr2=0x7f451c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:20.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f450c009fd0 tx=0x7f450c003680 comp rx=0 tx=0).stop 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 shutdown_connections 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45040a2f40 0x7f45040a5400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f451c1085d0 0x7f451c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f451c102530 0x7f451c19e890 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 --2- 172.21.15.87:0/788989202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c102ef0 0x7f451c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.054+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 >> 172.21.15.87:0/788989202 conn(0x7f451c079640 msgr2=0x7f451c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.055+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 shutdown_connections 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:20.055+0000 7f4521568700 1 -- 172.21.15.87:0/788989202 wait complete. 2024-06-06T07:49:20.055 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:49:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:20 smithi087 ceph-mon[32276]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:20 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/788989202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:20 smithi195 ceph-mon[37040]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:20 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/788989202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:20 smithi181 ceph-mon[36790]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:20 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/788989202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:20.879 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:49:21.132 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 -- 172.21.15.87:0/929563497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c106890 msgr2=0x7f484c106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 --2- 172.21.15.87:0/929563497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c106890 0x7f484c106c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4840009930 tx=0x7f484002a420 comp rx=0 tx=0).stop 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 -- 172.21.15.87:0/929563497 shutdown_connections 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 --2- 172.21.15.87:0/929563497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c101290 0x7f484c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 --2- 172.21.15.87:0/929563497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 0x7f484c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 --2- 172.21.15.87:0/929563497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c106890 0x7f484c106c70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 -- 172.21.15.87:0/929563497 >> 172.21.15.87:0/929563497 conn(0x7f484c077a20 msgr2=0x7f484c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 -- 172.21.15.87:0/929563497 shutdown_connections 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.594+0000 7f48531b7640 1 -- 172.21.15.87:0/929563497 wait complete. 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 Processor -- start 2024-06-06T07:49:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 -- start start 2024-06-06T07:49:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 0x7f484c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 0x7f484c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f484c110ca0 con 0x7f484c101290 2024-06-06T07:49:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f484c110b20 con 0x7f484c106890 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f48531b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f484c110e20 con 0x7f484c1008d0 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f484bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f484bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49502/0 (socket says 172.21.15.87:49502) 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f484bfff640 1 -- 172.21.15.87:0/1779313938 learned_addr learned my addr 172.21.15.87:0/1779313938 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f4850f53640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 0x7f484c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.595+0000 7f4851754640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 0x7f484c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 -- 172.21.15.87:0/1779313938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 msgr2=0x7f484c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 0x7f484c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 -- 172.21.15.87:0/1779313938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 msgr2=0x7f484c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 0x7f484c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 -- 172.21.15.87:0/1779313938 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c19c0d0 con 0x7f484c101290 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f4851754640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 0x7f484c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f484bfff640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f483800e9f0 tx=0x7f483800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f4849ffb640 1 -- 172.21.15.87:0/1779313938 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483800cd80 con 0x7f484c101290 2024-06-06T07:49:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c19c3c0 con 0x7f484c101290 2024-06-06T07:49:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f4849ffb640 1 -- 172.21.15.87:0/1779313938 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4838005500 con 0x7f484c101290 2024-06-06T07:49:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f4849ffb640 1 -- 172.21.15.87:0/1779313938 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838010690 con 0x7f484c101290 2024-06-06T07:49:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.596+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c069050 con 0x7f484c101290 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 Processor -- start 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 -- start start 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e119550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e123e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f13e07d4f0 con 0x55f13e124350 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f13e01b870 con 0x55f13e119a90 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4855fa2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f13e0794b0 con 0x55f13e119170 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4849ffb640 1 -- 172.21.15.87:0/1779313938 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f48380046c0 con 0x7f484c101290 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f484bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4851754640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e119550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f4850f53640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e123e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f484bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49514/0 (socket says 172.21.15.87:49514) 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.599+0000 7f484bfff640 1 -- 172.21.15.87:0/3222461469 learned_addr learned my addr 172.21.15.87:0/3222461469 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 -- 172.21.15.87:0/3222461469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 msgr2=0x55f13e119550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e119550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 -- 172.21.15.87:0/3222461469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 msgr2=0x55f13e123e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e123e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 -- 172.21.15.87:0/3222461469 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13e0379b0 con 0x55f13e124350 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4850f53640 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e123e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4851754640 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e119550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4849ffb640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 0x7f483c0a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f484bfff640 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f48380cd070 tx=0x7f48380d80e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89222f0421f33d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4851754640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 0x7f483c0a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4849ffb640 1 -- 172.21.15.87:0/1779313938 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f48380cff90 con 0x7f484c101290 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4836ffd640 1 -- 172.21.15.87:0/3222461469 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838098140 con 0x55f13e124350 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4836ffd640 1 -- 172.21.15.87:0/3222461469 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483809c5a0 con 0x55f13e124350 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.600+0000 7f4836ffd640 1 -- 172.21.15.87:0/3222461469 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483809b800 con 0x55f13e124350 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- 172.21.15.87:0/3222461469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 msgr2=0x55f13e126740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4851754640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 0x7f483c0a7620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4844007fd0 tx=0x7f48440076a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f48380cd070 tx=0x7f48380d80e0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- 172.21.15.87:0/3222461469 shutdown_connections 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e126740 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e123e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 --2- 172.21.15.87:0/3222461469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e119550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- 172.21.15.87:0/3222461469 >> 172.21.15.87:0/3222461469 conn(0x55f13e115680 msgr2=0x55f13e115a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- 172.21.15.87:0/3222461469 shutdown_connections 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- 172.21.15.87:0/3222461469 wait complete. 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.601+0000 7f4855fa2740 1 -- start start 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 Processor -- start 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e121350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e121890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f13e126f80 con 0x55f13e124350 2024-06-06T07:49:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f13e006ae0 con 0x55f13e119a90 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f13e127100 con 0x55f13e119170 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4855fa2740 10 client.0 _ll_register_callbacks cb 0x55f13e03f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4851754640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4851754640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49522/0 (socket says 172.21.15.87:49522) 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.602+0000 7f4851754640 1 -- 172.21.15.87:0/500721320 learned_addr learned my addr 172.21.15.87:0/500721320 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 -- 172.21.15.87:0/500721320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 msgr2=0x55f13e121350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 -- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 msgr2=0x55f13e121890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 -- 172.21.15.87:0/500721320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13dd91210 con 0x55f13e124350 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4851754640 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4844007310 tx=0x7f4844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942a7b28914fb6bb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844004d20 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4844005020 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483809c5a0 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4844005220 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f13de5d620 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483809c5a0 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f484401e830 con 0x55f13e124350 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.603+0000 7f4855fa2740 20 client.15783 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e134600 0x55f13e136a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f4855fa2740 4 client.15783 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f13e090f10 con 0x55f13e134600 2024-06-06T07:49:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f4850f53640 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e134600 0x55f13e136a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f4850f53640 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e134600 0x55f13e136a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b916d8643158c599 server_cookie=a1d5351861cf5420 in_seq=0 out_seq=0 2024-06-06T07:49:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.604+0000 7f482bfff640 10 client.15783 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f482bfff640 1 -- 172.21.15.87:0/500721320 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f13e090f10 con 0x55f13e134600 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f482bfff640 10 client.15783 handle_command_reply: tid=0 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 client.15783 shutdown 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 msgr2=0x55f13e121dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4844007310 tx=0x7f4844004a40 comp rx=0 tx=0).stop 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 shutdown_connections 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e134600 0x55f13e136a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e121dd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e121890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 --2- 172.21.15.87:0/500721320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f482bfff640 0 client.15783 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 >> 172.21.15.87:0/500721320 conn(0x55f13e115680 msgr2=0x55f13e11b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 shutdown_connections 2024-06-06T07:49:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 1 -- 172.21.15.87:0/500721320 wait complete. 2024-06-06T07:49:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.605+0000 7f4855fa2740 20 client.15783 trim_cache size 0 max 16384 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 Processor -- start 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 -- start start 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119a90 0x55f13e13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e124350 0x55f13e13c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1af600 con 0x55f13e119170 2024-06-06T07:49:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1aced0 con 0x55f13e124350 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4855fa2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1aad20 con 0x55f13e119a90 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49538/0 (socket says 172.21.15.87:49538) 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 -- 172.21.15.87:0/1755163505 learned_addr learned my addr 172.21.15.87:0/1755163505 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4851754640 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e124350 0x55f13e13c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f484bfff640 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119a90 0x55f13e13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 -- 172.21.15.87:0/1755163505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119a90 msgr2=0x55f13e13c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119a90 0x55f13e13c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 -- 172.21.15.87:0/1755163505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e124350 msgr2=0x55f13e13c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e124350 0x55f13e13c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.635+0000 7f4850f53640 1 -- 172.21.15.87:0/1755163505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13e119880 con 0x55f13e119170 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.636+0000 7f4850f53640 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f484002a940 tx=0x7f4840005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730fc5a4d4b44ef7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.636+0000 7f48367fc640 1 -- 172.21.15.87:0/1755163505 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840048030 con 0x55f13e119170 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.636+0000 7f48367fc640 1 -- 172.21.15.87:0/1755163505 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48400319f0 con 0x55f13e119170 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.636+0000 7f48367fc640 1 -- 172.21.15.87:0/1755163505 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840035040 con 0x55f13e119170 2024-06-06T07:49:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- 172.21.15.87:0/1755163505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 msgr2=0x55f13e11b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f484002a940 tx=0x7f4840005fd0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- 172.21.15.87:0/1755163505 shutdown_connections 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e124350 0x55f13e13c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119a90 0x55f13e13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1755163505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e119170 0x55f13e11b230 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- 172.21.15.87:0/1755163505 >> 172.21.15.87:0/1755163505 conn(0x55f13e115680 msgr2=0x55f13e11ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- 172.21.15.87:0/1755163505 shutdown_connections 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- 172.21.15.87:0/1755163505 wait complete. 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 -- start start 2024-06-06T07:49:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.637+0000 7f4855fa2740 1 Processor -- start 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e12c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e127bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1ab390 con 0x55f13e124350 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1ac800 con 0x55f13e119a90 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1aba60 con 0x55f13e119170 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4855fa2740 10 client.0 _ll_register_callbacks cb 0x7f4844007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49548/0 (socket says 172.21.15.87:49548) 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 -- 172.21.15.87:0/1662155564 learned_addr learned my addr 172.21.15.87:0/1662155564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f484bfff640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e127bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4850f53640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e12c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 -- 172.21.15.87:0/1662155564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 msgr2=0x55f13e12c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e12c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 -- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 msgr2=0x55f13e127bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e127bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 -- 172.21.15.87:0/1662155564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13e128660 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f4851754640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f484401e010 tx=0x7f4844005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab255a9c162cc1d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f482bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.638+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484403a070 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4844004cb0 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48400319f0 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f484401dc10 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f13e19f150 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48400319f0 con 0x55f13e124350 2024-06-06T07:49:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4844004040 con 0x55f13e124350 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 20 client.15789 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e19f8a0 0x55f13e1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 4 client.15789 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f13e08ac20 con 0x55f13e19f8a0 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.639+0000 7f4850f53640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e19f8a0 0x55f13e1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.640+0000 7f4850f53640 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e19f8a0 0x55f13e1a1cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc1aa7fc83b2b3a server_cookie=9eb4f5e7fa273b66 in_seq=0 out_seq=0 2024-06-06T07:49:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.640+0000 7f482bfff640 10 client.15789 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f482bfff640 1 -- 172.21.15.87:0/1662155564 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f13e08ac20 con 0x55f13e19f8a0 2024-06-06T07:49:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f482bfff640 10 client.15789 handle_command_reply: tid=0 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 client.15789 shutdown 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 msgr2=0x55f13e1280f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f484401e010 tx=0x7f4844005670 comp rx=0 tx=0).stop 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 shutdown_connections 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f13e19f8a0 0x55f13e1a1cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f482bfff640 0 client.15789 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f13e124350 0x55f13e1280f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f13e119a90 0x55f13e127bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 --2- 172.21.15.87:0/1662155564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f13e119170 0x55f13e12c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 >> 172.21.15.87:0/1662155564 conn(0x55f13e115680 msgr2=0x55f13e11b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 shutdown_connections 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 1 -- 172.21.15.87:0/1662155564 wait complete. 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.647+0000 7f4855fa2740 20 client.15789 trim_cache size 0 max 16384 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:49:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:49:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 msgr2=0x7f483c0a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 0x7f483c0a7620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4844007fd0 tx=0x7f48440076a0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 msgr2=0x7f484c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f483800e9f0 tx=0x7f483800edf0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 shutdown_connections 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f483c0a5160 0x7f483c0a7620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f484c106890 0x7f484c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f484c101290 0x7f484c197620 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 --2- 172.21.15.87:0/1779313938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f484c1008d0 0x7f484c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 >> 172.21.15.87:0/1779313938 conn(0x7f484c077a20 msgr2=0x7f484c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 shutdown_connections 2024-06-06T07:49:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:21.648+0000 7f48531b7640 1 -- 172.21.15.87:0/1779313938 wait complete. 2024-06-06T07:49:22.392 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:22 smithi087 ceph-mon[32276]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:22.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:22 smithi195 ceph-mon[37040]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:22 smithi181 ceph-mon[36790]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 -- 172.21.15.87:0/3142935559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c101df0 msgr2=0x7f854c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 --2- 172.21.15.87:0/3142935559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c101df0 0x7f854c1021d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f85480099c0 tx=0x7f8548009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 -- 172.21.15.87:0/3142935559 shutdown_connections 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 --2- 172.21.15.87:0/3142935559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c106750 0x7f854c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 --2- 172.21.15.87:0/3142935559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c101df0 0x7f854c1021d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 --2- 172.21.15.87:0/3142935559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 0x7f854c106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 -- 172.21.15.87:0/3142935559 >> 172.21.15.87:0/3142935559 conn(0x7f854c077a00 msgr2=0x7f854c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 -- 172.21.15.87:0/3142935559 shutdown_connections 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 -- 172.21.15.87:0/3142935559 wait complete. 2024-06-06T07:49:22.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.972+0000 7f855452c700 1 Processor -- start 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 -- start start 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c101df0 0x7f854c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 0x7f854c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f854c111710 con 0x7f854c106750 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f854c111590 con 0x7f854c101df0 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f855452c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f854c111890 con 0x7f854c1027a0 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49572/0 (socket says 172.21.15.87:49572) 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 -- 172.21.15.87:0/1447405220 learned_addr learned my addr 172.21.15.87:0/1447405220 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8551ac7700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 0x7f854c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f85522c8700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c101df0 0x7f854c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 -- 172.21.15.87:0/1447405220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 msgr2=0x7f854c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 0x7f854c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 -- 172.21.15.87:0/1447405220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c101df0 msgr2=0x7f854c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c101df0 0x7f854c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.973+0000 7f8552ac9700 1 -- 172.21.15.87:0/1447405220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c199250 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f8552ac9700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f854400b970 tx=0x7f854400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f85437fe700 1 -- 172.21.15.87:0/1447405220 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854401d070 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c199540 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f85437fe700 1 -- 172.21.15.87:0/1447405220 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544010d40 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f85437fe700 1 -- 172.21.15.87:0/1447405220 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544007790 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.974+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c0fcda0 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.976+0000 7f85437fe700 1 -- 172.21.15.87:0/1447405220 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f8544020070 con 0x7f854c106750 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.977+0000 7f85437fe700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 0x7f85380a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.977+0000 7f85522c8700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 0x7f85380a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.977+0000 7f85437fe700 1 -- 172.21.15.87:0/1447405220 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f85440d3220 con 0x7f854c106750 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.977+0000 7f85522c8700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 0x7f85380a5400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f8548005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.977+0000 7f855edafb80 1 Processor -- start 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 -- start start 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a2e19c4810 con 0x55a2e1a78f60 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a2e196c720 con 0x55a2e1a78640 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f855edafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a2e19c07d0 con 0x55a2e1a72f20 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49586/0 (socket says 172.21.15.87:49586) 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 -- 172.21.15.87:0/543634990 learned_addr learned my addr 172.21.15.87:0/543634990 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f85522c8700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8552ac9700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 -- 172.21.15.87:0/543634990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 msgr2=0x55a2e1a73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a73340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 -- 172.21.15.87:0/543634990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 msgr2=0x55a2e1a78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a78a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f8551ac7700 1 -- 172.21.15.87:0/543634990 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e1962f50 con 0x55a2e1a78f60 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.978+0000 7f85522c8700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f8551ac7700 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f853c00ac70 tx=0x7f853c00af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358b5f95cc5573f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f8540ff9700 1 -- 172.21.15.87:0/543634990 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c01d070 con 0x55a2e1a78f60 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f8540ff9700 1 -- 172.21.15.87:0/543634990 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c00fe90 con 0x55a2e1a78f60 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f8540ff9700 1 -- 172.21.15.87:0/543634990 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c010860 con 0x55a2e1a78f60 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f855edafb80 1 -- 172.21.15.87:0/543634990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 msgr2=0x55a2e1a729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.979+0000 7f855edafb80 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f853c00ac70 tx=0x7f853c00af80 comp rx=0 tx=0).stop 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- 172.21.15.87:0/543634990 shutdown_connections 2024-06-06T07:49:22.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- 172.21.15.87:0/543634990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a729e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- 172.21.15.87:0/543634990 >> 172.21.15.87:0/543634990 conn(0x55a2e199dcc0 msgr2=0x55a2e1a6d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- 172.21.15.87:0/543634990 shutdown_connections 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- 172.21.15.87:0/543634990 wait complete. 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- start start 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 Processor -- start 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a72f20 0x55a2e1a7ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a7c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a2e1a7f9d0 con 0x55a2e1a78f60 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a2e1964bf0 con 0x55a2e1a72f20 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a2e1a7fb50 con 0x55a2e1a78640 2024-06-06T07:49:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.980+0000 7f855edafb80 10 client.0 _ll_register_callbacks cb 0x55a2e195e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:41484/0 (socket says 172.21.15.87:41484) 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 -- 172.21.15.87:0/2714919747 learned_addr learned my addr 172.21.15.87:0/2714919747 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f85522c8700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a72f20 0x55a2e1a7ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 -- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a72f20 msgr2=0x55a2e1a7ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a72f20 0x55a2e1a7ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 -- 172.21.15.87:0/2714919747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 msgr2=0x55a2e1a7c510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a7c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.981+0000 7f8551ac7700 1 -- 172.21.15.87:0/2714919747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e19138d0 con 0x55a2e1a78640 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c01b040 tx=0x7f853c018b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=277392428b40fa7d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c01d040 con 0x55a2e1a78640 2024-06-06T07:49:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f853c00fe90 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f853c00fc90 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f853c030050 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 10 client.25267 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2e1a8a2d0 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c00fe90 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f853c020030 con 0x55a2e1a78640 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 10 client.25267 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 10 client.25267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 20 client.25267 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1a8c800 0x55a2e1a8ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 4 client.25267 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.982+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e19d1d10 con 0x55a2e1a8c800 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.983+0000 7f85522c8700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1a8c800 0x55a2e1a8ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.983+0000 7f8535ffb700 10 client.25267 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.983+0000 7f85522c8700 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1a8c800 0x55a2e1a8ebf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c2fcf1cbcce245c server_cookie=f6290c2d291c9c4a in_seq=0 out_seq=0 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f8535ffb700 1 -- 172.21.15.87:0/2714919747 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a2e19d1d10 con 0x55a2e1a8c800 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f8535ffb700 10 client.25267 handle_command_reply: tid=0 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 client.25267 shutdown 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 msgr2=0x55a2e1a7bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c01b040 tx=0x7f853c018b10 comp rx=0 tx=0).stop 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 shutdown_connections 2024-06-06T07:49:23.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a72f20 0x55a2e1a7ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1a8c800 0x55a2e1a8ebf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a7c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 --2- 172.21.15.87:0/2714919747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78640 0x55a2e1a7bfd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f8535ffb700 0 client.25267 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 >> 172.21.15.87:0/2714919747 conn(0x55a2e199dcc0 msgr2=0x55a2e1a719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 shutdown_connections 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 1 -- 172.21.15.87:0/2714919747 wait complete. 2024-06-06T07:49:23.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:22.984+0000 7f855edafb80 20 client.25267 trim_cache size 0 max 16384 2024-06-06T07:49:23.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 Processor -- start 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 -- start start 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 0x55a2e1a729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78f60 0x55a2e1a79400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1aed70 con 0x55a2e1a72f20 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1ac600 con 0x55a2e1a78640 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f855edafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1aa450 con 0x55a2e1a78f60 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f85522c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 0x55a2e1a729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f8551ac7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41556/0 (socket says 172.21.15.87:41556) 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.000+0000 7f8551ac7700 1 -- 172.21.15.87:0/2594873163 learned_addr learned my addr 172.21.15.87:0/2594873163 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8552ac9700 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78f60 0x55a2e1a79400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 -- 172.21.15.87:0/2594873163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78f60 msgr2=0x55a2e1a79400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78f60 0x55a2e1a79400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 -- 172.21.15.87:0/2594873163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 msgr2=0x55a2e1a729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 0x55a2e1a729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 -- 172.21.15.87:0/2594873163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e1a78d70 con 0x55a2e1a78640 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f85522c8700 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 0x55a2e1a729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8551ac7700 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f853c010790 tx=0x7f853c018da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d510248d59626471 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8537fff700 1 -- 172.21.15.87:0/2594873163 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c039070 con 0x55a2e1a78640 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8537fff700 1 -- 172.21.15.87:0/2594873163 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c005d50 con 0x55a2e1a78640 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.001+0000 7f8537fff700 1 -- 172.21.15.87:0/2594873163 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c01d070 con 0x55a2e1a78640 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- 172.21.15.87:0/2594873163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 msgr2=0x55a2e1a7c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f853c010790 tx=0x7f853c018da0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- 172.21.15.87:0/2594873163 shutdown_connections 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a78f60 0x55a2e1a79400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a72f20 0x55a2e1a729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 --2- 172.21.15.87:0/2594873163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a7c6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- 172.21.15.87:0/2594873163 >> 172.21.15.87:0/2594873163 conn(0x55a2e199dcc0 msgr2=0x55a2e1a7bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- 172.21.15.87:0/2594873163 shutdown_connections 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- 172.21.15.87:0/2594873163 wait complete. 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.002+0000 7f855edafb80 1 -- start start 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 Processor -- start 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a85530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1aaac0 con 0x55a2e1a78f60 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1abf30 con 0x55a2e1a78640 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1ab190 con 0x55a2e1a72f20 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f855edafb80 10 client.0 _ll_register_callbacks cb 0x55a2e195e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:23.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49598/0 (socket says 172.21.15.87:49598) 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 -- 172.21.15.87:0/631735625 learned_addr learned my addr 172.21.15.87:0/631735625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f85522c8700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8551ac7700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a85530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 -- 172.21.15.87:0/631735625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 msgr2=0x55a2e1a84ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a84ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 -- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 msgr2=0x55a2e1a85530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a85530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.003+0000 7f8552ac9700 1 -- 172.21.15.87:0/631735625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e1a93cb0 con 0x55a2e1a78f60 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8552ac9700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f854409bc10 tx=0x7f854409f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57123b807fb9548e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440a3370 con 0x55a2e1a78f60 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f853c005d50 con 0x55a2e1a78f60 2024-06-06T07:49:23.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f85440a7a20 con 0x55a2e1a78f60 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85440bd050 con 0x55a2e1a78f60 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 10 client.15804 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2e1af6bd0 con 0x55a2e1a78f60 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c005d50 con 0x55a2e1a78f60 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f854409e690 con 0x55a2e1a78f60 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 10 client.15804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 20 client.15804 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1af72b0 0x55a2e1af96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 4 client.15804 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.004+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2e19cc290 con 0x55a2e1af72b0 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f85522c8700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1af72b0 0x55a2e1af96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f8535ffb700 10 client.15804 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:23.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f85522c8700 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1af72b0 0x55a2e1af96d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da16c117af668922 server_cookie=f729f7a1ebdd8c3 in_seq=0 out_seq=0 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f8535ffb700 1 -- 172.21.15.87:0/631735625 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2e19cc290 con 0x55a2e1af72b0 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f8535ffb700 10 client.15804 handle_command_reply: tid=0 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 client.15804 shutdown 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 msgr2=0x55a2e1a93610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f854409bc10 tx=0x7f854409f140 comp rx=0 tx=0).stop 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 shutdown_connections 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x55a2e1af72b0 0x55a2e1af96d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a2e1a78f60 0x55a2e1a93610 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2e1a72f20 0x55a2e1a84ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 --2- 172.21.15.87:0/631735625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a2e1a78640 0x55a2e1a85530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 >> 172.21.15.87:0/631735625 conn(0x55a2e199dcc0 msgr2=0x55a2e1a79340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f8535ffb700 0 client.15804 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 shutdown_connections 2024-06-06T07:49:23.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.005+0000 7f855edafb80 1 -- 172.21.15.87:0/631735625 wait complete. 2024-06-06T07:49:23.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855edafb80 20 client.15804 trim_cache size 0 max 16384 2024-06-06T07:49:23.012 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 msgr2=0x7f85380a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 0x7f85380a5400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f8548005fb0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 msgr2=0x7f854c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f854400b970 tx=0x7f854400bc80 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 shutdown_connections 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f854c101df0 0x7f854c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f85380a2f40 0x7f85380a5400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f854c106750 0x7f854c198af0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 --2- 172.21.15.87:0/1447405220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f854c1027a0 0x7f854c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 >> 172.21.15.87:0/1447405220 conn(0x7f854c077a00 msgr2=0x7f854c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 shutdown_connections 2024-06-06T07:49:23.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:23.006+0000 7f855452c700 1 -- 172.21.15.87:0/1447405220 wait complete. 2024-06-06T07:49:23.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:49:24.152 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:24.421 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:24 smithi087 ceph-mon[32276]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:24 smithi195 ceph-mon[37040]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:24 smithi181 ceph-mon[36790]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:26.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3781985421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101390 msgr2=0x7ff5f41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3781985421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101390 0x7ff5f41017f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009990 tx=0x7ff5e8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3781985421 shutdown_connections 2024-06-06T07:49:26.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3781985421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f41073d0 0x7ff5f41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3781985421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f4101d30 0x7ff5f410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3781985421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101390 0x7ff5f41017f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3781985421 >> 172.21.15.87:0/3781985421 conn(0x7ff5f40782c0 msgr2=0x7ff5f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3781985421 shutdown_connections 2024-06-06T07:49:26.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.035+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3781985421 wait complete. 2024-06-06T07:49:26.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 Processor -- start 2024-06-06T07:49:26.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 -- start start 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f4101390 0x7ff5f4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f41073d0 0x7ff5f41a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4111630 con 0x7ff5f4101d30 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41114b0 con 0x7ff5f4101390 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f9b4e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41117b0 con 0x7ff5f41073d0 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49616/0 (socket says 172.21.15.87:49616) 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/3278736828 learned_addr learned my addr 172.21.15.87:0/3278736828 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:26.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.036+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f4101390 0x7ff5f4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f3fff700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f41073d0 0x7ff5f41a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/3278736828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f41073d0 msgr2=0x7ff5f41a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f41073d0 0x7ff5f41a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/3278736828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f4101390 msgr2=0x7ff5f4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f4101390 0x7ff5f4199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/3278736828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41a5030 con 0x7ff5f4101d30 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80038b0 tx=0x7ff5e8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f0ff9700 1 -- 172.21.15.87:0/3278736828 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8027070 con 0x7ff5f4101d30 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f0ff9700 1 -- 172.21.15.87:0/3278736828 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8004050 con 0x7ff5f4101d30 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f0ff9700 1 -- 172.21.15.87:0/3278736828 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8022c00 con 0x7ff5f4101d30 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f4197860 con 0x7ff5f4101d30 2024-06-06T07:49:26.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.037+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f4197cc0 con 0x7ff5f4101d30 2024-06-06T07:49:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.040+0000 7ff5f0ff9700 1 -- 172.21.15.87:0/3278736828 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7ff5e802a070 con 0x7ff5f4101d30 2024-06-06T07:49:26.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.041+0000 7ff5f0ff9700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 0x7ff5dc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.041+0000 7ff5f0ff9700 1 -- 172.21.15.87:0/3278736828 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff5e80d08e0 con 0x7ff5f4101d30 2024-06-06T07:49:26.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.041+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 0x7ff5dc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.041+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 0x7ff5dc0a5480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400fcd0 tx=0x7ff5e400f7a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 Processor -- start 2024-06-06T07:49:26.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 -- start start 2024-06-06T07:49:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559ba0c2a810 con 0x559ba0cdef60 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bd2720 con 0x559ba0cde640 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559ba0c267d0 con 0x559ba0cd8f20 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49628/0 (socket says 172.21.15.87:49628) 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/2029454778 learned_addr learned my addr 172.21.15.87:0/2029454778 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/2029454778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 msgr2=0x559ba0cd9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/2029454778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 msgr2=0x559ba0cdea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.043+0000 7ff5f2ffd700 1 -- 172.21.15.87:0/2029454778 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba0bc8f50 con 0x559ba0cdef60 2024-06-06T07:49:26.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.044+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8098ce0 tx=0x7ff5e8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee3d964c61e5731 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.044+0000 7ff5d9ffb700 1 -- 172.21.15.87:0/2029454778 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e809cf80 con 0x559ba0cdef60 2024-06-06T07:49:26.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.044+0000 7ff5d9ffb700 1 -- 172.21.15.87:0/2029454778 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e809c880 con 0x559ba0cdef60 2024-06-06T07:49:26.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.044+0000 7ff5d9ffb700 1 -- 172.21.15.87:0/2029454778 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e809cd40 con 0x559ba0cdef60 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- 172.21.15.87:0/2029454778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 msgr2=0x559ba0cd89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8098ce0 tx=0x7ff5e8009310 comp rx=0 tx=0).stop 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- 172.21.15.87:0/2029454778 shutdown_connections 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/2029454778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cd89e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- 172.21.15.87:0/2029454778 >> 172.21.15.87:0/2029454778 conn(0x559ba0c03cc0 msgr2=0x559ba0cd3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- 172.21.15.87:0/2029454778 shutdown_connections 2024-06-06T07:49:26.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- 172.21.15.87:0/2029454778 wait complete. 2024-06-06T07:49:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.045+0000 7ff6043d1b80 1 -- start start 2024-06-06T07:49:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 Processor -- start 2024-06-06T07:49:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0ce2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cdef60 0x559ba0ce2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559ba0ce56f0 con 0x559ba0cd8f20 2024-06-06T07:49:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bcabf0 con 0x559ba0cde640 2024-06-06T07:49:26.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559ba0ce5870 con 0x559ba0cdef60 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff5f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff5f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49640/0 (socket says 172.21.15.87:49640) 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff5f37fe700 1 -- 172.21.15.87:0/4031487589 learned_addr learned my addr 172.21.15.87:0/4031487589 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff6043d1b80 10 client.0 _ll_register_callbacks cb 0x559ba0bc4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.046+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0ce2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 -- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cdef60 msgr2=0x559ba0ce2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cdef60 0x559ba0ce2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 -- 172.21.15.87:0/4031487589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 msgr2=0x559ba0ce2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0ce2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 -- 172.21.15.87:0/4031487589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba0b798d0 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0ce2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4023ac0 tx=0x7ff5e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a44a4ee1874e57 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4021a00 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff5e4021ba0 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e809c880 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5e4037050 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff6043d1b80 10 client.15816 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.047+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ba0cefff0 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e809c880 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff5e402b070 con 0x559ba0cd8f20 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:26.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 10 client.15816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 20 client.15816 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0cf4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 4 client.15816 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba0c37d10 con 0x559ba0cf2630 2024-06-06T07:49:26.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.048+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0cf4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.049+0000 7ff5d2ffd700 10 client.15816 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.049+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0cf4a80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76eaa6b3f3240e41 server_cookie=e4b5c4409ffb78e1 in_seq=0 out_seq=0 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/4031487589 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ba0c37d10 con 0x559ba0cf2630 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff5d2ffd700 10 client.15816 handle_command_reply: tid=0 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 client.15816 shutdown 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 msgr2=0x559ba0ce4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4023ac0 tx=0x7ff5e40047c0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 shutdown_connections 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cdef60 0x559ba0ce2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ce4d30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0cf4a80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff5d2ffd700 0 client.15816 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/4031487589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0ce2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 >> 172.21.15.87:0/4031487589 conn(0x559ba0c03cc0 msgr2=0x559ba0cd79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 shutdown_connections 2024-06-06T07:49:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.050+0000 7ff6043d1b80 1 -- 172.21.15.87:0/4031487589 wait complete. 2024-06-06T07:49:26.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.051+0000 7ff6043d1b80 20 client.15816 trim_cache size 0 max 16384 2024-06-06T07:49:26.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 Processor -- start 2024-06-06T07:49:26.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 -- start start 2024-06-06T07:49:26.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41aed10 con 0x559ba0cdef60 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41ac5a0 con 0x559ba0cde640 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41aa3f0 con 0x559ba0cd8f20 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49652/0 (socket says 172.21.15.87:49652) 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f3fff700 1 -- 172.21.15.87:0/1217151031 learned_addr learned my addr 172.21.15.87:0/1217151031 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 -- 172.21.15.87:0/1217151031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 msgr2=0x559ba0cdfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 -- 172.21.15.87:0/1217151031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 msgr2=0x559ba0cfa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f37fe700 1 -- 172.21.15.87:0/1217151031 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba0ce05a0 con 0x559ba0cd8f20 2024-06-06T07:49:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.066+0000 7ff5f3fff700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.067+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd89e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402c000 tx=0x7ff5e4021720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=834cb62fb5694a74 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.067+0000 7ff5d97fa700 1 -- 172.21.15.87:0/1217151031 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e403f070 con 0x559ba0cd8f20 2024-06-06T07:49:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.067+0000 7ff5d97fa700 1 -- 172.21.15.87:0/1217151031 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e4018e50 con 0x559ba0cd8f20 2024-06-06T07:49:26.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.067+0000 7ff5d97fa700 1 -- 172.21.15.87:0/1217151031 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4004150 con 0x559ba0cd8f20 2024-06-06T07:49:26.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- 172.21.15.87:0/1217151031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 msgr2=0x559ba0cd89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd89e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402c000 tx=0x7ff5e4021720 comp rx=0 tx=0).stop 2024-06-06T07:49:26.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- 172.21.15.87:0/1217151031 shutdown_connections 2024-06-06T07:49:26.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cdef60 0x559ba0cfa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cd8f20 0x559ba0cd89e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/1217151031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cde640 0x559ba0cdfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- 172.21.15.87:0/1217151031 >> 172.21.15.87:0/1217151031 conn(0x559ba0c03cc0 msgr2=0x559ba0cdf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- 172.21.15.87:0/1217151031 shutdown_connections 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- 172.21.15.87:0/1217151031 wait complete. 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 -- start start 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.068+0000 7ff6043d1b80 1 Processor -- start 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cde640 0x559ba0cf9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 0x559ba0cf9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41aaa60 con 0x559ba0cd8f20 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41abed0 con 0x559ba0cdef60 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41ab130 con 0x559ba0cde640 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff6043d1b80 10 client.0 _ll_register_callbacks cb 0x559ba0bc4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:26.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49668/0 (socket says 172.21.15.87:49668) 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cde640 0x559ba0cf9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 -- 172.21.15.87:0/3065664409 learned_addr learned my addr 172.21.15.87:0/3065664409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f3fff700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 0x559ba0cf9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 -- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cde640 msgr2=0x559ba0cf9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cde640 0x559ba0cf9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 -- 172.21.15.87:0/3065664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 msgr2=0x559ba0cf9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 0x559ba0cf9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f37fe700 1 -- 172.21.15.87:0/3065664409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba0cfa570 con 0x559ba0cd8f20 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.069+0000 7ff5f3fff700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 0x559ba0cf9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5f37fe700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff5e401d040 tx=0x7ff5e4021e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af460d7d8329dfc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e403f040 con 0x559ba0cd8f20 2024-06-06T07:49:26.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e4018e50 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ff5e40050b0 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5e4052050 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff6043d1b80 10 client.15822 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ba0d5d0f0 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.070+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e4018e50 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff5e402b040 con 0x559ba0cd8f20 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 20 client.15822 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0d5f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 4 client.15822 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ba0c32290 con 0x559ba0cf2630 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0d5f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:26.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff5d2ffd700 10 client.15822 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.071+0000 7ff5f2ffd700 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0d5f770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=319a5a4bd2d3b426 server_cookie=78cceb3ec79f015e in_seq=0 out_seq=0 2024-06-06T07:49:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff5d2ffd700 1 -- 172.21.15.87:0/3065664409 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ba0c32290 con 0x559ba0cf2630 2024-06-06T07:49:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff5d2ffd700 10 client.15822 handle_command_reply: tid=0 2024-06-06T07:49:26.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 client.15822 shutdown 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 msgr2=0x559ba0ceb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff5e401d040 tx=0x7ff5e4021e30 comp rx=0 tx=0).stop 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 shutdown_connections 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x559ba0cf2630 0x559ba0d5f770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559ba0cdef60 0x559ba0cf9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559ba0cd8f20 0x559ba0ceb040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 --2- 172.21.15.87:0/3065664409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ba0cde640 0x559ba0cf9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 >> 172.21.15.87:0/3065664409 conn(0x559ba0c03cc0 msgr2=0x559ba0cdf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff5d2ffd700 0 client.15822 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 shutdown_connections 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 1 -- 172.21.15.87:0/3065664409 wait complete. 2024-06-06T07:49:26.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.072+0000 7ff6043d1b80 20 client.15822 trim_cache size 0 max 16384 2024-06-06T07:49:26.077 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 msgr2=0x7ff5dc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 0x7ff5dc0a5480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400fcd0 tx=0x7ff5e400f7a0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 msgr2=0x7ff5f4199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80038b0 tx=0x7ff5e8009f50 comp rx=0 tx=0).stop 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 shutdown_connections 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5f4101390 0x7ff5f4199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff5dc0a2fc0 0x7ff5dc0a5480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f41073d0 0x7ff5f41a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 --2- 172.21.15.87:0/3278736828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5f4101d30 0x7ff5f4199fb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 >> 172.21.15.87:0/3278736828 conn(0x7ff5f40782c0 msgr2=0x7ff5f4100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 shutdown_connections 2024-06-06T07:49:26.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:26.073+0000 7ff5f9b4e700 1 -- 172.21.15.87:0/3278736828 wait complete. 2024-06-06T07:49:26.140 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:26 smithi087 ceph-mon[32276]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T07:49:26.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:26 smithi195 ceph-mon[37040]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T07:49:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:26 smithi181 ceph-mon[36790]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T07:49:27.046 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:49:27.342 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:28.490 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:28 smithi087 ceph-mon[32276]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:28 smithi195 ceph-mon[37040]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:28 smithi181 ceph-mon[36790]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T07:49:29.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 -- 172.21.15.87:0/2544280826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30101510 msgr2=0x7fbe3010d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 --2- 172.21.15.87:0/2544280826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30101510 0x7fbe3010d9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800b0a0 tx=0x7fbe2800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 -- 172.21.15.87:0/2544280826 shutdown_connections 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 --2- 172.21.15.87:0/2544280826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 0x7fbe30106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 --2- 172.21.15.87:0/2544280826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30101510 0x7fbe3010d9d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 --2- 172.21.15.87:0/2544280826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe30100b50 0x7fbe30100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 -- 172.21.15.87:0/2544280826 >> 172.21.15.87:0/2544280826 conn(0x7fbe30077a00 msgr2=0x7fbe300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 -- 172.21.15.87:0/2544280826 shutdown_connections 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.135+0000 7fbe3723b700 1 -- 172.21.15.87:0/2544280826 wait complete. 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 Processor -- start 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 -- start start 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe30101510 0x7fbe3019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 0x7fbe301989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.136+0000 7fbe3723b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30110e50 con 0x7fbe30100b50 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe3723b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30110cd0 con 0x7fbe30106b90 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe3723b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30110fd0 con 0x7fbe30101510 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51484/0 (socket says 172.21.15.87:51484) 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 -- 172.21.15.87:0/3663394000 learned_addr learned my addr 172.21.15.87:0/3663394000 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe357d8700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 0x7fbe301989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 -- 172.21.15.87:0/3663394000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe30101510 msgr2=0x7fbe3019ed30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe30101510 0x7fbe3019ed30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 -- 172.21.15.87:0/3663394000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 msgr2=0x7fbe301989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 0x7fbe301989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 -- 172.21.15.87:0/3663394000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe30199110 con 0x7fbe30100b50 2024-06-06T07:49:29.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.137+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9a0 tx=0x7fbe2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.138+0000 7fbe2dffb700 1 -- 172.21.15.87:0/3663394000 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400cbd0 con 0x7fbe30100b50 2024-06-06T07:49:29.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.138+0000 7fbe2dffb700 1 -- 172.21.15.87:0/3663394000 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400cd70 con 0x7fbe30100b50 2024-06-06T07:49:29.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.138+0000 7fbe2dffb700 1 -- 172.21.15.87:0/3663394000 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24018750 con 0x7fbe30100b50 2024-06-06T07:49:29.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.138+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe30199400 con 0x7fbe30100b50 2024-06-06T07:49:29.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.138+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe301a56c0 con 0x7fbe30100b50 2024-06-06T07:49:29.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.141+0000 7fbe2dffb700 1 -- 172.21.15.87:0/3663394000 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fbe24010430 con 0x7fbe30100b50 2024-06-06T07:49:29.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe2dffb700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 0x7fbe200a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe41abeb80 1 Processor -- start 2024-06-06T07:49:29.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe41abeb80 1 -- start start 2024-06-06T07:49:29.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe2dffb700 1 -- 172.21.15.87:0/3663394000 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fbe240cfcf0 con 0x7fbe30100b50 2024-06-06T07:49:29.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 0x7fbe200a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.142+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 0x559bcc9f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559bcc8e0f60 con 0x559bcc9f2bb0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559bcc93f850 con 0x559bcc9f8c90 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559bcc8a7460 con 0x559bcc9f8370 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe34fd7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 0x7fbe200a5480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe30104a10 tx=0x7fbe1c009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51490/0 (socket says 172.21.15.87:51490) 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 -- 172.21.15.87:0/520140641 learned_addr learned my addr 172.21.15.87:0/520140641 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 0x559bcc9f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 -- 172.21.15.87:0/520140641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 msgr2=0x559bcc9f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 -- 172.21.15.87:0/520140641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 msgr2=0x559bcc9f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 0x559bcc9f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe357d8700 1 -- 172.21.15.87:0/520140641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bcc88e8d0 con 0x559bcc9f2bb0 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.143+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 0x559bcc9f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:29.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.144+0000 7fbe357d8700 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009e60 tx=0x7fbe280092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227085f18914c52 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.147+0000 7fbe1affd700 1 -- 172.21.15.87:0/520140641 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28020070 con 0x559bcc9f2bb0 2024-06-06T07:49:29.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.147+0000 7fbe1affd700 1 -- 172.21.15.87:0/520140641 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe28006020 con 0x559bcc9f2bb0 2024-06-06T07:49:29.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.147+0000 7fbe1affd700 1 -- 172.21.15.87:0/520140641 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007c50 con 0x559bcc9f2bb0 2024-06-06T07:49:29.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 -- 172.21.15.87:0/520140641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 msgr2=0x559bcc9f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009e60 tx=0x7fbe280092e0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 -- 172.21.15.87:0/520140641 shutdown_connections 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f2ff0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/520140641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f8c90 0x559bcc9f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 -- 172.21.15.87:0/520140641 >> 172.21.15.87:0/520140641 conn(0x559bcc91b080 msgr2=0x559bcc9e9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 -- 172.21.15.87:0/520140641 shutdown_connections 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.148+0000 7fbe41abeb80 1 -- 172.21.15.87:0/520140641 wait complete. 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.149+0000 7fbe41abeb80 1 -- start start 2024-06-06T07:49:29.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.149+0000 7fbe41abeb80 1 Processor -- start 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.149+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.149+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 0x559bcc9fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559bcc8dffc0 con 0x559bcc9f8c90 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559bcc93b810 con 0x559bcc9f2bb0 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559bcc9ff620 con 0x559bcc9f8370 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe41abeb80 10 client.0 _ll_register_callbacks cb 0x559bcc8d55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:29.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 0x559bcc9fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe34fd7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54164/0 (socket says 172.21.15.87:54164) 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe2ffff700 1 -- 172.21.15.87:0/205952661 learned_addr learned my addr 172.21.15.87:0/205952661 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 -- 172.21.15.87:0/205952661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 msgr2=0x559bcc9fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 -- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 msgr2=0x559bcc9fea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9fea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 -- 172.21.15.87:0/205952661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bcc905750 con 0x559bcc9f8c90 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:29.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe357d8700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 0x559bcc9fc270 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009e30 tx=0x7fbe28003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e93ddc30161c8e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.150+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28020040 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fbe28007ce0 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe28006020 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe28009500 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe41abeb80 10 client.15834 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bcca09df0 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe28006020 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe28011070 con 0x559bcc9f8c90 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe41abeb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.151+0000 7fbe41abeb80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.152+0000 7fbe41abeb80 20 client.15834 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.152+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca0c2c0 0x559bcca0e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.152+0000 7fbe41abeb80 4 client.15834 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:29.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.152+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bcc94cd50 con 0x559bcca0c2c0 2024-06-06T07:49:29.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.152+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca0c2c0 0x559bcca0e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.153+0000 7fbe0ffff700 10 client.15834 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.153+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca0c2c0 0x559bcca0e6b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8182a340494f40e3 server_cookie=2fe2a417517c29b6 in_seq=0 out_seq=0 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.153+0000 7fbe0ffff700 1 -- 172.21.15.87:0/205952661 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559bcc94cd50 con 0x559bcca0c2c0 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.153+0000 7fbe0ffff700 10 client.15834 handle_command_reply: tid=0 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.153+0000 7fbe41abeb80 1 client.15834 shutdown 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 msgr2=0x559bcc9fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 0x559bcc9fc270 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009e30 tx=0x7fbe28003c30 comp rx=0 tx=0).stop 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 shutdown_connections 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca0c2c0 0x559bcca0e6b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8c90 0x559bcc9fc270 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/205952661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8370 0x559bcc9fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 >> 172.21.15.87:0/205952661 conn(0x559bcc91b080 msgr2=0x559bcc9e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe0ffff700 0 client.15834 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 shutdown_connections 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 1 -- 172.21.15.87:0/205952661 wait complete. 2024-06-06T07:49:29.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.154+0000 7fbe41abeb80 20 client.15834 trim_cache size 0 max 16384 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.182+0000 7fbe41abeb80 1 Processor -- start 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.182+0000 7fbe41abeb80 1 -- start start 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcc9e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301aec20 con 0x559bcc9f8370 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301ac4b0 con 0x559bcc9f2bb0 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301aa300 con 0x559bcc9f8c90 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51520/0 (socket says 172.21.15.87:51520) 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 -- 172.21.15.87:0/463375944 learned_addr learned my addr 172.21.15.87:0/463375944 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe357d8700 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcc9e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 -- 172.21.15.87:0/463375944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 msgr2=0x559bcc9e93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcc9e93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 -- 172.21.15.87:0/463375944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 msgr2=0x559bcc9f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.183+0000 7fbe2ffff700 1 -- 172.21.15.87:0/463375944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bcc9f7e50 con 0x559bcc9f8370 2024-06-06T07:49:29.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.184+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ec10 tx=0x7fbe1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995cace6c1496ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.184+0000 7fbe1a7fc700 1 -- 172.21.15.87:0/463375944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c01d840 con 0x559bcc9f8370 2024-06-06T07:49:29.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.184+0000 7fbe1a7fc700 1 -- 172.21.15.87:0/463375944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1c004830 con 0x559bcc9f8370 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.184+0000 7fbe1a7fc700 1 -- 172.21.15.87:0/463375944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c0051a0 con 0x559bcc9f8370 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 -- 172.21.15.87:0/463375944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 msgr2=0x559bcc9e8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ec10 tx=0x7fbe1c011000 comp rx=0 tx=0).stop 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 -- 172.21.15.87:0/463375944 shutdown_connections 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcc9f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcc9e93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/463375944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcc9e8e80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.185+0000 7fbe41abeb80 1 -- 172.21.15.87:0/463375944 >> 172.21.15.87:0/463375944 conn(0x559bcc91b080 msgr2=0x559bcc9f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.186+0000 7fbe41abeb80 1 -- 172.21.15.87:0/463375944 shutdown_connections 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.186+0000 7fbe41abeb80 1 -- 172.21.15.87:0/463375944 wait complete. 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.186+0000 7fbe41abeb80 1 -- start start 2024-06-06T07:49:29.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.186+0000 7fbe41abeb80 1 Processor -- start 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcca0e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcca0ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301aa970 con 0x559bcc9f8370 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301abde0 con 0x559bcc9f2bb0 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301ab040 con 0x559bcc9f8c90 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe41abeb80 10 client.0 _ll_register_callbacks cb 0x559bcc9fc910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe34fd7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcca0e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51534/0 (socket says 172.21.15.87:51534) 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.187+0000 7fbe2ffff700 1 -- 172.21.15.87:0/4048163801 learned_addr learned my addr 172.21.15.87:0/4048163801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 -- 172.21.15.87:0/4048163801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 msgr2=0x559bcca0ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcca0ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 -- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 msgr2=0x559bcca0e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcca0e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 -- 172.21.15.87:0/4048163801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bcca74e50 con 0x559bcc9f8370 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe2ffff700 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0059d0 tx=0x7fbe1c01de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32adff7bb008c37b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:29.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1c004830 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe0ffff700 1 -- 172.21.15.87:0/4048163801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c005110 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe0ffff700 1 -- 172.21.15.87:0/4048163801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fbe1c009040 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe0ffff700 1 -- 172.21.15.87:0/4048163801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c0052b0 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe41abeb80 10 client.15840 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.188+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({fsmap=27}) -- 0x559bcca76020 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe0ffff700 1 -- 172.21.15.87:0/4048163801 <== mon.0 v2:172.21.15.87:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe1c02f070 con 0x559bcc9f8370 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 20 client.15840 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca76500 0x559bcca78920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 4 client.15840 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:29.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbe1c004830 con 0x559bcca76500 2024-06-06T07:49:29.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.189+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca76500 0x559bcca78920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe0ffff700 10 client.15840 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe34fd7700 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca76500 0x559bcca78920 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3672a9626d56f2 server_cookie=ff0fc07fca12fcc4 in_seq=0 out_seq=0 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe0ffff700 1 -- 172.21.15.87:0/4048163801 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbe1c004830 con 0x559bcca76500 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe0ffff700 10 client.15840 handle_command_reply: tid=0 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 client.15840 shutdown 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 msgr2=0x559bcca0e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0059d0 tx=0x7fbe1c01de60 comp rx=0 tx=0).stop 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 shutdown_connections 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559bcca76500 0x559bcca78920 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559bcc9f2bb0 0x559bcca0e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcc9f8c90 0x559bcca0ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe0ffff700 0 client.15840 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:29.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 --2- 172.21.15.87:0/4048163801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bcc9f8370 0x559bcca0e850 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 >> 172.21.15.87:0/4048163801 conn(0x559bcc91b080 msgr2=0x559bcc9ed250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 shutdown_connections 2024-06-06T07:49:29.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 1 -- 172.21.15.87:0/4048163801 wait complete. 2024-06-06T07:49:29.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.190+0000 7fbe41abeb80 20 client.15840 trim_cache size 0 max 16384 2024-06-06T07:49:29.194 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 msgr2=0x7fbe200a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 0x7fbe200a5480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe30104a10 tx=0x7fbe1c009260 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 msgr2=0x7fbe3019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400e9a0 tx=0x7fbe2400ecb0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 shutdown_connections 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe30106b90 0x7fbe301989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbe200a2fc0 0x7fbe200a5480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe30100b50 0x7fbe3019e7f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 --2- 172.21.15.87:0/3663394000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe30101510 0x7fbe3019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:29.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 >> 172.21.15.87:0/3663394000 conn(0x7fbe30077a00 msgr2=0x7fbe301039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:29.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 shutdown_connections 2024-06-06T07:49:29.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:29.191+0000 7fbe3723b700 1 -- 172.21.15.87:0/3663394000 wait complete. 2024-06-06T07:49:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:30 smithi087 ceph-mon[32276]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:30 smithi195 ceph-mon[37040]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:30 smithi181 ceph-mon[36790]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:31.005 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:49:31.006 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:49:31.287 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:32.424 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:32 smithi087 ceph-mon[32276]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:32.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:32 smithi195 ceph-mon[37040]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:32 smithi181 ceph-mon[36790]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T07:49:33.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 -- 172.21.15.87:0/3344324551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 msgr2=0x7f980010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 --2- 172.21.15.87:0/3344324551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980010f380 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b0a0 tx=0x7f97f800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 -- 172.21.15.87:0/3344324551 shutdown_connections 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 --2- 172.21.15.87:0/3344324551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98001085f0 0x7f98001089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 --2- 172.21.15.87:0/3344324551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980010f380 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 --2- 172.21.15.87:0/3344324551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001024e0 0x7f9800102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 -- 172.21.15.87:0/3344324551 >> 172.21.15.87:0/3344324551 conn(0x7f9800079640 msgr2=0x7f9800079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.214+0000 7f9806841700 1 -- 172.21.15.87:0/3344324551 shutdown_connections 2024-06-06T07:49:33.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.215+0000 7f9806841700 1 -- 172.21.15.87:0/3344324551 wait complete. 2024-06-06T07:49:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.215+0000 7f9806841700 1 Processor -- start 2024-06-06T07:49:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.215+0000 7f9806841700 1 -- start start 2024-06-06T07:49:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98001024e0 0x7f980019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001085f0 0x7f98001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51562/0 (socket says 172.21.15.87:51562) 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9800112790 con 0x7f9800102ea0 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 -- 172.21.15.87:0/750508815 learned_addr learned my addr 172.21.15.87:0/750508815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9800112610 con 0x7f98001024e0 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9800112910 con 0x7f98001085f0 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f9804dde700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001085f0 0x7f98001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 -- 172.21.15.87:0/750508815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001085f0 msgr2=0x7f98001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001085f0 0x7f98001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 -- 172.21.15.87:0/750508815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98001024e0 msgr2=0x7f980019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98001024e0 0x7f980019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.216+0000 7f97ff7fe700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9800199110 con 0x7f9800102ea0 2024-06-06T07:49:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.217+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f97f400e9a0 tx=0x7f97f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.217+0000 7f97fd7fa700 1 -- 172.21.15.87:0/750508815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f400cbd0 con 0x7f9800102ea0 2024-06-06T07:49:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.217+0000 7f97fd7fa700 1 -- 172.21.15.87:0/750508815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f400cd70 con 0x7f9800102ea0 2024-06-06T07:49:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.217+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9800199400 con 0x7f9800102ea0 2024-06-06T07:49:33.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.218+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001a56c0 con 0x7f9800102ea0 2024-06-06T07:49:33.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.219+0000 7f97fd7fa700 1 -- 172.21.15.87:0/750508815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4018750 con 0x7f9800102ea0 2024-06-06T07:49:33.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.221+0000 7f97fd7fa700 1 -- 172.21.15.87:0/750508815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f97f40188f0 con 0x7f9800102ea0 2024-06-06T07:49:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.222+0000 7f97fd7fa700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 0x7f97e80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.222+0000 7f97fd7fa700 1 -- 172.21.15.87:0/750508815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f97f40cfa80 con 0x7f9800102ea0 2024-06-06T07:49:33.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.223+0000 7f97fffff700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 0x7f97e80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.224+0000 7f97fffff700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 0x7f97e80a52d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9800199af0 tx=0x7f97f0009620 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.224+0000 7f98110c4b80 1 Processor -- start 2024-06-06T07:49:33.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.224+0000 7f98110c4b80 1 -- start start 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1267900 0x55c9b12612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1261c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c9b1153fe0 con 0x55c9b1261820 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f97fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c9b11b28d0 con 0x55c9b1266fe0 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f98110c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9b111a4e0 con 0x55c9b1267900 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f97fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12673c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:52928/0 (socket says 172.21.15.87:52928) 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f97fffff700 1 -- 172.21.15.87:0/4138615251 learned_addr learned my addr 172.21.15.87:0/4138615251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:33.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1261c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1267900 0x55c9b12612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 -- 172.21.15.87:0/4138615251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1267900 msgr2=0x55c9b12612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1267900 0x55c9b12612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 -- 172.21.15.87:0/4138615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 msgr2=0x55c9b12673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.225+0000 7f9804dde700 1 -- 172.21.15.87:0/4138615251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b1101950 con 0x55c9b1261820 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.226+0000 7f9804dde700 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1261c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b070 tx=0x7f97f80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b57d88b1b35c197 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.226+0000 7f97e67fc700 1 -- 172.21.15.87:0/4138615251 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8020070 con 0x55c9b1261820 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.226+0000 7f97e67fc700 1 -- 172.21.15.87:0/4138615251 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f800bcb0 con 0x55c9b1261820 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.226+0000 7f97e67fc700 1 -- 172.21.15.87:0/4138615251 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8007cf0 con 0x55c9b1261820 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 -- 172.21.15.87:0/4138615251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 msgr2=0x55c9b1261c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1261c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b070 tx=0x7f97f80092e0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 -- 172.21.15.87:0/4138615251 shutdown_connections 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1261c40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 --2- 172.21.15.87:0/4138615251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1267900 0x55c9b12612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 -- 172.21.15.87:0/4138615251 >> 172.21.15.87:0/4138615251 conn(0x55c9b118e100 msgr2=0x55c9b125c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 -- 172.21.15.87:0/4138615251 shutdown_connections 2024-06-06T07:49:33.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.227+0000 7f98110c4b80 1 -- 172.21.15.87:0/4138615251 wait complete. 2024-06-06T07:49:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.228+0000 7f98110c4b80 1 -- start start 2024-06-06T07:49:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.228+0000 7f98110c4b80 1 Processor -- start 2024-06-06T07:49:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.228+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b126d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.228+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b126a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.228+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b126aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f9804dde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b126aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f9804dde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b126aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51586/0 (socket says 172.21.15.87:51586) 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f9804dde700 1 -- 172.21.15.87:0/1946301685 learned_addr learned my addr 172.21.15.87:0/1946301685 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b126a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97fffff700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b126d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f98110c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c9b1153040 con 0x55c9b1267900 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c9b11ae890 con 0x55c9b1266fe0 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9b126e280 con 0x55c9b1261820 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f98110c4b80 10 client.0 _ll_register_callbacks cb 0x55c9b1148630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 -- 172.21.15.87:0/1946301685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 msgr2=0x55c9b126d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b126d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 -- 172.21.15.87:0/1946301685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 msgr2=0x55c9b126aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b126aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b11787d0 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b126a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97f40cce40 tx=0x7f97f409c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11ee202e0985546 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.229+0000 7f97df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f409d000 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f97f40a5380 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f800bcb0 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97f409cba0 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 10 client.25250 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9b1278a70 con 0x55c9b1266fe0 2024-06-06T07:49:33.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f40a0f50 con 0x55c9b1266fe0 2024-06-06T07:49:33.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f97f409ba60 con 0x55c9b1266fe0 2024-06-06T07:49:33.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:33.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 20 client.25250 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b127d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 4 client.25250 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.230+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f97f800bcb0 con 0x55c9b127b000 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.231+0000 7f97fffff700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b127d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.231+0000 7f97df7fe700 10 client.25250 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.231+0000 7f97fffff700 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b127d450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f7acf90d1f3863 server_cookie=79dc347e6885ad4f in_seq=0 out_seq=0 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f97df7fe700 1 -- 172.21.15.87:0/1946301685 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f97f800bcb0 con 0x55c9b127b000 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f97df7fe700 10 client.25250 handle_command_reply: tid=0 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 client.25250 shutdown 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 msgr2=0x55c9b126a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b126a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97f40cce40 tx=0x7f97f409c500 comp rx=0 tx=0).stop 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 shutdown_connections 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b127d450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b126aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f97df7fe700 0 client.25250 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b126d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 --2- 172.21.15.87:0/1946301685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b126a9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 >> 172.21.15.87:0/1946301685 conn(0x55c9b118e100 msgr2=0x55c9b125b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 shutdown_connections 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.232+0000 7f98110c4b80 1 -- 172.21.15.87:0/1946301685 wait complete. 2024-06-06T07:49:33.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.233+0000 7f98110c4b80 20 client.25250 trim_cache size 0 max 16384 2024-06-06T07:49:33.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 Processor -- start 2024-06-06T07:49:33.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 -- start start 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b125bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b0e80 con 0x55c9b1267900 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f98001ae710 con 0x55c9b1266fe0 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.274+0000 7f98110c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98001ac560 con 0x55c9b1261820 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f9804dde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b125bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54234/0 (socket says 172.21.15.87:54234) 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f9804dde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b125bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51600/0 (socket says 172.21.15.87:51600) 2024-06-06T07:49:33.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 -- 172.21.15.87:0/2635194042 learned_addr learned my addr 172.21.15.87:0/2635194042 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 -- 172.21.15.87:0/2635194042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 msgr2=0x55c9b12683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 -- 172.21.15.87:0/2635194042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 msgr2=0x55c9b125bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b125bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 -- 172.21.15.87:0/2635194042 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b1268930 con 0x55c9b1261820 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97fffff700 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97f000efc0 tx=0x7f97f0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e46a2bf5db3311e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.275+0000 7f97e5ffb700 1 -- 172.21.15.87:0/2635194042 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004a30 con 0x55c9b1261820 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f97e5ffb700 1 -- 172.21.15.87:0/2635194042 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f0010ca0 con 0x55c9b1261820 2024-06-06T07:49:33.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f97e5ffb700 1 -- 172.21.15.87:0/2635194042 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0005680 con 0x55c9b1261820 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f98110c4b80 1 -- 172.21.15.87:0/2635194042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 msgr2=0x55c9b12612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f98110c4b80 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97f000efc0 tx=0x7f97f0011000 comp rx=0 tx=0).stop 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f98110c4b80 1 -- 172.21.15.87:0/2635194042 shutdown_connections 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.276+0000 7f98110c4b80 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1267900 0x55c9b125bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1261820 0x55c9b12612e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 --2- 172.21.15.87:0/2635194042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1266fe0 0x55c9b12683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 -- 172.21.15.87:0/2635194042 >> 172.21.15.87:0/2635194042 conn(0x55c9b118e100 msgr2=0x55c9b1267ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 -- 172.21.15.87:0/2635194042 shutdown_connections 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 -- 172.21.15.87:0/2635194042 wait complete. 2024-06-06T07:49:33.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 -- start start 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.277+0000 7f98110c4b80 1 Processor -- start 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1272570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1267900 0x55c9b1272ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98001acbd0 con 0x55c9b1261820 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f98001ae040 con 0x55c9b1267900 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98001ad2a0 con 0x55c9b1266fe0 2024-06-06T07:49:33.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f98110c4b80 10 client.0 _ll_register_callbacks cb 0x55c9b1148630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54246/0 (socket says 172.21.15.87:54246) 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f97ff7fe700 1 -- 172.21.15.87:0/229779348 learned_addr learned my addr 172.21.15.87:0/229779348 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f97fffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1272570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.278+0000 7f9804dde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1267900 0x55c9b1272ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 -- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1267900 msgr2=0x55c9b1272ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1267900 0x55c9b1272ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 -- 172.21.15.87:0/229779348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 msgr2=0x55c9b1272570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1272570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 -- 172.21.15.87:0/229779348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b1273830 con 0x55c9b1266fe0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97fffff700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1272570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97ff7fe700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97f400c5b0 tx=0x7f97f40a5610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc87b61f917025e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:33.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40a5810 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f97f40bb2c0 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f0010ca0 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97f40a0f50 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f98110c4b80 10 client.25315 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.279+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9b12ea860 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f0010ca0 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f97f4082020 con 0x55c9b1266fe0 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 10 client.25315 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 20 client.25315 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b12ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 4 client.25315 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:49:33.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9b11bfdd0 con 0x55c9b127b000 2024-06-06T07:49:33.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.280+0000 7f97fffff700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b12ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.281+0000 7f97df7fe700 10 client.25315 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.281+0000 7f97fffff700 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b12ece70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09050f5e3291201 server_cookie=833a3217b676d371 in_seq=0 out_seq=0 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.282+0000 7f97df7fe700 1 -- 172.21.15.87:0/229779348 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9b11bfdd0 con 0x55c9b127b000 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.282+0000 7f97df7fe700 10 client.25315 handle_command_reply: tid=0 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.282+0000 7f98110c4b80 1 client.25315 shutdown 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.282+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 msgr2=0x55c9b1272ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.282+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97f400c5b0 tx=0x7f97f40a5610 comp rx=0 tx=0).stop 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 shutdown_connections 2024-06-06T07:49:33.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c9b127b000 0x55c9b12ece70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b1267900 0x55c9b1272ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f97df7fe700 0 client.25315 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9b1261820 0x55c9b1272570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 --2- 172.21.15.87:0/229779348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9b1266fe0 0x55c9b1272ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 >> 172.21.15.87:0/229779348 conn(0x55c9b118e100 msgr2=0x55c9b1269710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 shutdown_connections 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.283+0000 7f98110c4b80 1 -- 172.21.15.87:0/229779348 wait complete. 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.284+0000 7f98110c4b80 20 client.25315 trim_cache size 0 max 16384 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:33.286 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-06-06T07:49:33.287 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "8e12779d-2059-4e65-9741-cbb763a3018e", 2024-06-06T07:49:33.287 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-06-06T07:49:33.287 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:33.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.284+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 msgr2=0x7f97e80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.284+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 0x7f97e80a52d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9800199af0 tx=0x7f97f0009620 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.284+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 msgr2=0x7f980019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.284+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f97f400e9a0 tx=0x7f97f400ecb0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 shutdown_connections 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98001024e0 0x7f980019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97e80a2e10 0x7f97e80a52d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98001085f0 0x7f98001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 --2- 172.21.15.87:0/750508815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9800102ea0 0x7f980019ed30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 >> 172.21.15.87:0/750508815 conn(0x7f9800079640 msgr2=0x7f9800100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 shutdown_connections 2024-06-06T07:49:33.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:33.285+0000 7f9806841700 1 -- 172.21.15.87:0/750508815 wait complete. 2024-06-06T07:49:34.104 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:49:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T07:49:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-06T07:49:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: scrub queued for path: / 2024-06-06T07:49:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: scrub summary: active paths [/] 2024-06-06T07:49:34.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:34 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.427 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T07:49:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-06T07:49:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: scrub queued for path: / 2024-06-06T07:49:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: scrub summary: active paths [/] 2024-06-06T07:49:34.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:34 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T07:49:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-06T07:49:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: scrub queued for path: / 2024-06-06T07:49:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: scrub summary: active paths [/] 2024-06-06T07:49:34.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:34 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:49:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 -- 172.21.15.87:0/3877476772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 msgr2=0x7fb97410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/3877476772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97410e410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009ca0 tx=0x7fb96c02a9e0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 -- 172.21.15.87:0/3877476772 shutdown_connections 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/3877476772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97410e410 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/3877476772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb97410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/3877476772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 -- 172.21.15.87:0/3877476772 >> 172.21.15.87:0/3877476772 conn(0x7fb974077a20 msgr2=0x7fb974077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 -- 172.21.15.87:0/3877476772 shutdown_connections 2024-06-06T07:49:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.751+0000 7fb97bf7e640 1 -- 172.21.15.87:0/3877476772 wait complete. 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 Processor -- start 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 -- start start 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb974198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb974111d70 con 0x7fb97410bf70 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb974111bf0 con 0x7fb974102c30 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97bf7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb974111ef0 con 0x7fb974103550 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97a51b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97a51b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51636/0 (socket says 172.21.15.87:51636) 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.752+0000 7fb97a51b640 1 -- 172.21.15.87:0/642438747 learned_addr learned my addr 172.21.15.87:0/642438747 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb979519640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb974198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb979d1a640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 -- 172.21.15.87:0/642438747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 msgr2=0x7fb974198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb974198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 -- 172.21.15.87:0/642438747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 msgr2=0x7fb974198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 -- 172.21.15.87:0/642438747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97419d200 con 0x7fb97410bf70 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb979519640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb974198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb979d1a640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97a51b640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006010 tx=0x7fb96c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb96affd640 1 -- 172.21.15.87:0/642438747 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c047070 con 0x7fb97410bf70 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97419d490 con 0x7fb97410bf70 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb96affd640 1 -- 172.21.15.87:0/642438747 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c02fb20 con 0x7fb97410bf70 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb96affd640 1 -- 172.21.15.87:0/642438747 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c033050 con 0x7fb97410bf70 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.753+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9741007b0 con 0x7fb97410bf70 2024-06-06T07:49:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.756+0000 7fb97ed6f740 1 Processor -- start 2024-06-06T07:49:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.757+0000 7fb96affd640 1 -- 172.21.15.87:0/642438747 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb96c04a070 con 0x7fb97410bf70 2024-06-06T07:49:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.757+0000 7fb97ed6f740 1 -- start start 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.757+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a89c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560e1a8004f0 con 0x560e1a8a7350 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e1a79e870 con 0x560e1a89c170 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e1a7fc4b0 con 0x560e1a89ca90 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51646/0 (socket says 172.21.15.87:51646) 2024-06-06T07:49:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 -- 172.21.15.87:0/1505837882 learned_addr learned my addr 172.21.15.87:0/1505837882 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a89c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 -- 172.21.15.87:0/1505837882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 msgr2=0x560e1a8a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 -- 172.21.15.87:0/1505837882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 msgr2=0x560e1a89c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a89c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 -- 172.21.15.87:0/1505837882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1a7ba9b0 con 0x560e1a8a7350 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.758+0000 7fb979519640 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb964005f60 tx=0x7fb964005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f567fd9226d4d010 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb95ffff640 1 -- 172.21.15.87:0/1505837882 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400fc50 con 0x560e1a8a7350 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb95ffff640 1 -- 172.21.15.87:0/1505837882 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96400fdf0 con 0x560e1a8a7350 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb95ffff640 1 -- 172.21.15.87:0/1505837882 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003a50 con 0x560e1a8a7350 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb96affd640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 0x7fb9600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb979d1a640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 0x7fb9600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.759+0000 7fb96affd640 1 -- 172.21.15.87:0/642438747 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb96c0f0330 con 0x7fb97410bf70 2024-06-06T07:49:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1505837882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 msgr2=0x560e1a8a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb964005f60 tx=0x7fb964005850 comp rx=0 tx=0).stop 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1505837882 shutdown_connections 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a9740 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1505837882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a89c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1505837882 >> 172.21.15.87:0/1505837882 conn(0x560e1a898680 msgr2=0x560e1a898a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb979d1a640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 0x7fb9600a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x560e1a89d710 tx=0x7fb970010040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1505837882 shutdown_connections 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1505837882 wait complete. 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 -- start start 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.760+0000 7fb97ed6f740 1 Processor -- start 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a8a4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560e1a8a9f80 con 0x560e1a8a7350 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e1a789ae0 con 0x560e1a89c170 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e1a8aa100 con 0x560e1a89ca90 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97ed6f740 10 client.0 _ll_register_callbacks cb 0x560e1a7c29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97a51b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979519640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a8a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979519640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54284/0 (socket says 172.21.15.87:54284) 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979519640 1 -- 172.21.15.87:0/1530853416 learned_addr learned my addr 172.21.15.87:0/1530853416 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 -- 172.21.15.87:0/1530853416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 msgr2=0x560e1a8a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 -- 172.21.15.87:0/1530853416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 msgr2=0x560e1a8a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb979d1a640 1 -- 172.21.15.87:0/1530853416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1a514210 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.761+0000 7fb97a51b640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a8a4290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb970007f50 tx=0x7fb970004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d102066077babd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97001a610 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96400fdf0 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb97001a7b0 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb970035050 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb97ed6f740 10 client.25265 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e1a5e0620 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.762+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96400fdf0 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb970025070 con 0x560e1a89c170 2024-06-06T07:49:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 10 client.25265 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 10 client.25265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 20 client.25265 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a8b7610 0x560e1a8b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 4 client.25265 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1a813f10 con 0x560e1a8b7610 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb979519640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a8b7610 0x560e1a8b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb979519640 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a8b7610 0x560e1a8b9a00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9366d6783b7e185 server_cookie=aa7390dae88aac56 in_seq=0 out_seq=0 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.763+0000 7fb95d7fa640 10 client.25265 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb95d7fa640 1 -- 172.21.15.87:0/1530853416 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e1a813f10 con 0x560e1a8b7610 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb95d7fa640 10 client.25265 handle_command_reply: tid=0 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 client.25265 shutdown 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 msgr2=0x560e1a8a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a8a4290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb970007f50 tx=0x7fb970004730 comp rx=0 tx=0).stop 2024-06-06T07:49:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 shutdown_connections 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a8b7610 0x560e1a8b9a00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a8a7350 0x560e1a8a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a89ca90 0x560e1a8a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1530853416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89c170 0x560e1a8a4290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb95d7fa640 0 client.25265 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.764+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 >> 172.21.15.87:0/1530853416 conn(0x560e1a898680 msgr2=0x560e1a89e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.765+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 shutdown_connections 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.765+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1530853416 wait complete. 2024-06-06T07:49:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.765+0000 7fb97ed6f740 20 client.25265 trim_cache size 0 max 16384 2024-06-06T07:49:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 Processor -- start 2024-06-06T07:49:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 -- start start 2024-06-06T07:49:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89ca90 0x560e1a897660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8a7350 0x560e1a8bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741af540 con 0x560e1a89c170 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ace10 con 0x560e1a89ca90 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741aac60 con 0x560e1a8a7350 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb979d1a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb979d1a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51670/0 (socket says 172.21.15.87:51670) 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb979d1a640 1 -- 172.21.15.87:0/1829700366 learned_addr learned my addr 172.21.15.87:0/1829700366 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb979519640 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89ca90 0x560e1a897660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.780+0000 7fb97a51b640 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8a7350 0x560e1a8bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 -- 172.21.15.87:0/1829700366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8a7350 msgr2=0x560e1a8bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8a7350 0x560e1a8bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 -- 172.21.15.87:0/1829700366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89ca90 msgr2=0x560e1a897660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89ca90 0x560e1a897660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 -- 172.21.15.87:0/1829700366 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1a8b9d70 con 0x560e1a89c170 2024-06-06T07:49:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb979d1a640 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb97002e5e0 tx=0x7fb97002e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cadce2b3ae8488 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb95f7fe640 1 -- 172.21.15.87:0/1829700366 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000c220 con 0x560e1a89c170 2024-06-06T07:49:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb95f7fe640 1 -- 172.21.15.87:0/1829700366 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970020070 con 0x560e1a89c170 2024-06-06T07:49:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.781+0000 7fb95f7fe640 1 -- 172.21.15.87:0/1829700366 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700267d0 con 0x560e1a89c170 2024-06-06T07:49:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1829700366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 msgr2=0x560e1a89eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb97002e5e0 tx=0x7fb97002e610 comp rx=0 tx=0).stop 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1829700366 shutdown_connections 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8a7350 0x560e1a8bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a89ca90 0x560e1a897660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/1829700366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89c170 0x560e1a89eb30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1829700366 >> 172.21.15.87:0/1829700366 conn(0x560e1a898680 msgr2=0x560e1a896f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1829700366 shutdown_connections 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- 172.21.15.87:0/1829700366 wait complete. 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 -- start start 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.782+0000 7fb97ed6f740 1 Processor -- start 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 0x560e1a8af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 0x560e1a8a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ab2d0 con 0x560e1a89ca90 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ac740 con 0x560e1a8a7350 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ab9a0 con 0x560e1a8b7610 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97ed6f740 10 client.0 _ll_register_callbacks cb 0x7fb970007420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97a51b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97a51b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54300/0 (socket says 172.21.15.87:54300) 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb97a51b640 1 -- 172.21.15.87:0/2706133654 learned_addr learned my addr 172.21.15.87:0/2706133654 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb979d1a640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 0x560e1a8af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb979519640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 0x560e1a8a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.783+0000 7fb979519640 1 -- 172.21.15.87:0/2706133654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 msgr2=0x560e1a8a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979519640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979519640 1 -- 172.21.15.87:0/2706133654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 msgr2=0x560e1a8af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979519640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 0x560e1a8af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979519640 1 -- 172.21.15.87:0/2706133654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1a8a17d0 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb97a51b640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979d1a640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 0x560e1a8af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb979519640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 0x560e1a8a0d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x560e1a8a5860 tx=0x7fb9640035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66aed7be26f2487e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400fc20 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb970020070 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb9640050c0 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb96400fdc0 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb97ed6f740 10 client.25271 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e1a92a840 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.784+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970020070 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb96402b440 con 0x560e1a8a7350 2024-06-06T07:49:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 10 client.25271 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 10 client.25271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 20 client.25271 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a92ace0 0x560e1a92d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 4 client.25271 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e1a80dc20 con 0x560e1a92ace0 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb979d1a640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a92ace0 0x560e1a92d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.785+0000 7fb95d7fa640 10 client.25271 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.786+0000 7fb979d1a640 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a92ace0 0x560e1a92d100 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee366e8eb6944e server_cookie=117fe5140131c762 in_seq=0 out_seq=0 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb95d7fa640 1 -- 172.21.15.87:0/2706133654 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560e1a80dc20 con 0x560e1a92ace0 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb95d7fa640 10 client.25271 handle_command_reply: tid=0 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 client.25271 shutdown 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 msgr2=0x560e1a8a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 0x560e1a8a0d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x560e1a8a5860 tx=0x7fb9640035f0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 shutdown_connections 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560e1a92ace0 0x560e1a92d100 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e1a8b7610 0x560e1a8a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb95d7fa640 0 client.25271 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e1a8a7350 0x560e1a8a0d20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 --2- 172.21.15.87:0/2706133654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560e1a89ca90 0x560e1a8af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.791+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 >> 172.21.15.87:0/2706133654 conn(0x560e1a898680 msgr2=0x560e1a8977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 shutdown_connections 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97ed6f740 1 -- 172.21.15.87:0/2706133654 wait complete. 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97ed6f740 20 client.25271 trim_cache size 0 max 16384 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:49:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 msgr2=0x7fb9600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 0x7fb9600a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x560e1a89d710 tx=0x7fb970010040 comp rx=0 tx=0).stop 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 msgr2=0x7fb97419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006010 tx=0x7fb96c003ba0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 shutdown_connections 2024-06-06T07:49:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb9600a2da0 0x7fb9600a5260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb97410bf70 0x7fb97419cad0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb974103550 0x7fb974198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 --2- 172.21.15.87:0/642438747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb974102c30 0x7fb974198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 >> 172.21.15.87:0/642438747 conn(0x7fb974077a20 msgr2=0x7fb9740fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 shutdown_connections 2024-06-06T07:49:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:34.792+0000 7fb97bf7e640 1 -- 172.21.15.87:0/642438747 wait complete. 2024-06-06T07:49:36.263 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:36 smithi087 ceph-mon[32276]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:49:36.263 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:36 smithi087 ceph-mon[32276]: scrub summary: idle 2024-06-06T07:49:36.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 -- 172.21.15.87:0/1799133164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320106b90 msgr2=0x7f4320106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 --2- 172.21.15.87:0/1799133164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320106b90 0x7f4320106f70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4314009990 tx=0x7f4314009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 -- 172.21.15.87:0/1799133164 shutdown_connections 2024-06-06T07:49:36.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 --2- 172.21.15.87:0/1799133164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 --2- 172.21.15.87:0/1799133164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320106b90 0x7f4320106f70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 --2- 172.21.15.87:0/1799133164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320100b50 0x7f4320100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 -- 172.21.15.87:0/1799133164 >> 172.21.15.87:0/1799133164 conn(0x7f4320077a00 msgr2=0x7f43200fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 -- 172.21.15.87:0/1799133164 shutdown_connections 2024-06-06T07:49:36.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.381+0000 7f432715a700 1 -- 172.21.15.87:0/1799133164 wait complete. 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 Processor -- start 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 -- start start 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 0x7f432019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320106b90 0x7f4320197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4320110e30 con 0x7f4320100b50 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4320110cb0 con 0x7f4320101510 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.382+0000 7f432715a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4320110fb0 con 0x7f4320106b90 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53028/0 (socket says 172.21.15.87:53028) 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 -- 172.21.15.87:0/2696255184 learned_addr learned my addr 172.21.15.87:0/2696255184 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f43256f7700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320106b90 0x7f4320197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 0x7f432019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 -- 172.21.15.87:0/2696255184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320106b90 msgr2=0x7f4320197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320106b90 0x7f4320197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 -- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 msgr2=0x7f432019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 0x7f432019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 -- 172.21.15.87:0/2696255184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320197fe0 con 0x7f4320101510 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 0x7f432019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:36.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.383+0000 7f431ffff700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f430c007b50 tx=0x7f430c007e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.384+0000 7f431dffb700 1 -- 172.21.15.87:0/2696255184 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00e500 con 0x7f4320101510 2024-06-06T07:49:36.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.384+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43201982d0 con 0x7f4320101510 2024-06-06T07:49:36.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.384+0000 7f431dffb700 1 -- 172.21.15.87:0/2696255184 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c00eb80 con 0x7f4320101510 2024-06-06T07:49:36.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.384+0000 7f431dffb700 1 -- 172.21.15.87:0/2696255184 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ab90 con 0x7f4320101510 2024-06-06T07:49:36.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.384+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4320103660 con 0x7f4320101510 2024-06-06T07:49:36.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.387+0000 7f431dffb700 1 -- 172.21.15.87:0/2696255184 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f430c00e6a0 con 0x7f4320101510 2024-06-06T07:49:36.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.388+0000 7f431dffb700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 0x7f43100a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.388+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 0x7f43100a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.388+0000 7f431dffb700 1 -- 172.21.15.87:0/2696255184 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f430c0d0c10 con 0x7f4320101510 2024-06-06T07:49:36.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.388+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 0x7f43100a52d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43140098e0 tx=0x7f431400b560 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.389+0000 7f43319ddb80 1 Processor -- start 2024-06-06T07:49:36.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.389+0000 7f43319ddb80 1 -- start start 2024-06-06T07:49:36.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.389+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0209260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 0x55ffa0209bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53036/0 (socket says 172.21.15.87:53036) 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 -- 172.21.15.87:0/3387189900 learned_addr learned my addr 172.21.15.87:0/3387189900 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43256f7700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 0x55ffa0209bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ffa00fbf60 con 0x55ffa02097a0 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ffa015a850 con 0x55ffa020ef60 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ffa00c2460 con 0x55ffa020f880 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f431ffff700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0209260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 -- 172.21.15.87:0/3387189900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 msgr2=0x55ffa0209260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0209260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 -- 172.21.15.87:0/3387189900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 msgr2=0x55ffa0209bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 0x55ffa0209bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f4324ef6700 1 -- 172.21.15.87:0/3387189900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa00a98d0 con 0x55ffa020ef60 2024-06-06T07:49:36.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.390+0000 7f43256f7700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 0x55ffa0209bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:36.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.391+0000 7f4324ef6700 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f431400ba60 tx=0x7f431401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1298514951d4c0d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.391+0000 7f430affd700 1 -- 172.21.15.87:0/3387189900 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400bea0 con 0x55ffa020ef60 2024-06-06T07:49:36.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.391+0000 7f430affd700 1 -- 172.21.15.87:0/3387189900 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f431400f940 con 0x55ffa020ef60 2024-06-06T07:49:36.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.391+0000 7f430affd700 1 -- 172.21.15.87:0/3387189900 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140248c0 con 0x55ffa020ef60 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 msgr2=0x55ffa020f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f431400ba60 tx=0x7f431401f000 comp rx=0 tx=0).stop 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 shutdown_connections 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020ef60 0x55ffa020f340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa02097a0 0x55ffa0209bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 --2- 172.21.15.87:0/3387189900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0209260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.392+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 >> 172.21.15.87:0/3387189900 conn(0x55ffa0136080 msgr2=0x55ffa0204220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.393+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 shutdown_connections 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.393+0000 7f43319ddb80 1 -- 172.21.15.87:0/3387189900 wait complete. 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.393+0000 7f43319ddb80 1 -- start start 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.393+0000 7f43319ddb80 1 Processor -- start 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.393+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa0215640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0212ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ffa00fafc0 con 0x55ffa020ef60 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0156810 con 0x55ffa02097a0 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ffa02161a0 con 0x55ffa020f880 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43319ddb80 10 client.0 _ll_register_callbacks cb 0x55ffa00f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51706/0 (socket says 172.21.15.87:51706) 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 -- 172.21.15.87:0/1509339758 learned_addr learned my addr 172.21.15.87:0/1509339758 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f43256f7700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0212ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f4324ef6700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa0215640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 -- 172.21.15.87:0/1509339758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 msgr2=0x55ffa0212ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0212ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 -- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 msgr2=0x55ffa0215640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa0215640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f431ffff700 1 -- 172.21.15.87:0/1509339758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa0120750 con 0x55ffa020ef60 2024-06-06T07:49:36.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.394+0000 7f4324ef6700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa0215640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:36.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f431ffff700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f430c09cd80 tx=0x7f430c00ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f900bf555c23e0c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f42fffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:36.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c0a11a0 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f430c09d050 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f431400f940 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f430c0a59d0 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f43319ddb80 10 client.15888 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.395+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffa0220780 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f431400f940 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f430c014070 con 0x55ffa020ef60 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 20 client.15888 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa0222d10 0x55ffa0225100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 4 client.15888 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.396+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa0167d50 con 0x55ffa0222d10 2024-06-06T07:49:36.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.397+0000 7f4324ef6700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa0222d10 0x55ffa0225100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.397+0000 7f4324ef6700 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa0222d10 0x55ffa0225100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0566f67d42a465 server_cookie=23f339a75acfc772 in_seq=0 out_seq=0 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.397+0000 7f42fffff700 10 client.15888 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f42fffff700 1 -- 172.21.15.87:0/1509339758 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ffa0167d50 con 0x55ffa0222d10 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f42fffff700 10 client.15888 handle_command_reply: tid=0 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 client.15888 shutdown 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 msgr2=0x55ffa0212990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f430c09cd80 tx=0x7f430c00ae30 comp rx=0 tx=0).stop 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 shutdown_connections 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa0222d10 0x55ffa0225100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa0215640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa0212ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1509339758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0212990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 >> 172.21.15.87:0/1509339758 conn(0x55ffa0136080 msgr2=0x55ffa0203700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f42fffff700 0 client.15888 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:36.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 shutdown_connections 2024-06-06T07:49:36.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 1 -- 172.21.15.87:0/1509339758 wait complete. 2024-06-06T07:49:36.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.398+0000 7f43319ddb80 20 client.15888 trim_cache size 0 max 16384 2024-06-06T07:49:36.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 Processor -- start 2024-06-06T07:49:36.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 -- start start 2024-06-06T07:49:36.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa02097a0 0x55ffa0209260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020f880 0x55ffa02043c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43201aed70 con 0x55ffa020ef60 2024-06-06T07:49:36.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f43201ac600 con 0x55ffa020f880 2024-06-06T07:49:36.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f43319ddb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f43201aa450 con 0x55ffa02097a0 2024-06-06T07:49:36.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.438+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51720/0 (socket says 172.21.15.87:51720) 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 -- 172.21.15.87:0/1217384801 learned_addr learned my addr 172.21.15.87:0/1217384801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f4324ef6700 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa02097a0 0x55ffa0209260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f43256f7700 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020f880 0x55ffa02043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 -- 172.21.15.87:0/1217384801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa02097a0 msgr2=0x55ffa0209260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa02097a0 0x55ffa0209260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 -- 172.21.15.87:0/1217384801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020f880 msgr2=0x55ffa02043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020f880 0x55ffa02043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 -- 172.21.15.87:0/1217384801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa0210f00 con 0x55ffa020ef60 2024-06-06T07:49:36.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f431ffff700 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f430c0ce040 tx=0x7f430c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12307c9711c59cf5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f430a7fc700 1 -- 172.21.15.87:0/1217384801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c09c510 con 0x55ffa020ef60 2024-06-06T07:49:36.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f430a7fc700 1 -- 172.21.15.87:0/1217384801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c0a11a0 con 0x55ffa020ef60 2024-06-06T07:49:36.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.439+0000 7f430a7fc700 1 -- 172.21.15.87:0/1217384801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c098be0 con 0x55ffa020ef60 2024-06-06T07:49:36.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 -- 172.21.15.87:0/1217384801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 msgr2=0x55ffa0203e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f430c0ce040 tx=0x7f430c00a5b0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 -- 172.21.15.87:0/1217384801 shutdown_connections 2024-06-06T07:49:36.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa020f880 0x55ffa02043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa02097a0 0x55ffa0209260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 --2- 172.21.15.87:0/1217384801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa0203e80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.440+0000 7f43319ddb80 1 -- 172.21.15.87:0/1217384801 >> 172.21.15.87:0/1217384801 conn(0x55ffa0136080 msgr2=0x55ffa0211280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.441+0000 7f43319ddb80 1 -- 172.21.15.87:0/1217384801 shutdown_connections 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.441+0000 7f43319ddb80 1 -- 172.21.15.87:0/1217384801 wait complete. 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.441+0000 7f43319ddb80 1 -- start start 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.441+0000 7f43319ddb80 1 Processor -- start 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa022a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51726/0 (socket says 172.21.15.87:51726) 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 -- 172.21.15.87:0/2660492137 learned_addr learned my addr 172.21.15.87:0/2660492137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:36.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa022af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43201aaac0 con 0x55ffa020ef60 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f43201abf30 con 0x55ffa02097a0 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f43201ab190 con 0x55ffa020f880 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f43319ddb80 10 client.0 _ll_register_callbacks cb 0x55ffa00f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa022a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 -- 172.21.15.87:0/2660492137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 msgr2=0x55ffa022af80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa022af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 -- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 msgr2=0x55ffa022a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa022a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.442+0000 7f431ffff700 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa028c930 con 0x55ffa020ef60 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f431ffff700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f430c0a1360 tx=0x7f430c09d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89163a1e002c7ed5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f42fffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:36.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c099090 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f430c0a33b0 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffa01622d0 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f430c0860d0 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f43319ddb80 10 client.15894 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.443+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffa028c930 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ffa01622d0 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f430c09bcb0 con 0x55ffa020ef60 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 20 client.15894 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa028de40 0x55ffa0290230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 4 client.15894 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffa0115f50 con 0x55ffa028de40 2024-06-06T07:49:36.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.444+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa028de40 0x55ffa0290230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f4324ef6700 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa028de40 0x55ffa0290230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d6eb515e8c1ac2 server_cookie=f62ac0e70915c66a in_seq=0 out_seq=0 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f42fffff700 10 client.15894 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f42fffff700 1 -- 172.21.15.87:0/2660492137 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ffa0115f50 con 0x55ffa028de40 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f42fffff700 10 client.15894 handle_command_reply: tid=0 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 client.15894 shutdown 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 msgr2=0x55ffa022aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f430c0a1360 tx=0x7f430c09d3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 shutdown_connections 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ffa028de40 0x55ffa0290230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f42fffff700 0 client.15894 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ffa02097a0 0x55ffa022a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ffa020f880 0x55ffa022af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 --2- 172.21.15.87:0/2660492137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ffa020ef60 0x55ffa022aa40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.445+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 >> 172.21.15.87:0/2660492137 conn(0x55ffa0136080 msgr2=0x55ffa020fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 shutdown_connections 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f43319ddb80 1 -- 172.21.15.87:0/2660492137 wait complete. 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f43319ddb80 20 client.15894 trim_cache size 0 max 16384 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-06-06T07:49:36.455 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 msgr2=0x7f43100a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 0x7f43100a52d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43140098e0 tx=0x7f431400b560 comp rx=0 tx=0).stop 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.446+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 msgr2=0x7f432019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f430c007b50 tx=0x7f430c007e60 comp rx=0 tx=0).stop 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 shutdown_connections 2024-06-06T07:49:36.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f43100a2e10 0x7f43100a52d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4320106b90 0x7f4320197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4320100b50 0x7f432019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 --2- 172.21.15.87:0/2696255184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4320101510 0x7f432019dc00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 >> 172.21.15.87:0/2696255184 conn(0x7f4320077a00 msgr2=0x7f43200fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 shutdown_connections 2024-06-06T07:49:36.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:36.447+0000 7f432715a700 1 -- 172.21.15.87:0/2696255184 wait complete. 2024-06-06T07:49:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:36 smithi195 ceph-mon[37040]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:49:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:36 smithi195 ceph-mon[37040]: scrub summary: idle 2024-06-06T07:49:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:36 smithi181 ceph-mon[36790]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-06T07:49:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:36 smithi181 ceph-mon[36790]: scrub summary: idle 2024-06-06T07:49:37.247 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:49:37.247 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:49:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:38 smithi087 ceph-mon[32276]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T07:49:38.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:38 smithi195 ceph-mon[37040]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T07:49:38.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:38 smithi181 ceph-mon[36790]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T07:49:39.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 -- 172.21.15.87:0/3671563336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 msgr2=0x7fa26c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 --2- 172.21.15.87:0/3671563336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c106f70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa260009990 tx=0x7fa260009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 -- 172.21.15.87:0/3671563336 shutdown_connections 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 --2- 172.21.15.87:0/3671563336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 --2- 172.21.15.87:0/3671563336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c106f70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 --2- 172.21.15.87:0/3671563336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 0x7fa26c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 -- 172.21.15.87:0/3671563336 >> 172.21.15.87:0/3671563336 conn(0x7fa26c077a00 msgr2=0x7fa26c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.488+0000 7fa272a98700 1 -- 172.21.15.87:0/3671563336 shutdown_connections 2024-06-06T07:49:39.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.489+0000 7fa272a98700 1 -- 172.21.15.87:0/3671563336 wait complete. 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.489+0000 7fa272a98700 1 Processor -- start 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.489+0000 7fa272a98700 1 -- start start 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.489+0000 7fa272a98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 0x7fa26c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa272a98700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa272a98700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa271035700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa270834700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 0x7fa26c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa271035700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48876/0 (socket says 172.21.15.87:48876) 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa271035700 1 -- 172.21.15.87:0/410029945 learned_addr learned my addr 172.21.15.87:0/410029945 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c1116b0 con 0x7fa26c106b90 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c111530 con 0x7fa26c100b50 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c111830 con 0x7fa26c101510 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa26bfff700 1 -- 172.21.15.87:0/410029945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 msgr2=0x7fa26c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa26bfff700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 0x7fa26c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa26bfff700 1 -- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 msgr2=0x7fa26c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.490+0000 7fa26bfff700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa26bfff700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c19fdc0 con 0x7fa26c101510 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa271035700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa26bfff700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c19b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d950 tx=0x7fa25800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258016070 con 0x7fa26c101510 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c1a00b0 con 0x7fa26c101510 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa25800fd70 con 0x7fa26c101510 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800e3b0 con 0x7fa26c101510 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.491+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c1a7930 con 0x7fa26c101510 2024-06-06T07:49:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.493+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c069180 con 0x7fa26c101510 2024-06-06T07:49:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.494+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa258019070 con 0x7fa26c101510 2024-06-06T07:49:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.494+0000 7fa269ffb700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 0x7fa25c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.494+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fa2580cf1f0 con 0x7fa26c101510 2024-06-06T07:49:39.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.494+0000 7fa270834700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 0x7fa25c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:39.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.495+0000 7fa270834700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 0x7fa25c0a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b5c0 tx=0x7fa260005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:39.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.499+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa258010460 con 0x7fa26c101510 2024-06-06T07:49:39.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.763+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa26c1a7bc0 con 0x7fa26c101510 2024-06-06T07:49:39.764 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:49:39.765 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:49:39.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.764+0000 7fa269ffb700 1 -- 172.21.15.87:0/410029945 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7fa25809ba40 con 0x7fa26c101510 2024-06-06T07:49:39.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 msgr2=0x7fa25c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:39.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 0x7fa25c0a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b5c0 tx=0x7fa260005960 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 msgr2=0x7fa26c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c19b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d950 tx=0x7fa25800dc60 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 shutdown_connections 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c100b50 0x7fa26c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa25c0a2d90 0x7fa25c0a5250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa26c106b90 0x7fa26c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 --2- 172.21.15.87:0/410029945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa26c101510 0x7fa26c19b2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:39.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 >> 172.21.15.87:0/410029945 conn(0x7fa26c077a00 msgr2=0x7fa26c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:39.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.767+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 shutdown_connections 2024-06-06T07:49:39.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:39.768+0000 7fa272a98700 1 -- 172.21.15.87:0/410029945 wait complete. 2024-06-06T07:49:39.769 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:49:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:40 smithi087 ceph-mon[32276]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:40.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:40 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/410029945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:40.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:49:40.423 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:49:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:40 smithi195 ceph-mon[37040]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:40 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/410029945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:40 smithi181 ceph-mon[36790]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:40 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/410029945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:40.696 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:40.713 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:42.382 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:42 smithi087 ceph-mon[32276]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:42 smithi195 ceph-mon[37040]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:42 smithi181 ceph-mon[36790]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T07:49:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:44 smithi195 ceph-mon[37040]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-06T07:49:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:44 smithi181 ceph-mon[36790]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-06T07:49:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:44 smithi087 ceph-mon[32276]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-06T07:49:45.704 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:49:45.704 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:47:58.307983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 -- 172.21.15.87:0/3912174137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94102550 msgr2=0x7f9e9410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 --2- 172.21.15.87:0/3912174137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94102550 0x7f9e9410e210 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00b0a0 tx=0x7f9e8c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 -- 172.21.15.87:0/3912174137 shutdown_connections 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 --2- 172.21.15.87:0/3912174137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94102550 0x7f9e9410e210 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 --2- 172.21.15.87:0/3912174137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94106370 0x7f9e94106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 --2- 172.21.15.87:0/3912174137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94101b90 0x7f9e94102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 -- 172.21.15.87:0/3912174137 >> 172.21.15.87:0/3912174137 conn(0x7f9e94077a00 msgr2=0x7f9e940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 -- 172.21.15.87:0/3912174137 shutdown_connections 2024-06-06T07:49:45.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.857+0000 7f9e9a533700 1 -- 172.21.15.87:0/3912174137 wait complete. 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 Processor -- start 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 -- start start 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94101b90 0x7f9e9419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94102550 0x7f9e9419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94111690 con 0x7f9e94106370 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94111510 con 0x7f9e94102550 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.858+0000 7f9e9a533700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94111810 con 0x7f9e94101b90 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48896/0 (socket says 172.21.15.87:48896) 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 -- 172.21.15.87:0/3634587192 learned_addr learned my addr 172.21.15.87:0/3634587192 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e93fff700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94101b90 0x7f9e9419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:45.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 -- 172.21.15.87:0/3634587192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94101b90 msgr2=0x7f9e9419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94101b90 0x7f9e9419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 -- 172.21.15.87:0/3634587192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94102550 msgr2=0x7f9e9419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94102550 0x7f9e9419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 -- 172.21.15.87:0/3634587192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94199120 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.859+0000 7f9e98ad0700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00b780 tx=0x7f9e8c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.860+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c021070 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.860+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c007b80 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.860+0000 7f9e9a533700 1 -- 172.21.15.87:0/3634587192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e941993b0 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.860+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c005280 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.860+0000 7f9e9a533700 1 -- 172.21.15.87:0/3634587192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941081e0 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.861+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74005330 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.864+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e8c024070 con 0x7f9e94106370 2024-06-06T07:49:45.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.865+0000 7f9e917fa700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 0x7f9e7c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.865+0000 7f9e93fff700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 0x7f9e7c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.866+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9e8c00f020 con 0x7f9e94106370 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.866+0000 7f9e93fff700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 0x7f9e7c0a5450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e94199b00 tx=0x7f9e84009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:44.867+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e8c02b020 con 0x7f9e94106370 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.259+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e74005620 con 0x7f9e94106370 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.260+0000 7f9e917fa700 1 -- 172.21.15.87:0/3634587192 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11873 (secure 0 0 0) 0x7f9e8c09d310 con 0x7f9e94106370 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.262+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 msgr2=0x7f9e7c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.262+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 0x7f9e7c0a5450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e94199b00 tx=0x7f9e84009260 comp rx=0 tx=0).stop 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.262+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 msgr2=0x7f9e941989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.262+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00b780 tx=0x7f9e8c004d30 comp rx=0 tx=0).stop 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 shutdown_connections 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9e7c0a2f90 0x7f9e7c0a5450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e94106370 0x7f9e941989c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e94101b90 0x7f9e9419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 --2- 172.21.15.87:0/3634587192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e94102550 0x7f9e9419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:45.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 >> 172.21.15.87:0/3634587192 conn(0x7f9e94077a00 msgr2=0x7f9e94102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:45.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.263+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 shutdown_connections 2024-06-06T07:49:45.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:45.264+0000 7f9e7affd700 1 -- 172.21.15.87:0/3634587192 wait complete. 2024-06-06T07:49:45.709 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 27 2024-06-06T07:49:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:46 smithi195 ceph-mon[37040]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:49:46.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:46 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3634587192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:46 smithi181 ceph-mon[36790]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:49:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:46 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3634587192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:46 smithi087 ceph-mon[32276]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T07:49:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:46 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3634587192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 -- 172.21.15.87:0/3084113692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c101290 msgr2=0x7f006c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 --2- 172.21.15.87:0/3084113692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c101290 0x7f006c10d750 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0064009ca0 tx=0x7f006402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 -- 172.21.15.87:0/3084113692 shutdown_connections 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 --2- 172.21.15.87:0/3084113692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c101290 0x7f006c10d750 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 --2- 172.21.15.87:0/3084113692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c1008d0 0x7f006c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 --2- 172.21.15.87:0/3084113692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 -- 172.21.15.87:0/3084113692 >> 172.21.15.87:0/3084113692 conn(0x7f006c077a20 msgr2=0x7f006c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 -- 172.21.15.87:0/3084113692 shutdown_connections 2024-06-06T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.929+0000 7f007199e640 1 -- 172.21.15.87:0/3084113692 wait complete. 2024-06-06T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.930+0000 7f007199e640 1 Processor -- start 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.930+0000 7f007199e640 1 -- start start 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.930+0000 7f007199e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.930+0000 7f007199e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c101290 0x7f006c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.930+0000 7f007199e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f007199e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f006c111da0 con 0x7f006c1008d0 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f007199e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f006c111c20 con 0x7f006c106890 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60616/0 (socket says 172.21.15.87:60616) 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 -- 172.21.15.87:0/1375541053 learned_addr learned my addr 172.21.15.87:0/1375541053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f006c111f20 con 0x7f006c101290 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006b7fe640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f0063fff640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c101290 0x7f006c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 -- 172.21.15.87:0/1375541053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c101290 msgr2=0x7f006c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c101290 0x7f006c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 -- 172.21.15.87:0/1375541053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 msgr2=0x7f006c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 -- 172.21.15.87:0/1375541053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19d1d0 con 0x7f006c1008d0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006b7fe640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f006affd640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d950 tx=0x7f005c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.931+0000 7f007099c640 1 -- 172.21.15.87:0/1375541053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c017070 con 0x7f006c1008d0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.932+0000 7f007099c640 1 -- 172.21.15.87:0/1375541053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c009c30 con 0x7f006c1008d0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.932+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19d4c0 con 0x7f006c1008d0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.933+0000 7f007099c640 1 -- 172.21.15.87:0/1375541053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c00f6b0 con 0x7f006c1008d0 2024-06-06T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.933+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a4d50 con 0x7f006c1008d0 2024-06-06T07:49:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.935+0000 7f007099c640 1 -- 172.21.15.87:0/1375541053 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f005c00f850 con 0x7f006c1008d0 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.936+0000 7f007099c640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 0x7f00580a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.936+0000 7f0074784740 1 Processor -- start 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 -- start start 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0063fff640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 0x7f00580a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a81e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562a8a7824f0 con 0x562a8a82d760 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562a8a720870 con 0x562a8a81e050 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0074784740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a8a77e4b0 con 0x562a8a81e970 2024-06-06T07:49:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f007099c640 1 -- 172.21.15.87:0/1375541053 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f005c0cefd0 con 0x7f006c1008d0 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60630/0 (socket says 172.21.15.87:60630) 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 -- 172.21.15.87:0/878128353 learned_addr learned my addr 172.21.15.87:0/878128353 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f0063fff640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 0x7f00580a51d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006c077ef0 tx=0x7f005400b390 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006affd640 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a81e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 -- 172.21.15.87:0/878128353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 msgr2=0x562a8a82d220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 -- 172.21.15.87:0/878128353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 msgr2=0x562a8a81e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a81e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.937+0000 7f006b7fe640 1 -- 172.21.15.87:0/878128353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8a73c9b0 con 0x562a8a82d760 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.938+0000 7f006b7fe640 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0064009ca0 tx=0x7f0064031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652543a14e629f88 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.938+0000 7f0061ffb640 1 -- 172.21.15.87:0/878128353 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064031d10 con 0x562a8a82d760 2024-06-06T07:49:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.938+0000 7f0061ffb640 1 -- 172.21.15.87:0/878128353 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0064036070 con 0x562a8a82d760 2024-06-06T07:49:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.938+0000 7f0061ffb640 1 -- 172.21.15.87:0/878128353 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640305e0 con 0x562a8a82d760 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 -- 172.21.15.87:0/878128353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 msgr2=0x562a8a82fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0064009ca0 tx=0x7f0064031880 comp rx=0 tx=0).stop 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 -- 172.21.15.87:0/878128353 shutdown_connections 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82fb50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 --2- 172.21.15.87:0/878128353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a81e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 -- 172.21.15.87:0/878128353 >> 172.21.15.87:0/878128353 conn(0x562a8a81a680 msgr2=0x562a8a81aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 -- 172.21.15.87:0/878128353 shutdown_connections 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.939+0000 7f0074784740 1 -- 172.21.15.87:0/878128353 wait complete. 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.940+0000 7f0074784740 1 -- start start 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.940+0000 7f0074784740 1 Processor -- start 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f006affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f006affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:37176/0 (socket says 172.21.15.87:37176) 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0063fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f006b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0063fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:45208/0 (socket says 172.21.15.87:45208) 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f006b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60632/0 (socket says 172.21.15.87:60632) 2024-06-06T07:49:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f006affd640 1 -- 172.21.15.87:0/2736516274 learned_addr learned my addr 172.21.15.87:0/2736516274 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562a8a830330 con 0x562a8a82d760 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562a8a70bae0 con 0x562a8a81e050 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.941+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a8a8304b0 con 0x562a8a81e970 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 -- 172.21.15.87:0/2736516274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 msgr2=0x562a8a82abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 -- 172.21.15.87:0/2736516274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 msgr2=0x562a8a82b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8a496210 con 0x562a8a81e050 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006b7fe640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f0074784740 10 client.0 _ll_register_callbacks cb 0x562a8a7449b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f006affd640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721a3473e46c42d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f004affd640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0064036070 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f004affd640 1 -- 172.21.15.87:0/2736516274 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c020040 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f004affd640 1 -- 172.21.15.87:0/2736516274 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f005c0cc570 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f004affd640 1 -- 172.21.15.87:0/2736516274 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f005c09af40 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f0074784740 10 client.25304 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.942+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=27}) -- 0x562a8a562620 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f004affd640 1 -- 172.21.15.87:0/2736516274 <== mon.2 v2:172.21.15.181:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f005c095ce0 con 0x562a8a81e050 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 10 client.25304 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 10 client.25304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 20 client.25304 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a83d7c0 0x562a8a83fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 4 client.25304 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a8a795f10 con 0x562a8a83d7c0 2024-06-06T07:49:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.943+0000 7f0063fff640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a83d7c0 0x562a8a83fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.944+0000 7f004affd640 10 client.25304 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.944+0000 7f0063fff640 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a83d7c0 0x562a8a83fc50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d59c1c06a43fe7 server_cookie=b6ed3cb046a4a559 in_seq=0 out_seq=0 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f004affd640 1 -- 172.21.15.87:0/2736516274 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a8a795f10 con 0x562a8a83d7c0 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f004affd640 10 client.25304 handle_command_reply: tid=0 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 client.25304 shutdown 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 msgr2=0x562a8a82a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c009f50 comp rx=0 tx=0).stop 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 shutdown_connections 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a83d7c0 0x562a8a83fc50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a82b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e970 0x562a8a82abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f004affd640 0 client.25304 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 --2- 172.21.15.87:0/2736516274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e050 0x562a8a82a690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 >> 172.21.15.87:0/2736516274 conn(0x562a8a81a680 msgr2=0x562a8a820490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 shutdown_connections 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 1 -- 172.21.15.87:0/2736516274 wait complete. 2024-06-06T07:49:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.945+0000 7f0074784740 20 client.25304 trim_cache size 0 max 16384 2024-06-06T07:49:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 Processor -- start 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 -- start start 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a819660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a82d760 0x562a8a8455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1af130 con 0x562a8a81e970 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1aca00 con 0x562a8a82d760 2024-06-06T07:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0074784740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1aa850 con 0x562a8a81e050 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.975+0000 7f0063fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60636/0 (socket says 172.21.15.87:60636) 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 -- 172.21.15.87:0/3904204027 learned_addr learned my addr 172.21.15.87:0/3904204027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f006affd640 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a819660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f006b7fe640 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a82d760 0x562a8a8455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 -- 172.21.15.87:0/3904204027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 msgr2=0x562a8a819660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a819660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 -- 172.21.15.87:0/3904204027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a82d760 msgr2=0x562a8a8455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a82d760 0x562a8a8455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 -- 172.21.15.87:0/3904204027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8a81e7a0 con 0x562a8a81e970 2024-06-06T07:49:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f0063fff640 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x562a8a829ef0 tx=0x7f0054013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745e6c99f4f84ec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f00617fa640 1 -- 172.21.15.87:0/3904204027 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054005220 con 0x562a8a81e970 2024-06-06T07:49:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f00617fa640 1 -- 172.21.15.87:0/3904204027 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0054004c80 con 0x562a8a81e970 2024-06-06T07:49:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.976+0000 7f00617fa640 1 -- 172.21.15.87:0/3904204027 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540279d0 con 0x562a8a81e970 2024-06-06T07:49:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 -- 172.21.15.87:0/3904204027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 msgr2=0x562a8a82af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x562a8a829ef0 tx=0x7f0054013000 comp rx=0 tx=0).stop 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 -- 172.21.15.87:0/3904204027 shutdown_connections 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a82d760 0x562a8a8455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a81e970 0x562a8a82af30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 --2- 172.21.15.87:0/3904204027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a819660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.977+0000 7f0074784740 1 -- 172.21.15.87:0/3904204027 >> 172.21.15.87:0/3904204027 conn(0x562a8a81a680 msgr2=0x562a8a818f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.978+0000 7f0074784740 1 -- 172.21.15.87:0/3904204027 shutdown_connections 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.978+0000 7f0074784740 1 -- 172.21.15.87:0/3904204027 wait complete. 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.978+0000 7f0074784740 1 -- start start 2024-06-06T07:49:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.978+0000 7f0074784740 1 Processor -- start 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e970 0x562a8a8312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a8a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1aaec0 con 0x562a8a82d760 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1ac330 con 0x562a8a81e970 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1ab590 con 0x562a8a81e050 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0074784740 10 client.0 _ll_register_callbacks cb 0x7f005c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:45226/0 (socket says 172.21.15.87:45226) 2024-06-06T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 -- 172.21.15.87:0/483764856 learned_addr learned my addr 172.21.15.87:0/483764856 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006b7fe640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a8a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f0063fff640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e970 0x562a8a8312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 -- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e970 msgr2=0x562a8a8312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e970 0x562a8a8312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 -- 172.21.15.87:0/483764856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 msgr2=0x562a8a8a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a8a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006affd640 1 -- 172.21.15.87:0/483764856 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8a8a6ab0 con 0x562a8a81e050 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.979+0000 7f006b7fe640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a8a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f006affd640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f005c099420 tx=0x7f005c0bb050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57b5f9dd79ae924 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f004affd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0a7070 con 0x562a8a81e050 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0054004c80 con 0x562a8a81e050 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f005c09f120 con 0x562a8a81e050 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f005c020040 con 0x562a8a81e050 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f0074784740 10 client.25372 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a8a8a7d70 con 0x562a8a81e050 2024-06-06T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.980+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c07e020 con 0x562a8a81e050 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f005c095ce0 con 0x562a8a81e050 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 10 client.25372 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 20 client.25372 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a8a8480 0x562a8a8aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 4 client.25372 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a8a78fc20 con 0x562a8a8a8480 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.981+0000 7f0063fff640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a8a8480 0x562a8a8aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.982+0000 7f004affd640 10 client.25372 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.982+0000 7f0063fff640 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a8a8480 0x562a8a8aa8a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c1b91e01bc3064 server_cookie=5321a6b7c00bb50f in_seq=0 out_seq=0 2024-06-06T07:49:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.985+0000 7f004affd640 1 -- 172.21.15.87:0/483764856 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a8a78fc20 con 0x562a8a8a8480 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.985+0000 7f004affd640 10 client.25372 handle_command_reply: tid=0 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.985+0000 7f0074784740 1 client.25372 shutdown 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.985+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 msgr2=0x562a8a830d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.985+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f005c099420 tx=0x7f005c0bb050 comp rx=0 tx=0).stop 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 shutdown_connections 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562a8a8a8480 0x562a8a8aa8a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562a8a82d760 0x562a8a8a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f004affd640 0 client.25372 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a8a81e970 0x562a8a8312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 --2- 172.21.15.87:0/483764856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a8a81e050 0x562a8a830d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 >> 172.21.15.87:0/483764856 conn(0x562a8a81a680 msgr2=0x562a8a8197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 shutdown_connections 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 1 -- 172.21.15.87:0/483764856 wait complete. 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.986+0000 7f0074784740 20 client.25372 trim_cache size 0 max 16384 2024-06-06T07:49:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:49:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:49:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 msgr2=0x7f00580a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 0x7f00580a51d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006c077ef0 tx=0x7f005400b390 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 msgr2=0x7f006c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d950 tx=0x7f005c00de10 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 shutdown_connections 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00580a2d10 0x7f00580a51d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f006c106890 0x7f006c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f006c101290 0x7f006c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 --2- 172.21.15.87:0/1375541053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006c1008d0 0x7f006c1981e0 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d950 tx=0x7f005c00de10 comp rx=0 tx=0).stop 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 >> 172.21.15.87:0/1375541053 conn(0x7f006c077a20 msgr2=0x7f006c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 shutdown_connections 2024-06-06T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:49:47.987+0000 7f007199e640 1 -- 172.21.15.87:0/1375541053 wait complete. 2024-06-06T07:49:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:48 smithi087 ceph-mon[32276]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:49:48.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:48 smithi195 ceph-mon[37040]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:49:48.531 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 544e979e --release --await' 2024-06-06T07:49:48.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 544e979e --release --await 2024-06-06T07:49:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:48 smithi181 ceph-mon[36790]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:49:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 -- 172.21.15.87:0/2224126422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8101390 msgr2=0x7f36f8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/2224126422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8101390 0x7f36f8101810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009990 tx=0x7f36ec009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 -- 172.21.15.87:0/2224126422 shutdown_connections 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/2224126422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/2224126422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8107400 0x7f36f81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/2224126422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8101390 0x7f36f8101810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.121+0000 7f36fcbc4700 1 -- 172.21.15.87:0/2224126422 >> 172.21.15.87:0/2224126422 conn(0x7f36f8078340 msgr2=0x7f36f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.123+0000 7f36fcbc4700 1 -- 172.21.15.87:0/2224126422 shutdown_connections 2024-06-06T07:49:49.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.123+0000 7f36fcbc4700 1 -- 172.21.15.87:0/2224126422 wait complete. 2024-06-06T07:49:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 Processor -- start 2024-06-06T07:49:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 -- start start 2024-06-06T07:49:49.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 0x7f36f810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8107400 0x7f36f81a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8110050 con 0x7f36f8101390 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36f810fed0 con 0x7f36f8107400 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36fcbc4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36f81101d0 con 0x7f36f8101d50 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 0x7f36f810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 0x7f36f810cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60648/0 (socket says 172.21.15.87:60648) 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.124+0000 7f36f659c700 1 -- 172.21.15.87:0/3024857818 learned_addr learned my addr 172.21.15.87:0/3024857818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8107400 0x7f36f81a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3024857818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8107400 msgr2=0x7f36f81a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8107400 0x7f36f81a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 msgr2=0x7f36f810cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 0x7f36f810cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3024857818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f81a54d0 con 0x7f36f8101d50 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810d2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0098e0 tx=0x7f36ec004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.125+0000 7f36e77fe700 1 -- 172.21.15.87:0/3024857818 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec01a030 con 0x7f36f8101d50 2024-06-06T07:49:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.126+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f81a5760 con 0x7f36f8101d50 2024-06-06T07:49:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.126+0000 7f36e77fe700 1 -- 172.21.15.87:0/3024857818 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec01c070 con 0x7f36f8101d50 2024-06-06T07:49:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.126+0000 7f36e77fe700 1 -- 172.21.15.87:0/3024857818 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec015ba0 con 0x7f36f8101d50 2024-06-06T07:49:49.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.126+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f81a5bd0 con 0x7f36f8101d50 2024-06-06T07:49:49.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.129+0000 7f36e77fe700 1 -- 172.21.15.87:0/3024857818 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f36ec027060 con 0x7f36f8101d50 2024-06-06T07:49:49.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.130+0000 7f36e77fe700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 0x7f36e00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.130+0000 7f3707447b80 1 Processor -- start 2024-06-06T07:49:49.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.130+0000 7f36e77fe700 1 -- 172.21.15.87:0/3024857818 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f36ec0cf6d0 con 0x7f36f8101d50 2024-06-06T07:49:49.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.131+0000 7f3707447b80 1 -- start start 2024-06-06T07:49:49.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.131+0000 7f36f659c700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 0x7f36e00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.131+0000 7f3707447b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.131+0000 7f3707447b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.131+0000 7f3707447b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60660/0 (socket says 172.21.15.87:60660) 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 -- 172.21.15.87:0/4082492952 learned_addr learned my addr 172.21.15.87:0/4082492952 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f659c700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 0x7f36e00a5350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009c90 tx=0x7f36e8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5648890ccf60 con 0x5648891e0880 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56488912b850 con 0x5648891da7a0 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564889093460 con 0x5648891dff60 2024-06-06T07:49:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f659c700 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 -- 172.21.15.87:0/4082492952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 msgr2=0x5648891e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 -- 172.21.15.87:0/4082492952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 msgr2=0x5648891dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 -- 172.21.15.87:0/4082492952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488907a8d0 con 0x5648891e0880 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.132+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36ec098220 tx=0x7f36ec003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7c1aa8a98699dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.133+0000 7f36e4ff9700 1 -- 172.21.15.87:0/4082492952 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec09bf50 con 0x5648891e0880 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.133+0000 7f36e4ff9700 1 -- 172.21.15.87:0/4082492952 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec09b850 con 0x5648891e0880 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.133+0000 7f36e4ff9700 1 -- 172.21.15.87:0/4082492952 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0094b0 con 0x5648891e0880 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 msgr2=0x5648891da260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36ec098220 tx=0x7f36ec003680 comp rx=0 tx=0).stop 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 shutdown_connections 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 --2- 172.21.15.87:0/4082492952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891da260 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 >> 172.21.15.87:0/4082492952 conn(0x564889107080 msgr2=0x5648891d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 shutdown_connections 2024-06-06T07:49:49.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- 172.21.15.87:0/4082492952 wait complete. 2024-06-06T07:49:49.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.134+0000 7f3707447b80 1 -- start start 2024-06-06T07:49:49.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 Processor -- start 2024-06-06T07:49:49.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60672/0 (socket says 172.21.15.87:60672) 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f5d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:37230/0 (socket says 172.21.15.87:37230) 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f36f6d9d700 1 -- 172.21.15.87:0/141036127 learned_addr learned my addr 172.21.15.87:0/141036127 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5648890cbfc0 con 0x5648891e0880 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564889127810 con 0x5648891da7a0 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.135+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5648890ccf60 con 0x5648891dff60 2024-06-06T07:49:49.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 -- 172.21.15.87:0/141036127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 msgr2=0x5648891e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 -- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 msgr2=0x5648891e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648890f1750 con 0x5648891e0880 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f659c700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009790 tx=0x7f36f0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56288afba700810 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f3707447b80 10 client.0 _ll_register_callbacks cb 0x5648890c15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec09b850 con 0x5648891e0880 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36ddffb700 10 client.15936 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36ddffb700 1 -- 172.21.15.87:0/141036127 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f001a030 con 0x5648891e0880 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.136+0000 7f36ddffb700 1 -- 172.21.15.87:0/141036127 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f36f00138c0 con 0x5648891e0880 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f36ddffb700 1 -- 172.21.15.87:0/141036127 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f0013b20 con 0x5648891e0880 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 10 client.15936 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:49.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({fsmap=27}) -- 0x564889202270 con 0x5648891e0880 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f36ddffb700 1 -- 172.21.15.87:0/141036127 <== mon.0 v2:172.21.15.87:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f36f0013cc0 con 0x5648891e0880 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 20 client.15936 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889206b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 4 client.15936 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564889138d50 con 0x5648892047a0 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.137+0000 7f36f659c700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889206b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.138+0000 7f36f659c700 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889206b90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca37e2c68d3680cb server_cookie=5ee66a0156dad0e0 in_seq=0 out_seq=0 2024-06-06T07:49:49.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.138+0000 7f36ddffb700 10 client.15936 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:49.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.139+0000 7f36ddffb700 1 -- 172.21.15.87:0/141036127 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564889138d50 con 0x5648892047a0 2024-06-06T07:49:49.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.139+0000 7f36ddffb700 10 client.15936 handle_command_reply: tid=0 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 client.15936 shutdown 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 msgr2=0x5648891e3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009790 tx=0x7f36f0007f40 comp rx=0 tx=0).stop 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 shutdown_connections 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889206b90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f36ddffb700 0 client.15936 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891e0880 0x5648891e3e40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.140+0000 7f3707447b80 1 --2- 172.21.15.87:0/141036127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.141+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 >> 172.21.15.87:0/141036127 conn(0x564889107080 msgr2=0x5648891d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.141+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 shutdown_connections 2024-06-06T07:49:49.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.141+0000 7f3707447b80 1 -- 172.21.15.87:0/141036127 wait complete. 2024-06-06T07:49:49.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.141+0000 7f3707447b80 20 client.15936 trim_cache size 0 max 16384 2024-06-06T07:49:49.145 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:49.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 Processor -- start 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 -- start start 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 0x5648891d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36f81aecb0 con 0x5648891dff60 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36f81ac540 con 0x5648891da7a0 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.170+0000 7f3707447b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8072ba0 con 0x5648891e0880 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f5d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 0x5648891d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f5d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60680/0 (socket says 172.21.15.87:60680) 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f5d9b700 1 -- 172.21.15.87:0/481754924 learned_addr learned my addr 172.21.15.87:0/481754924 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:49.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f659c700 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 -- 172.21.15.87:0/481754924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 msgr2=0x5648891da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 -- 172.21.15.87:0/481754924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 msgr2=0x5648891e2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f6d9d700 1 -- 172.21.15.87:0/481754924 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564889201570 con 0x5648891e0880 2024-06-06T07:49:49.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.171+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.172+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 0x5648891d4e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36f0016550 tx=0x7f36f0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97cf106b574428d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.172+0000 7f36dffff700 1 -- 172.21.15.87:0/481754924 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f002d070 con 0x5648891e0880 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.172+0000 7f36dffff700 1 -- 172.21.15.87:0/481754924 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec09b850 con 0x5648891e0880 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.172+0000 7f36dffff700 1 -- 172.21.15.87:0/481754924 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f001a030 con 0x5648891e0880 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.173+0000 7f3707447b80 1 -- 172.21.15.87:0/481754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 msgr2=0x5648891d4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.173+0000 7f3707447b80 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 0x5648891d4e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36f0016550 tx=0x7f36f0011000 comp rx=0 tx=0).stop 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.173+0000 7f3707447b80 1 -- 172.21.15.87:0/481754924 shutdown_connections 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.173+0000 7f3707447b80 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891da7a0 0x5648891da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.173+0000 7f3707447b80 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891e0880 0x5648891d4e80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 --2- 172.21.15.87:0/481754924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891dff60 0x5648891e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 -- 172.21.15.87:0/481754924 >> 172.21.15.87:0/481754924 conn(0x564889107080 msgr2=0x5648891e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 -- 172.21.15.87:0/481754924 shutdown_connections 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 -- 172.21.15.87:0/481754924 wait complete. 2024-06-06T07:49:49.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 -- start start 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.174+0000 7f3707447b80 1 Processor -- start 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891e0880 0x56488927e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8073210 con 0x5648891da7a0 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36f81abea0 con 0x5648891e0880 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.175+0000 7f3707447b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80738e0 con 0x5648891dff60 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f3707447b80 10 client.0 _ll_register_callbacks cb 0x5648890c15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:49.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60694/0 (socket says 172.21.15.87:60694) 2024-06-06T07:49:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f659c700 1 -- 172.21.15.87:0/3747776776 learned_addr learned my addr 172.21.15.87:0/3747776776 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f6d9d700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891e0880 0x56488927e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891e0880 msgr2=0x56488927e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891e0880 0x56488927e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3747776776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 msgr2=0x5648891f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f5d9b700 1 -- 172.21.15.87:0/3747776776 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488927e990 con 0x5648891dff60 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.176+0000 7f36f659c700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.177+0000 7f36f5d9b700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891f8bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36ec09b550 tx=0x7f36ec09b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c851f8b9db7984 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.177+0000 7f36ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.177+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0a4430 con 0x5648891dff60 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.177+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f36ec0094b0 con 0x5648891dff60 2024-06-06T07:49:49.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.177+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec09b850 con 0x5648891dff60 2024-06-06T07:49:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.181+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36ec09fc00 con 0x5648891dff60 2024-06-06T07:49:49.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.181+0000 7f3707447b80 10 client.25387 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.181+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56488927fc20 con 0x5648891dff60 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.181+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec0ba180 con 0x5648891dff60 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f36ec0afce0 con 0x5648891dff60 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 10 client.25387 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 10 client.25387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 20 client.25387 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889282270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:49.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 4 client.25387 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:49.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.182+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f36ec09b850 con 0x5648892047a0 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.183+0000 7f36f659c700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889282270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.183+0000 7f36f659c700 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889282270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11d687d9ef8826f7 server_cookie=e0bfb405908c92c3 in_seq=0 out_seq=0 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.183+0000 7f36ddffb700 10 client.25387 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f36ddffb700 1 -- 172.21.15.87:0/3747776776 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f36ec09b850 con 0x5648892047a0 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f36ddffb700 10 client.25387 handle_command_reply: tid=0 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 client.25387 shutdown 2024-06-06T07:49:49.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 msgr2=0x5648891f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891f8bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36ec09b550 tx=0x7f36ec09b490 comp rx=0 tx=0).stop 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 shutdown_connections 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648891e0880 0x56488927e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5648892047a0 0x564889282270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f36ddffb700 0 client.25387 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648891da7a0 0x5648891f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 --2- 172.21.15.87:0/3747776776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648891dff60 0x5648891f8bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.184+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 >> 172.21.15.87:0/3747776776 conn(0x564889107080 msgr2=0x5648891e59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.185+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 shutdown_connections 2024-06-06T07:49:49.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.185+0000 7f3707447b80 1 -- 172.21.15.87:0/3747776776 wait complete. 2024-06-06T07:49:49.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.185+0000 7f3707447b80 20 client.25387 trim_cache size 0 max 16384 2024-06-06T07:49:49.186 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 msgr2=0x7f36e00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 0x7f36e00a5350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009c90 tx=0x7f36e8009260 comp rx=0 tx=0).stop 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 msgr2=0x7f36f810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810d2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0098e0 tx=0x7f36ec004310 comp rx=0 tx=0).stop 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 shutdown_connections 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f8107400 0x7f36f81a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f36e00a2e90 0x7f36e00a5350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f8101390 0x7f36f810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 --2- 172.21.15.87:0/3024857818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f8101d50 0x7f36f810d2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 >> 172.21.15.87:0/3024857818 conn(0x7f36f8078340 msgr2=0x7f36f8112af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 shutdown_connections 2024-06-06T07:49:49.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:49.186+0000 7f36fcbc4700 1 -- 172.21.15.87:0/3024857818 wait complete. 2024-06-06T07:49:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:50 smithi195 ceph-mon[37040]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:50.546 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:50 smithi087 ceph-mon[32276]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:50 smithi181 ceph-mon[36790]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:50.583 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:49:51.326 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:51.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 -- 172.21.15.87:0/1445692847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 msgr2=0x7fab2410d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 --2- 172.21.15.87:0/1445692847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2410d9d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b0a0 tx=0x7fab1c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 -- 172.21.15.87:0/1445692847 shutdown_connections 2024-06-06T07:49:51.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 --2- 172.21.15.87:0/1445692847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 0x7fab24106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 --2- 172.21.15.87:0/1445692847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2410d9d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 --2- 172.21.15.87:0/1445692847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 0x7fab24100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 -- 172.21.15.87:0/1445692847 >> 172.21.15.87:0/1445692847 conn(0x7fab24077a00 msgr2=0x7fab240fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 -- 172.21.15.87:0/1445692847 shutdown_connections 2024-06-06T07:49:51.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.581+0000 7fab28a41700 1 -- 172.21.15.87:0/1445692847 wait complete. 2024-06-06T07:49:51.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 Processor -- start 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 -- start start 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 0x7fab2419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 0x7fab24197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab241116f0 con 0x7fab24101510 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111570 con 0x7fab24106b90 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab28a41700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111870 con 0x7fab24100b50 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.582+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60714/0 (socket says 172.21.15.87:60714) 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 -- 172.21.15.87:0/4024417361 learned_addr learned my addr 172.21.15.87:0/4024417361 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab2259c700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 0x7fab2419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab22d9d700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 0x7fab24197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 -- 172.21.15.87:0/4024417361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 msgr2=0x7fab2419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 0x7fab2419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 -- 172.21.15.87:0/4024417361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 msgr2=0x7fab24197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 0x7fab24197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 -- 172.21.15.87:0/4024417361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab241980d0 con 0x7fab24101510 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab21d9b700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c9b0 tx=0x7fab1800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab137fe700 1 -- 172.21.15.87:0/4024417361 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18007b00 con 0x7fab24101510 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241983c0 con 0x7fab24101510 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab137fe700 1 -- 172.21.15.87:0/4024417361 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1800ce50 con 0x7fab24101510 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab137fe700 1 -- 172.21.15.87:0/4024417361 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18018660 con 0x7fab24101510 2024-06-06T07:49:51.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.583+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2410e390 con 0x7fab24101510 2024-06-06T07:49:51.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.587+0000 7fab137fe700 1 -- 172.21.15.87:0/4024417361 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fab18004830 con 0x7fab24101510 2024-06-06T07:49:51.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.587+0000 7fab137fe700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 0x7fab0c0a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.587+0000 7fab332c4b80 1 Processor -- start 2024-06-06T07:49:51.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab137fe700 1 -- 172.21.15.87:0/4024417361 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fab180cf7a0 con 0x7fab24101510 2024-06-06T07:49:51.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab2259c700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 0x7fab0c0a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 -- start start 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122bf1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a122b39910 con 0x55a122bf6d70 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a122ae1820 con 0x55a122bf0be0 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab332c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a122b358d0 con 0x55a122bf6450 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab2259c700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 0x7fab0c0a9660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab24100090 tx=0x7fab14009190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60720/0 (socket says 172.21.15.87:60720) 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 -- 172.21.15.87:0/874749308 learned_addr learned my addr 172.21.15.87:0/874749308 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab2259c700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab22d9d700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122bf1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 -- 172.21.15.87:0/874749308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 msgr2=0x55a122bf6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 -- 172.21.15.87:0/874749308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 msgr2=0x55a122bf1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122bf1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab21d9b700 1 -- 172.21.15.87:0/874749308 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a122ad8050 con 0x55a122bf6d70 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.588+0000 7fab22d9d700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122bf1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.589+0000 7fab21d9b700 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fab18097d50 tx=0x7fab1809c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e2fa5b5bbaef0a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.589+0000 7fab10ff9700 1 -- 172.21.15.87:0/874749308 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18099bf0 con 0x55a122bf6d70 2024-06-06T07:49:51.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.589+0000 7fab10ff9700 1 -- 172.21.15.87:0/874749308 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab180ccda0 con 0x55a122bf6d70 2024-06-06T07:49:51.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.589+0000 7fab10ff9700 1 -- 172.21.15.87:0/874749308 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180cf390 con 0x55a122bf6d70 2024-06-06T07:49:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 -- 172.21.15.87:0/874749308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 msgr2=0x55a122bf06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fab18097d50 tx=0x7fab1809c0f0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 -- 172.21.15.87:0/874749308 shutdown_connections 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122bf1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 --2- 172.21.15.87:0/874749308 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6d70 0x55a122bf06a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 -- 172.21.15.87:0/874749308 >> 172.21.15.87:0/874749308 conn(0x55a122b12dc0 msgr2=0x55a122be3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 -- 172.21.15.87:0/874749308 shutdown_connections 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.590+0000 7fab332c4b80 1 -- 172.21.15.87:0/874749308 wait complete. 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 -- start start 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 Processor -- start 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6d70 0x55a122bfa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a122bfd7e0 con 0x55a122bf0be0 2024-06-06T07:49:51.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a122ad9cf0 con 0x55a122bf6d70 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a122bfd960 con 0x55a122bf6450 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab2259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab2259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60732/0 (socket says 172.21.15.87:60732) 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab332c4b80 10 client.0 _ll_register_callbacks cb 0x55a122ad3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab2259c700 1 -- 172.21.15.87:0/2528856188 learned_addr learned my addr 172.21.15.87:0/2528856188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.591+0000 7fab21d9b700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 -- 172.21.15.87:0/2528856188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 msgr2=0x55a122bf9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 -- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6d70 msgr2=0x55a122bfa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6d70 0x55a122bfa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 -- 172.21.15.87:0/2528856188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a122a889d0 con 0x55a122bf0be0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab2259c700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fab1400a010 tx=0x7fab14004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8fe05c2b689d67 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:51.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14025020 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab180ccda0 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fab14015b60 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab14030050 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab332c4b80 10 client.15954 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.592+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a122c08180 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab180ccda0 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab140134e0 con 0x55a122bf0be0 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 10 client.15954 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 20 client.15954 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c0a710 0x55a122c0cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 4 client.15954 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a122b46e10 con 0x55a122c0a710 2024-06-06T07:49:51.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.593+0000 7fab21d9b700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c0a710 0x55a122c0cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.594+0000 7fab09ffb700 10 client.15954 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.594+0000 7fab21d9b700 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c0a710 0x55a122c0cb40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc901b663fb4068 server_cookie=c151d30e40aa82d5 in_seq=0 out_seq=0 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab09ffb700 1 -- 172.21.15.87:0/2528856188 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a122b46e10 con 0x55a122c0a710 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab09ffb700 10 client.15954 handle_command_reply: tid=0 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 client.15954 shutdown 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 msgr2=0x55a122bfcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fab1400a010 tx=0x7fab14004860 comp rx=0 tx=0).stop 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 shutdown_connections 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c0a710 0x55a122c0cb40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6d70 0x55a122bfa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122bfcb30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab09ffb700 0 client.15954 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2528856188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6450 0x55a122bf9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 >> 172.21.15.87:0/2528856188 conn(0x55a122b12dc0 msgr2=0x55a122be6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 shutdown_connections 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 1 -- 172.21.15.87:0/2528856188 wait complete. 2024-06-06T07:49:51.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.595+0000 7fab332c4b80 20 client.15954 trim_cache size 0 max 16384 2024-06-06T07:49:51.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.632+0000 7fab332c4b80 1 Processor -- start 2024-06-06T07:49:51.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.632+0000 7fab332c4b80 1 -- start start 2024-06-06T07:49:51.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 0x55a122c12410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122c12950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab241aed40 con 0x55a122bf0be0 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab241ac5d0 con 0x55a122bf6450 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab332c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab241aa420 con 0x55a122bf6d70 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60742/0 (socket says 172.21.15.87:60742) 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 -- 172.21.15.87:0/2647825074 learned_addr learned my addr 172.21.15.87:0/2647825074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab22d9d700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122c12950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab21d9b700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 0x55a122c12410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 -- 172.21.15.87:0/2647825074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 msgr2=0x55a122c12950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122c12950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 -- 172.21.15.87:0/2647825074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 msgr2=0x55a122c12410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 0x55a122c12410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 -- 172.21.15.87:0/2647825074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a122bf8df0 con 0x55a122bf0be0 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab21d9b700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 0x55a122c12410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab2259c700 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fab140097f0 tx=0x7fab1400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8238f41ec9647d6f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.633+0000 7fab0bfff700 1 -- 172.21.15.87:0/2647825074 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14038020 con 0x55a122bf0be0 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab0bfff700 1 -- 172.21.15.87:0/2647825074 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab14013650 con 0x55a122bf0be0 2024-06-06T07:49:51.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab0bfff700 1 -- 172.21.15.87:0/2647825074 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14025040 con 0x55a122bf0be0 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 -- 172.21.15.87:0/2647825074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 msgr2=0x55a122be2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fab140097f0 tx=0x7fab1400e6c0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 -- 172.21.15.87:0/2647825074 shutdown_connections 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122c12950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf0be0 0x55a122be2f40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 --2- 172.21.15.87:0/2647825074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf6450 0x55a122c12410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 -- 172.21.15.87:0/2647825074 >> 172.21.15.87:0/2647825074 conn(0x55a122b12dc0 msgr2=0x55a122be2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 -- 172.21.15.87:0/2647825074 shutdown_connections 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.634+0000 7fab332c4b80 1 -- 172.21.15.87:0/2647825074 wait complete. 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 -- start start 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 Processor -- start 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122c02e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122bffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab241aaa90 con 0x55a122bf6450 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab241abf00 con 0x55a122bf0be0 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab241ab160 con 0x55a122bf6d70 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab332c4b80 10 client.0 _ll_register_callbacks cb 0x55a122ad3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60756/0 (socket says 172.21.15.87:60756) 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab21d9b700 1 -- 172.21.15.87:0/3435553960 learned_addr learned my addr 172.21.15.87:0/3435553960 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.635+0000 7fab2259c700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122c02e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 -- 172.21.15.87:0/3435553960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 msgr2=0x55a122bffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab22d9d700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122bffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122bffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 -- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 msgr2=0x55a122c02e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122c02e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 -- 172.21.15.87:0/3435553960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a122c002f0 con 0x55a122bf6450 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab2259c700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122c02e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab22d9d700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122bffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab21d9b700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fab18004e20 tx=0x7fab1800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dea2403562a1ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180ba200 con 0x55a122bf6450 2024-06-06T07:49:51.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fab18097f60 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab14013650 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab180860d0 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 10 client.15960 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a122c75e30 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab14013650 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab1809bb50 con 0x55a122bf6450 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 10 client.15960 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.636+0000 7fab332c4b80 20 client.15960 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c760c0 0x55a122c784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab332c4b80 4 client.15960 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "544e979e", "release": true, "await": true}] 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "544e979e", "release": true, "await": true}) -- 0x55a122b41390 con 0x55a122c760c0 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab2259c700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c760c0 0x55a122c784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:51.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab09ffb700 10 client.15960 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:51.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.637+0000 7fab2259c700 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c760c0 0x55a122c784b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d015d31f515d572 server_cookie=2cd47572ad843be1 in_seq=0 out_seq=0 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab09ffb700 1 -- 172.21.15.87:0/3435553960 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a122b41390 con 0x55a122c760c0 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab09ffb700 10 client.15960 handle_command_reply: tid=0 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 client.15960 shutdown 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 msgr2=0x55a122c03340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fab18004e20 tx=0x7fab1800fcc0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 shutdown_connections 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a122c760c0 0x55a122c784b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a122bf0be0 0x55a122c02e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a122bf6d70 0x55a122bffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 --2- 172.21.15.87:0/3435553960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a122bf6450 0x55a122c03340 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 >> 172.21.15.87:0/3435553960 conn(0x55a122b12dc0 msgr2=0x55a122be30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab09ffb700 0 client.15960 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:51.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 shutdown_connections 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 1 -- 172.21.15.87:0/3435553960 wait complete. 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.639+0000 7fab332c4b80 20 client.15960 trim_cache size 0 max 16384 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 27, 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 4, 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "544e979e": { 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 4, 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:49:51.644 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 154.4, 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:51.645 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 msgr2=0x7fab0c0a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 0x7fab0c0a9660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab24100090 tx=0x7fab14009190 comp rx=0 tx=0).stop 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 msgr2=0x7fab2419dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c9b0 tx=0x7fab1800ccc0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 shutdown_connections 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab24106b90 0x7fab24197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fab0c0a71a0 0x7fab0c0a9660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab24100b50 0x7fab2419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 --2- 172.21.15.87:0/4024417361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab24101510 0x7fab2419dca0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 >> 172.21.15.87:0/4024417361 conn(0x7fab24077a00 msgr2=0x7fab241038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 shutdown_connections 2024-06-06T07:49:51.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:51.640+0000 7fab28a41700 1 -- 172.21.15.87:0/4024417361 wait complete. 2024-06-06T07:49:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:52 smithi195 ceph-mon[37040]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:52 smithi181 ceph-mon[36790]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:52 smithi087 ceph-mon[32276]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T07:49:52.868 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '544e979e', total seconds elapsed: 46.9 2024-06-06T07:49:54.349 INFO:tasks.workunit.client.0.smithi087.stderr:Note: switching to '3d746ca5346f34eef7e826a1eb71e59e02b682ae'. 2024-06-06T07:49:54.349 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.349 INFO:tasks.workunit.client.0.smithi087.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: git switch -c 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:Or undo this operation with: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: git switch - 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.350 INFO:tasks.workunit.client.0.smithi087.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T07:49:54.351 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-06-06T07:49:54.351 INFO:tasks.workunit.client.0.smithi087.stderr:HEAD is now at 3d746ca5346 Merge pull request #57870 from zdover23/wip-doc-2024-06-05-backport-57867-to-squid 2024-06-06T07:49:54.354 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-06T07:49:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:54 smithi087 ceph-mon[32276]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:54.421 INFO:tasks.workunit.client.0.smithi087.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-06T07:49:54.423 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T07:49:54.424 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-06T07:49:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:54 smithi195 ceph-mon[37040]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:54 smithi181 ceph-mon[36790]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 -- 172.21.15.87:0/1144159489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101390 msgr2=0x7f3d0c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/1144159489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101390 0x7f3d0c101810 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009990 tx=0x7f3cf8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 -- 172.21.15.87:0/1144159489 shutdown_connections 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/1144159489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 0x7f3d0c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/1144159489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101d50 0x7f3d0c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/1144159489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101390 0x7f3d0c101810 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 -- 172.21.15.87:0/1144159489 >> 172.21.15.87:0/1144159489 conn(0x7f3d0c078340 msgr2=0x7f3d0c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 -- 172.21.15.87:0/1144159489 shutdown_connections 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.699+0000 7f3d12ca3700 1 -- 172.21.15.87:0/1144159489 wait complete. 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.700+0000 7f3d12ca3700 1 Processor -- start 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.700+0000 7f3d12ca3700 1 -- start start 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.700+0000 7f3d12ca3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101390 0x7f3d0c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 0x7f3d0c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1116a0 con 0x7f3d0c101d50 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c111520 con 0x7f3d0c107400 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c111820 con 0x7f3d0c101390 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d10a3f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101390 0x7f3d0c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60776/0 (socket says 172.21.15.87:60776) 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 -- 172.21.15.87:0/3281143869 learned_addr learned my addr 172.21.15.87:0/3281143869 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d11240700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 0x7f3d0c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 -- 172.21.15.87:0/3281143869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101390 msgr2=0x7f3d0c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101390 0x7f3d0c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 -- 172.21.15.87:0/3281143869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 msgr2=0x7f3d0c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 0x7f3d0c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 -- 172.21.15.87:0/3281143869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c199160 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000c00 tx=0x7f3cf80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d09ffb700 1 -- 172.21.15.87:0/3281143869 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8027070 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1993f0 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d09ffb700 1 -- 172.21.15.87:0/3281143869 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8003e40 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.701+0000 7f3d09ffb700 1 -- 172.21.15.87:0/3281143869 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8022d10 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.702+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a5710 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.704+0000 7f3d09ffb700 1 -- 172.21.15.87:0/3281143869 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3cf802a070 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.704+0000 7f3d09ffb700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 0x7f3cfc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.704+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 0x7f3cfc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.704+0000 7f3d09ffb700 1 -- 172.21.15.87:0/3281143869 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cf80d0620 con 0x7f3d0c101d50 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.705+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 0x7f3cfc0a5250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c199b40 tx=0x7f3d0000b410 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.705+0000 7f3d1d526b80 1 Processor -- start 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.705+0000 7f3d1d526b80 1 -- start start 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a3657b3f60 con 0x55a3658c17a0 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a365812850 con 0x55a3658c6f60 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a36577a460 con 0x55a3658c7880 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60786/0 (socket says 172.21.15.87:60786) 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 -- 172.21.15.87:0/3654512917 learned_addr learned my addr 172.21.15.87:0/3654512917 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 -- 172.21.15.87:0/3654512917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 msgr2=0x55a3658c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 -- 172.21.15.87:0/3654512917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 msgr2=0x55a3658c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.752 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:54.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 -- 172.21.15.87:0/3654512917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3657618d0 con 0x55a3658c17a0 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3d11240700 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3d040098f0 tx=0x7f3d04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a89bd1627992d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3cf6ffd700 1 -- 172.21.15.87:0/3654512917 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04004bb0 con 0x55a3658c17a0 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3cf6ffd700 1 -- 172.21.15.87:0/3654512917 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04004d50 con 0x55a3658c17a0 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.706+0000 7f3cf6ffd700 1 -- 172.21.15.87:0/3654512917 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04015480 con 0x55a3658c17a0 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3654512917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 msgr2=0x55a3658c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3d040098f0 tx=0x7f3d04009c00 comp rx=0 tx=0).stop 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3654512917 shutdown_connections 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c17a0 0x55a3658c1bc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3654512917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3654512917 >> 172.21.15.87:0/3654512917 conn(0x55a3657ee080 msgr2=0x55a3658bc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3654512917 shutdown_connections 2024-06-06T07:49:54.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.707+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3654512917 wait complete. 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 -- start start 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 Processor -- start 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 0x55a3658cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a3657b2fc0 con 0x55a3658c6f60 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a36580e810 con 0x55a3658c7880 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a3658ce260 con 0x55a3658c17a0 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d1d526b80 10 client.0 _ll_register_callbacks cb 0x55a3657a85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60788/0 (socket says 172.21.15.87:60788) 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d0bfff700 1 -- 172.21.15.87:0/1901960800 learned_addr learned my addr 172.21.15.87:0/1901960800 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:54.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.708+0000 7f3d11240700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 0x55a3658cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 -- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 msgr2=0x55a3658cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 -- 172.21.15.87:0/1901960800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 msgr2=0x55a3658cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 0x55a3658cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 -- 172.21.15.87:0/1901960800 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3657d8750 con 0x55a3658c6f60 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d11240700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 0x55a3658cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80cda50 tx=0x7f3cf809c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ca192b2370429 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80a0bb0 con 0x55a3658c6f60 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cf8098e70 con 0x55a3658c6f60 2024-06-06T07:49:54.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d04004d50 con 0x55a3658c6f60 2024-06-06T07:49:54.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cf80a53e0 con 0x55a3658c6f60 2024-06-06T07:49:54.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 10 client.15972 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3658d8960 con 0x55a3658c6f60 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04004d50 con 0x55a3658c6f60 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3cf80d0300 con 0x55a3658c6f60 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 10 client.15972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 20 client.15972 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a3658daf20 0x55a3658dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 4 client.15972 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.709+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36581fd50 con 0x55a3658daf20 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.710+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a3658daf20 0x55a3658dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.710+0000 7f3cebfff700 10 client.15972 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:54.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.710+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a3658daf20 0x55a3658dd310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=245c88435c7a54f6 server_cookie=4220925dc85171f4 in_seq=0 out_seq=0 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3cebfff700 1 -- 172.21.15.87:0/1901960800 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a36581fd50 con 0x55a3658daf20 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3cebfff700 10 client.15972 handle_command_reply: tid=0 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 client.15972 shutdown 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 msgr2=0x55a3658ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80cda50 tx=0x7f3cf809c1e0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 shutdown_connections 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a3658daf20 0x55a3658dd310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c7880 0x55a3658cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3cebfff700 0 client.15972 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/1901960800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658ca8a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 >> 172.21.15.87:0/1901960800 conn(0x55a3657ee080 msgr2=0x55a3658bb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 shutdown_connections 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 1 -- 172.21.15.87:0/1901960800 wait complete. 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.711+0000 7f3d1d526b80 20 client.15972 trim_cache size 0 max 16384 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 Processor -- start 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 -- start start 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1b0da0 con 0x55a3658c7880 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1ae630 con 0x55a3658c6f60 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1ac480 con 0x55a3658c17a0 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d11240700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.726+0000 7f3d11240700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60790/0 (socket says 172.21.15.87:60790) 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 -- 172.21.15.87:0/3238080311 learned_addr learned my addr 172.21.15.87:0/3238080311 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:54.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 -- 172.21.15.87:0/3238080311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 msgr2=0x55a3658c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 -- 172.21.15.87:0/3238080311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 msgr2=0x55a3658c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 -- 172.21.15.87:0/3238080311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3658dd990 con 0x55a3658c7880 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3d11240700 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004040 tx=0x7f3d040095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9bde0f71b2de40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3cf67fc700 1 -- 172.21.15.87:0/3238080311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04005710 con 0x55a3658c7880 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3cf67fc700 1 -- 172.21.15.87:0/3238080311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d040058b0 con 0x55a3658c7880 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.727+0000 7f3cf67fc700 1 -- 172.21.15.87:0/3238080311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04015880 con 0x55a3658c7880 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3238080311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 msgr2=0x55a3658cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004040 tx=0x7f3d040095f0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3238080311 shutdown_connections 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c7880 0x55a3658cc7e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c17a0 0x55a3658c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/3238080311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c6f60 0x55a3658c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3238080311 >> 172.21.15.87:0/3238080311 conn(0x55a3657ee080 msgr2=0x55a3658cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3238080311 shutdown_connections 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- 172.21.15.87:0/3238080311 wait complete. 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 -- start start 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 Processor -- start 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c17a0 0x55a3658e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.728+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1acaf0 con 0x55a3658c6f60 2024-06-06T07:49:54.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1adf60 con 0x55a3658c17a0 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1ad1c0 con 0x55a3658c7880 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 10 client.0 _ll_register_callbacks cb 0x55a3657a85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60802/0 (socket says 172.21.15.87:60802) 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 -- 172.21.15.87:0/2216702891 learned_addr learned my addr 172.21.15.87:0/2216702891 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c17a0 0x55a3658e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d11240700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 -- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 msgr2=0x55a3658e32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658e32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 -- 172.21.15.87:0/2216702891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c17a0 msgr2=0x55a3658e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c17a0 0x55a3658e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 -- 172.21.15.87:0/2216702891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3658c5360 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d0bfff700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3cf809aab0 tx=0x7f3cf8005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982ee6907d8cfeea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3cebfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8065070 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cf8098e70 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d040058b0 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cf80a0bb0 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 10 client.15978 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.729+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3658c5360 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d040058b0 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3cf80d0200 con 0x55a3658c6f60 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 10 client.15978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 20 client.15978 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:54.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a365946050 0x55a365948470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 4 client.15978 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a36581a2d0 con 0x55a365946050 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a365946050 0x55a365948470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3cebfff700 10 client.15978 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.730+0000 7f3d10a3f700 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a365946050 0x55a365948470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5de28d6891db0241 server_cookie=a2af16029ee25849 in_seq=0 out_seq=0 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3cebfff700 1 -- 172.21.15.87:0/2216702891 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a36581a2d0 con 0x55a365946050 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3cebfff700 10 client.15978 handle_command_reply: tid=0 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 client.15978 shutdown 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 msgr2=0x55a3658e2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3cf809aab0 tx=0x7f3cf8005860 comp rx=0 tx=0).stop 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 shutdown_connections 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x55a365946050 0x55a365948470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3658c17a0 0x55a3658e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3658c7880 0x55a3658e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 --2- 172.21.15.87:0/2216702891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3658c6f60 0x55a3658e2d90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3cebfff700 0 client.15978 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 >> 172.21.15.87:0/2216702891 conn(0x55a3657ee080 msgr2=0x55a3658c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 shutdown_connections 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 1 -- 172.21.15.87:0/2216702891 wait complete. 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d1d526b80 20 client.15978 trim_cache size 0 max 16384 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 msgr2=0x7f3cfc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.731+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 0x7f3cfc0a5250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c199b40 tx=0x7f3d0000b410 comp rx=0 tx=0).stop 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 msgr2=0x7f3d0c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000c00 tx=0x7f3cf80047b0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 shutdown_connections 2024-06-06T07:49:54.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c107400 0x7f3d0c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cfc0a2d90 0x7f3cfc0a5250 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d0c101390 0x7f3d0c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 --2- 172.21.15.87:0/3281143869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d0c101d50 0x7f3d0c19ed80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 >> 172.21.15.87:0/3281143869 conn(0x7f3d0c078340 msgr2=0x7f3d0c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 shutdown_connections 2024-06-06T07:49:54.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:54.732+0000 7f3d12ca3700 1 -- 172.21.15.87:0/3281143869 wait complete. 2024-06-06T07:49:55.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:49:55.969 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:49:56.276 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-06T07:49:56.341 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-06T07:49:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:56 smithi087 ceph-mon[32276]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:49:56.381 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T07:49:56.383 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T07:49:56.383 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-06T07:49:56.427 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T07:49:56.431 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T07:49:56.431 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-06T07:49:56.448 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-06T07:49:56.449 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-06T07:49:56.450 DEBUG:teuthology.orchestra.run.smithi087:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-06T07:49:56.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:56 smithi195 ceph-mon[37040]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:49:56.510 INFO:tasks.workunit.client.0.smithi087.stderr:+ dbench 1 2024-06-06T07:49:56.532 INFO:tasks.workunit.client.0.smithi087.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-06T07:49:56.533 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-06-06T07:49:56.533 INFO:tasks.workunit.client.0.smithi087.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-06T07:49:56.548 INFO:tasks.workunit.client.0.smithi087.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-06T07:49:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:56 smithi181 ceph-mon[36790]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:49:56.648 INFO:tasks.workunit.client.0.smithi087.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-06T07:49:56.648 INFO:tasks.workunit.client.0.smithi087.stdout:releasing clients 2024-06-06T07:49:57.648 INFO:tasks.workunit.client.0.smithi087.stdout: 1 3024 33.54 MB/sec warmup 1 sec latency 33.221 ms 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 -- 172.21.15.87:0/3357095318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c102740 msgr2=0x7fca6c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 --2- 172.21.15.87:0/3357095318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c102740 0x7fca6c102b20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0099c0 tx=0x7fca5c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 -- 172.21.15.87:0/3357095318 shutdown_connections 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 --2- 172.21.15.87:0/3357095318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c107840 0x7fca6c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 --2- 172.21.15.87:0/3357095318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c102740 0x7fca6c102b20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 --2- 172.21.15.87:0/3357095318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 -- 172.21.15.87:0/3357095318 >> 172.21.15.87:0/3357095318 conn(0x7fca6c0795f0 msgr2=0x7fca6c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 -- 172.21.15.87:0/3357095318 shutdown_connections 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.882+0000 7fca7087f700 1 -- 172.21.15.87:0/3357095318 wait complete. 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 Processor -- start 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 -- start start 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 0x7fca6c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112780 con 0x7fca6c107840 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112600 con 0x7fca6c1030f0 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca7087f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112900 con 0x7fca6c102740 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca6ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 0x7fca6c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56020/0 (socket says 172.21.15.87:56020) 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca6ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 0x7fca6c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45846/0 (socket says 172.21.15.87:45846) 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca63fff700 1 -- 172.21.15.87:0/2282233306 learned_addr learned my addr 172.21.15.87:0/2282233306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.883+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54932/0 (socket says 172.21.15.87:54932) 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 -- 172.21.15.87:0/2282233306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 msgr2=0x7fca6c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 -- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 msgr2=0x7fca6c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 0x7fca6c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 -- 172.21.15.87:0/2282233306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c199160 con 0x7fca6c102740 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca63fff700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca6a59c700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0098e0 tx=0x7fca5c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca637fe700 1 -- 172.21.15.87:0/2282233306 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c027070 con 0x7fca6c102740 2024-06-06T07:49:57.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c1993f0 con 0x7fca6c102740 2024-06-06T07:49:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca637fe700 1 -- 172.21.15.87:0/2282233306 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c003680 con 0x7fca6c102740 2024-06-06T07:49:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca637fe700 1 -- 172.21.15.87:0/2282233306 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0173f0 con 0x7fca6c102740 2024-06-06T07:49:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.884+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c1a5710 con 0x7fca6c102740 2024-06-06T07:49:57.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.886+0000 7fca637fe700 1 -- 172.21.15.87:0/2282233306 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fca5c02a070 con 0x7fca6c102740 2024-06-06T07:49:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.887+0000 7fca637fe700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 0x7fca580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.887+0000 7fca63fff700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 0x7fca580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.887+0000 7fca637fe700 1 -- 172.21.15.87:0/2282233306 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fca5c0d0400 con 0x7fca6c102740 2024-06-06T07:49:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca63fff700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 0x7fca580a5250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca6c102060 tx=0x7fca54007480 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 Processor -- start 2024-06-06T07:49:57.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 -- start start 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 0x562f171396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 0x562f1713a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.888+0000 7fca7b102b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562f17082810 con 0x562f17134fd0 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca7b102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562f1702a720 con 0x562f17139c30 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca7b102b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f1707e7d0 con 0x562f171358f0 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca63fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 0x562f171396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45854/0 (socket says 172.21.15.87:45854) 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6ad9d700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 0x562f1713a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 -- 172.21.15.87:0/875115935 learned_addr learned my addr 172.21.15.87:0/875115935 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 -- 172.21.15.87:0/875115935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 msgr2=0x562f171396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 0x562f171396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 -- 172.21.15.87:0/875115935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 msgr2=0x562f1713a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 0x562f1713a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 -- 172.21.15.87:0/875115935 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f17020f50 con 0x562f17134fd0 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6ad9d700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 0x562f1713a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca63fff700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 0x562f171396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca6a59c700 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca5c098f50 tx=0x7fca5c09c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d535ae6c19d2da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca60ff9700 1 -- 172.21.15.87:0/875115935 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c09cc80 con 0x562f17134fd0 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca60ff9700 1 -- 172.21.15.87:0/875115935 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c09c580 con 0x562f17134fd0 2024-06-06T07:49:57.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.889+0000 7fca60ff9700 1 -- 172.21.15.87:0/875115935 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c09ca40 con 0x562f17134fd0 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 -- 172.21.15.87:0/875115935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 msgr2=0x562f171353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca5c098f50 tx=0x7fca5c09c0c0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 -- 172.21.15.87:0/875115935 shutdown_connections 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17139c30 0x562f1713a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f171353b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 --2- 172.21.15.87:0/875115935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f171358f0 0x562f171396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 -- 172.21.15.87:0/875115935 >> 172.21.15.87:0/875115935 conn(0x562f1705bcc0 msgr2=0x562f1712b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 -- 172.21.15.87:0/875115935 shutdown_connections 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.890+0000 7fca7b102b80 1 -- 172.21.15.87:0/875115935 wait complete. 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 -- start start 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 Processor -- start 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f17141240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1713eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562f17141af0 con 0x562f171358f0 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562f17022bf0 con 0x562f17134fd0 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f17141c70 con 0x562f17139c30 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca7b102b80 10 client.0 _ll_register_callbacks cb 0x562f1701c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f17141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca6ad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1713eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45862/0 (socket says 172.21.15.87:45862) 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f17141240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56038/0 (socket says 172.21.15.87:56038) 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.891+0000 7fca63fff700 1 -- 172.21.15.87:0/3151710437 learned_addr learned my addr 172.21.15.87:0/3151710437 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 -- 172.21.15.87:0/3151710437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 msgr2=0x562f1713eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1713eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 -- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 msgr2=0x562f17141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f17141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 -- 172.21.15.87:0/3151710437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f16fd18d0 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca63fff700 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca54011d40 tx=0x7fca540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c58d50e86b5569a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5401e850 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fca5401e9f0 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c09c580 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca5401ebf0 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca7b102b80 10 client.15990 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f1714c4e0 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c09c580 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca5401f430 con 0x562f171358f0 2024-06-06T07:49:57.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca7b102b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.892+0000 7fca7b102b80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca7b102b80 20 client.15990 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f1714eb20 0x562f17150f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca7b102b80 4 client.15990 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f1708fd10 con 0x562f1714eb20 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca6a59c700 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f1714eb20 0x562f17150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca6a59c700 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f1714eb20 0x562f17150f10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3473375d9ea392a0 server_cookie=267155469b66649c in_seq=0 out_seq=0 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.893+0000 7fca51ffb700 10 client.15990 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca51ffb700 1 -- 172.21.15.87:0/3151710437 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562f1708fd10 con 0x562f1714eb20 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca51ffb700 10 client.15990 handle_command_reply: tid=0 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 client.15990 shutdown 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 msgr2=0x562f1713e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca54011d40 tx=0x7fca540047c0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 shutdown_connections 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f1714eb20 0x562f17150f10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f17141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1713eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3151710437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1713e5e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 >> 172.21.15.87:0/3151710437 conn(0x562f1705bcc0 msgr2=0x562f1712f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.894+0000 7fca51ffb700 0 client.15990 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.895+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 shutdown_connections 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.895+0000 7fca7b102b80 1 -- 172.21.15.87:0/3151710437 wait complete. 2024-06-06T07:49:57.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.895+0000 7fca7b102b80 20 client.15990 trim_cache size 0 max 16384 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 Processor -- start 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 -- start start 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f1714ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1712b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aec70 con 0x562f171358f0 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ac500 con 0x562f17134fd0 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca7b102b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aa350 con 0x562f17139c30 2024-06-06T07:49:57.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45864/0 (socket says 172.21.15.87:45864) 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 -- 172.21.15.87:0/1891405113 learned_addr learned my addr 172.21.15.87:0/1891405113 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca6a59c700 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f1714ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca6ad9d700 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1712b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 -- 172.21.15.87:0/1891405113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 msgr2=0x562f1712b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1712b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 -- 172.21.15.87:0/1891405113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 msgr2=0x562f1714ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f1714ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 -- 172.21.15.87:0/1891405113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f1713f100 con 0x562f171358f0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.911+0000 7fca63fff700 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fca54007d70 tx=0x7fca5401efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f10724e255b28f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca53fff700 1 -- 172.21.15.87:0/1891405113 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5403a020 con 0x562f171358f0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca53fff700 1 -- 172.21.15.87:0/1891405113 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5401f630 con 0x562f171358f0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca53fff700 1 -- 172.21.15.87:0/1891405113 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540367f0 con 0x562f171358f0 2024-06-06T07:49:57.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 -- 172.21.15.87:0/1891405113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 msgr2=0x562f1712b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fca54007d70 tx=0x7fca5401efa0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 -- 172.21.15.87:0/1891405113 shutdown_connections 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f17134fd0 0x562f1714ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f1712b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 --2- 172.21.15.87:0/1891405113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f171358f0 0x562f1712b380 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 -- 172.21.15.87:0/1891405113 >> 172.21.15.87:0/1891405113 conn(0x562f1705bcc0 msgr2=0x562f1713e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 -- 172.21.15.87:0/1891405113 shutdown_connections 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.912+0000 7fca7b102b80 1 -- 172.21.15.87:0/1891405113 wait complete. 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 -- start start 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 Processor -- start 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 0x562f17147400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f171557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aa9c0 con 0x562f17134fd0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1abe30 con 0x562f171358f0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ab090 con 0x562f17139c30 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca7b102b80 10 client.0 _ll_register_callbacks cb 0x562f1701c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45874/0 (socket says 172.21.15.87:45874) 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca6a59c700 1 -- 172.21.15.87:0/3313314432 learned_addr learned my addr 172.21.15.87:0/3313314432 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca63fff700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 0x562f17147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.913+0000 7fca6ad9d700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f171557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 -- 172.21.15.87:0/3313314432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 msgr2=0x562f171557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f171557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 -- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 msgr2=0x562f17147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:49:57.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 0x562f17147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 -- 172.21.15.87:0/3313314432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f17155d30 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca63fff700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 0x562f17147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca6a59c700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0a4c30 tx=0x7fca5c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a666878a914e73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0b9020 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fca5c09a880 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5401f630 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca5c09c0c0 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca7b102b80 10 client.15996 fetch_fsmap learned FSMap version 27 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f171b8f90 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5401f630 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca5c09c7e0 con 0x562f17134fd0 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca7b102b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:49:57.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.914+0000 7fca7b102b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca7b102b80 20 client.15996 populate_metadata read hostname 'smithi087' 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f171b9530 0x562f171bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca7b102b80 4 client.15996 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f1708a290 con 0x562f171b9530 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca63fff700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f171b9530 0x562f171bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca51ffb700 10 client.15996 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:57.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca63fff700 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f171b9530 0x562f171bb950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9109d765f67bb2 server_cookie=9e85fd7de3752e92 in_seq=0 out_seq=0 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca51ffb700 1 -- 172.21.15.87:0/3313314432 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f1708a290 con 0x562f171b9530 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca51ffb700 10 client.15996 handle_command_reply: tid=0 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.915+0000 7fca7b102b80 1 client.15996 shutdown 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 msgr2=0x562f17146ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0a4c30 tx=0x7fca5c009310 comp rx=0 tx=0).stop 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 shutdown_connections 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562f171b9530 0x562f171bb950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f17139c30 0x562f171557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f17134fd0 0x562f17146ec0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca51ffb700 0 client.15996 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 --2- 172.21.15.87:0/3313314432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f171358f0 0x562f17147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 >> 172.21.15.87:0/3313314432 conn(0x562f1705bcc0 msgr2=0x562f1712b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 shutdown_connections 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 1 -- 172.21.15.87:0/3313314432 wait complete. 2024-06-06T07:49:57.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7b102b80 20 client.15996 trim_cache size 0 max 16384 2024-06-06T07:49:57.923 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:49:57.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 msgr2=0x7fca580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 0x7fca580a5250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca6c102060 tx=0x7fca54007480 comp rx=0 tx=0).stop 2024-06-06T07:49:57.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 msgr2=0x7fca6c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0098e0 tx=0x7fca5c00fa00 comp rx=0 tx=0).stop 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 shutdown_connections 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fca580a2d90 0x7fca580a5250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6c107840 0x7fca6c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca6c102740 0x7fca6c19e840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 --2- 172.21.15.87:0/2282233306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6c1030f0 0x7fca6c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 >> 172.21.15.87:0/2282233306 conn(0x7fca6c0795f0 msgr2=0x7fca6c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 shutdown_connections 2024-06-06T07:49:57.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:49:57.916+0000 7fca7087f700 1 -- 172.21.15.87:0/2282233306 wait complete. 2024-06-06T07:49:58.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:49:58 smithi087 ceph-mon[32276]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:49:58 smithi195 ceph-mon[37040]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:49:58 smithi181 ceph-mon[36790]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T07:49:58.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 5961 23.55 MB/sec warmup 2 sec latency 12.884 ms 2024-06-06T07:49:59.608 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:49:59.608 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:49:59.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 8265 20.42 MB/sec warmup 3 sec latency 17.674 ms 2024-06-06T07:49:59.862 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:00.249 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:00 smithi087 ceph-mon[32276]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:00.249 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:00 smithi087 ceph-mon[32276]: overall HEALTH_OK 2024-06-06T07:50:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:00 smithi195 ceph-mon[37040]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:00 smithi195 ceph-mon[37040]: overall HEALTH_OK 2024-06-06T07:50:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:00 smithi181 ceph-mon[36790]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:00 smithi181 ceph-mon[36790]: overall HEALTH_OK 2024-06-06T07:50:00.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 10839 19.40 MB/sec warmup 4 sec latency 48.791 ms 2024-06-06T07:50:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.083+0000 7f4ddab29640 1 -- 172.21.15.87:0/2389370274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 msgr2=0x7f4dd410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.083+0000 7f4ddab29640 1 --2- 172.21.15.87:0/2389370274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd410dc80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002a480 comp rx=0 tx=0).stop 2024-06-06T07:50:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- 172.21.15.87:0/2389370274 shutdown_connections 2024-06-06T07:50:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- 172.21.15.87:0/2389370274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd410e1c0 0x7f4dd4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- 172.21.15.87:0/2389370274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd410dc80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- 172.21.15.87:0/2389370274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd4073dd0 0x7f4dd40741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- 172.21.15.87:0/2389370274 >> 172.21.15.87:0/2389370274 conn(0x7f4dd40fd450 msgr2=0x7f4dd40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- 172.21.15.87:0/2389370274 shutdown_connections 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- 172.21.15.87:0/2389370274 wait complete. 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 Processor -- start 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- start start 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd4073dd0 0x7f4dd419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd410e1c0 0x7f4dd419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4113f60 con 0x7f4dd40746f0 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4113de0 con 0x7f4dd4073dd0 2024-06-06T07:50:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.084+0000 7f4ddab29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41140e0 con 0x7f4dd410e1c0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45902/0 (socket says 172.21.15.87:45902) 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 -- 172.21.15.87:0/4226684512 learned_addr learned my addr 172.21.15.87:0/4226684512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd4073dd0 0x7f4dd419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd410e1c0 0x7f4dd419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 -- 172.21.15.87:0/4226684512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd410e1c0 msgr2=0x7f4dd419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd410e1c0 0x7f4dd419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 -- 172.21.15.87:0/4226684512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd4073dd0 msgr2=0x7f4dd419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd4073dd0 0x7f4dd419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 -- 172.21.15.87:0/4226684512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd419f400 con 0x7f4dd40746f0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd1ffb640 1 -- 172.21.15.87:0/4226684512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc003f030 con 0x7f4dd40746f0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd1ffb640 1 -- 172.21.15.87:0/4226684512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0004440 con 0x7f4dd40746f0 2024-06-06T07:50:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd419f690 con 0x7f4dd40746f0 2024-06-06T07:50:01.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.085+0000 7f4dd1ffb640 1 -- 172.21.15.87:0/4226684512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0035070 con 0x7f4dd40746f0 2024-06-06T07:50:01.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.086+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd41a6f80 con 0x7f4dd40746f0 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4dd1ffb640 1 -- 172.21.15.87:0/4226684512 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4dc0042030 con 0x7f4dd40746f0 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 Processor -- start 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 -- start start 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 0x55fcee728740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fcee67f4f0 con 0x55fcee71ba90 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fcee61d870 con 0x55fcee726350 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.088+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fcee67b4b0 con 0x55fcee71b170 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd90c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45912/0 (socket says 172.21.15.87:45912) 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 -- 172.21.15.87:0/2064082309 learned_addr learned my addr 172.21.15.87:0/2064082309 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 0x55fcee728740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd1ffb640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 0x7f4dc40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 -- 172.21.15.87:0/2064082309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 msgr2=0x55fcee71b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 -- 172.21.15.87:0/2064082309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 msgr2=0x55fcee728740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 0x55fcee728740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 -- 172.21.15.87:0/2064082309 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcee6399b0 con 0x55fcee71ba90 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 0x55fcee728740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 0x7f4dc40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd1ffb640 1 -- 172.21.15.87:0/4226684512 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4dc00efae0 con 0x7f4dd40746f0 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009d20 tx=0x7f4dc8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952365bdb94e0772 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dbeffd640 1 -- 172.21.15.87:0/2064082309 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801e070 con 0x55fcee71ba90 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.089+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 0x7f4dc40a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc005c40 tx=0x7f4dcc004a40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4dbeffd640 1 -- 172.21.15.87:0/2064082309 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8019b10 con 0x55fcee71ba90 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4dbeffd640 1 -- 172.21.15.87:0/2064082309 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801e070 con 0x55fcee71ba90 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2064082309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 msgr2=0x55fcee725e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009d20 tx=0x7f4dc8011810 comp rx=0 tx=0).stop 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2064082309 shutdown_connections 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee726350 0x55fcee728740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee725e10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2064082309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2064082309 >> 172.21.15.87:0/2064082309 conn(0x55fcee717680 msgr2=0x55fcee717a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2064082309 shutdown_connections 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.090+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2064082309 wait complete. 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 -- start start 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 Processor -- start 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee7237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 0x55fcee723d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fcee728f80 con 0x55fcee71ba90 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fcee608ae0 con 0x55fcee71b170 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fcee729100 con 0x55fcee726350 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4ddd83b740 10 client.0 _ll_register_callbacks cb 0x55fcee6419b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee7237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd88c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee7237f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45926/0 (socket says 172.21.15.87:45926) 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd88c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56108/0 (socket says 172.21.15.87:56108) 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd3fff640 1 -- 172.21.15.87:0/3329756344 learned_addr learned my addr 172.21.15.87:0/3329756344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.091+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 0x55fcee723d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 -- 172.21.15.87:0/3329756344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 msgr2=0x55fcee723d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 0x55fcee723d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 -- 172.21.15.87:0/3329756344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 msgr2=0x55fcee7237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee7237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 -- 172.21.15.87:0/3329756344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcee393210 con 0x55fcee71b170 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 0x55fcee723d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4dc801c040 tx=0x7f4dc8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2408e8d2f9b7ee41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4db3fff640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:50:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8027020 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4dc8019630 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc8019b10 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dc8032050 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4ddd83b740 10 client.25367 fetch_fsmap learned FSMap version 27 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.092+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fcee45f620 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8019b10 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc8005050 con 0x55fcee71b170 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 10 client.25367 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 20 client.25367 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7363d0 0x55fcee7387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 4 client.25367 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcee692f10 con 0x55fcee7363d0 2024-06-06T07:50:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7363d0 0x55fcee7387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7363d0 0x55fcee7387c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2bb5ac209c9aa6 server_cookie=e5415c81b73ebf3a in_seq=0 out_seq=0 2024-06-06T07:50:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.093+0000 7f4db3fff640 10 client.25367 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.095+0000 7f4db3fff640 1 -- 172.21.15.87:0/3329756344 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fcee692f10 con 0x55fcee7363d0 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.095+0000 7f4db3fff640 10 client.25367 handle_command_reply: tid=0 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.095+0000 7f4ddd83b740 1 client.25367 shutdown 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.095+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 msgr2=0x55fcee7232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.095+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4dc801c040 tx=0x7f4dc8005dc0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 shutdown_connections 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7363d0 0x55fcee7387c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee726350 0x55fcee723d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee71ba90 0x55fcee7237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4db3fff640 0 client.25367 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/3329756344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71b170 0x55fcee7232b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 >> 172.21.15.87:0/3329756344 conn(0x55fcee717680 msgr2=0x55fcee71d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 shutdown_connections 2024-06-06T07:50:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 1 -- 172.21.15.87:0/3329756344 wait complete. 2024-06-06T07:50:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.096+0000 7f4ddd83b740 20 client.25367 trim_cache size 0 max 16384 2024-06-06T07:50:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 Processor -- start 2024-06-06T07:50:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 -- start start 2024-06-06T07:50:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee73ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41b3990 con 0x55fcee726350 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41b1260 con 0x55fcee71ba90 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41af0b0 con 0x55fcee71b170 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.111+0000 7f4dd90c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45934/0 (socket says 172.21.15.87:45934) 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 -- 172.21.15.87:0/1776865236 learned_addr learned my addr 172.21.15.87:0/1776865236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee73ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 -- 172.21.15.87:0/1776865236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 msgr2=0x55fcee71d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 -- 172.21.15.87:0/1776865236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 msgr2=0x55fcee73ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee73ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 -- 172.21.15.87:0/1776865236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcee721800 con 0x55fcee726350 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00f8e0 tx=0x7f4dcc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f0a4ab71d72975 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dbe7fc640 1 -- 172.21.15.87:0/1776865236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc003b70 con 0x55fcee726350 2024-06-06T07:50:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dbe7fc640 1 -- 172.21.15.87:0/1776865236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc003d10 con 0x55fcee726350 2024-06-06T07:50:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.112+0000 7f4dbe7fc640 1 -- 172.21.15.87:0/1776865236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc01b9f0 con 0x55fcee726350 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 -- 172.21.15.87:0/1776865236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 msgr2=0x55fcee73e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00f8e0 tx=0x7f4dcc019000 comp rx=0 tx=0).stop 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 -- 172.21.15.87:0/1776865236 shutdown_connections 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee73e410 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee73ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/1776865236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee71d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 -- 172.21.15.87:0/1776865236 >> 172.21.15.87:0/1776865236 conn(0x55fcee717680 msgr2=0x55fcee71cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 -- 172.21.15.87:0/1776865236 shutdown_connections 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.113+0000 7f4ddd83b740 1 -- 172.21.15.87:0/1776865236 wait complete. 2024-06-06T07:50:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 -- start start 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 Processor -- start 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee72e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee729b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41af720 con 0x55fcee726350 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41b0b90 con 0x55fcee71ba90 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41afdf0 con 0x55fcee71b170 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4ddd83b740 10 client.0 _ll_register_callbacks cb 0x7f4dc801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45940/0 (socket says 172.21.15.87:45940) 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 -- 172.21.15.87:0/2887261391 learned_addr learned my addr 172.21.15.87:0/2887261391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd3fff640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee729b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee72e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 -- 172.21.15.87:0/2887261391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 msgr2=0x55fcee72e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee72e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 -- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 msgr2=0x55fcee729b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.114+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee729b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4dd90c6640 1 -- 172.21.15.87:0/2887261391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcee72a600 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4dd90c6640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0049b0 tx=0x7f4dcc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2a98e8c690e2e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc036060 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4dcc004cc0 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dcc003d10 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dcc003ec0 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4ddd83b740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fcee7a0bc0 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc003d10 con 0x55fcee726350 2024-06-06T07:50:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.115+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dcc026070 con 0x55fcee726350 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 20 client.16014 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7a1260 0x55fcee7a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 4 client.16014 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcee68cc20 con 0x55fcee7a1260 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7a1260 0x55fcee7a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4db3fff640 10 client.16014 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.116+0000 7f4dd88c5640 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7a1260 0x55fcee7a3680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa164e8b94a6d27b server_cookie=304d20daee1c4526 in_seq=0 out_seq=0 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.332+0000 7f4db3fff640 1 -- 172.21.15.87:0/2887261391 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fcee68cc20 con 0x55fcee7a1260 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.332+0000 7f4db3fff640 10 client.16014 handle_command_reply: tid=0 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.332+0000 7f4ddd83b740 1 client.16014 shutdown 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 msgr2=0x55fcee72a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0049b0 tx=0x7f4dcc003c80 comp rx=0 tx=0).stop 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 shutdown_connections 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fcee7a1260 0x55fcee7a3680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fcee726350 0x55fcee72a0c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcee71ba90 0x55fcee729b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4db3fff640 0 client.16014 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 --2- 172.21.15.87:0/2887261391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fcee71b170 0x55fcee72e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 >> 172.21.15.87:0/2887261391 conn(0x55fcee717680 msgr2=0x55fcee71d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 shutdown_connections 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 1 -- 172.21.15.87:0/2887261391 wait complete. 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.333+0000 7f4ddd83b740 20 client.16014 trim_cache size 0 max 16384 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:50:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 msgr2=0x7f4dc40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 0x7f4dc40a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc005c40 tx=0x7f4dcc004a40 comp rx=0 tx=0).stop 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 msgr2=0x7f4dd419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc0004040 comp rx=0 tx=0).stop 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 shutdown_connections 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dc40a2e20 0x7f4dc40a52e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd410e1c0 0x7f4dd419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dd40746f0 0x7f4dd419a9a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 --2- 172.21.15.87:0/4226684512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dd4073dd0 0x7f4dd419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 >> 172.21.15.87:0/4226684512 conn(0x7f4dd40fd450 msgr2=0x7f4dd40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 shutdown_connections 2024-06-06T07:50:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:01.334+0000 7f4ddab29640 1 -- 172.21.15.87:0/4226684512 wait complete. 2024-06-06T07:50:01.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 12059 16.67 MB/sec warmup 5 sec latency 81.373 ms 2024-06-06T07:50:01.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 -- 172.21.15.87:0/2758544397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 msgr2=0x7fd5881088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/2758544397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd5881088f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd584009990 tx=0x7fd584009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 -- 172.21.15.87:0/2758544397 shutdown_connections 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/2758544397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588101440 0x7fd58810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/2758544397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd5881088f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/2758544397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588100a80 0x7fd588100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.685+0000 7fd58f8c3700 1 -- 172.21.15.87:0/2758544397 >> 172.21.15.87:0/2758544397 conn(0x7fd588077a00 msgr2=0x7fd5880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- 172.21.15.87:0/2758544397 shutdown_connections 2024-06-06T07:50:01.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- 172.21.15.87:0/2758544397 wait complete. 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 Processor -- start 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- start start 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 0x7fd58819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd588198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd588112790 con 0x7fd588108510 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd588112610 con 0x7fd588100a80 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.686+0000 7fd58f8c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd588112910 con 0x7fd588101440 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 0x7fd58819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58de60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd588198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56154/0 (socket says 172.21.15.87:56154) 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58d65f700 1 -- 172.21.15.87:0/731939327 learned_addr learned my addr 172.21.15.87:0/731939327 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 -- 172.21.15.87:0/731939327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 msgr2=0x7fd58819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 -- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 msgr2=0x7fd588198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd588198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 -- 172.21.15.87:0/731939327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd588199190 con 0x7fd588101440 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58d65f700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58de60700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd588198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 0x7fd58819edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e9f0 tx=0x7fd57800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd57e7fc700 1 -- 172.21.15.87:0/731939327 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578004220 con 0x7fd588101440 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd588199480 con 0x7fd588101440 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd57e7fc700 1 -- 172.21.15.87:0/731939327 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578004d10 con 0x7fd588101440 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd57e7fc700 1 -- 172.21.15.87:0/731939327 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578005020 con 0x7fd588101440 2024-06-06T07:50:01.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.687+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a5740 con 0x7fd588101440 2024-06-06T07:50:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.690+0000 7fd57e7fc700 1 -- 172.21.15.87:0/731939327 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd578004750 con 0x7fd588101440 2024-06-06T07:50:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.690+0000 7fd57e7fc700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 0x7fd5740a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.690+0000 7fd58d65f700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 0x7fd5740a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.690+0000 7fd57e7fc700 1 -- 172.21.15.87:0/731939327 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd5780cf730 con 0x7fd588101440 2024-06-06T07:50:01.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd58d65f700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 0x7fd5740a5250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd584009750 tx=0x7fd584005e80 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 Processor -- start 2024-06-06T07:50:01.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 -- start start 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740bfe0 0x56451740c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517406c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5645172f8fe0 con 0x56451740c900 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5645173578d0 con 0x564517406820 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.691+0000 7fd59a146b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645172bf4e0 con 0x56451740bfe0 2024-06-06T07:50:01.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45958/0 (socket says 172.21.15.87:45958) 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 -- 172.21.15.87:0/2037989368 learned_addr learned my addr 172.21.15.87:0/2037989368 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58d65f700 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740bfe0 0x56451740c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58de60700 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517406c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 -- 172.21.15.87:0/2037989368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740bfe0 msgr2=0x56451740c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740bfe0 0x56451740c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 -- 172.21.15.87:0/2037989368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 msgr2=0x564517406c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517406c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 -- 172.21.15.87:0/2037989368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645172a6950 con 0x56451740c900 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd5780981f0 tx=0x7fd57809b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c4c16e4fdde9ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd5737fe700 1 -- 172.21.15.87:0/2037989368 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57801c040 con 0x56451740c900 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd5737fe700 1 -- 172.21.15.87:0/2037989368 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57809b820 con 0x56451740c900 2024-06-06T07:50:01.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.692+0000 7fd5737fe700 1 -- 172.21.15.87:0/2037989368 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578086070 con 0x56451740c900 2024-06-06T07:50:01.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 -- 172.21.15.87:0/2037989368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 msgr2=0x5645174062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd5780981f0 tx=0x7fd57809b180 comp rx=0 tx=0).stop 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 -- 172.21.15.87:0/2037989368 shutdown_connections 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517406c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740bfe0 0x56451740c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 --2- 172.21.15.87:0/2037989368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x5645174062e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 -- 172.21.15.87:0/2037989368 >> 172.21.15.87:0/2037989368 conn(0x564517333100 msgr2=0x5645174012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 -- 172.21.15.87:0/2037989368 shutdown_connections 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.693+0000 7fd59a146b80 1 -- 172.21.15.87:0/2037989368 wait complete. 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.694+0000 7fd59a146b80 1 -- start start 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.694+0000 7fd59a146b80 1 Processor -- start 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 0x56451740f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x56451740fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5645172f8040 con 0x56451740c900 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564517353890 con 0x56451740bfe0 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645174132e0 con 0x564517406820 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd59a146b80 10 client.0 _ll_register_callbacks cb 0x5645172ed630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 0x56451740f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:55080/0 (socket says 172.21.15.87:55080) 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 -- 172.21.15.87:0/3278094051 learned_addr learned my addr 172.21.15.87:0/3278094051 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58de60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x56451740fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 -- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 msgr2=0x56451740f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 0x56451740f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 -- 172.21.15.87:0/3278094051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 msgr2=0x56451740fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x56451740fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 -- 172.21.15.87:0/3278094051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451731d7d0 con 0x564517406820 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58de60700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x56451740fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 0x56451740f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd58d65f700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd588199b70 tx=0x7fd584020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78426f3ec22328d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.695+0000 7fd570ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584005220 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd5840053c0 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57809b820 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5840055c0 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd59a146b80 10 client.25465 fetch_fsmap learned FSMap version 28 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645174262a0 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.696+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57809b820 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fd58400fa30 con 0x564517406820 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 10 client.25465 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 20 client.25465 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5645174288e0 0x56451742acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 4 client.25465 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564517364dd0 con 0x5645174288e0 2024-06-06T07:50:01.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.697+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5645174288e0 0x56451742acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.698+0000 7fd570ff9700 10 client.25465 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.698+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5645174288e0 0x56451742acf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1be927fc6104a74 server_cookie=e537d298d2eaa6d6 in_seq=0 out_seq=0 2024-06-06T07:50:01.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.698+0000 7fd570ff9700 1 -- 172.21.15.87:0/3278094051 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564517364dd0 con 0x5645174288e0 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.698+0000 7fd570ff9700 10 client.25465 handle_command_reply: tid=0 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 client.25465 shutdown 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 msgr2=0x564517412690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd588199b70 tx=0x7fd584020000 comp rx=0 tx=0).stop 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 shutdown_connections 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5645174288e0 0x56451742acf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740c900 0x56451740fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x564517412690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd570ff9700 0 client.25465 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 --2- 172.21.15.87:0/3278094051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740bfe0 0x56451740f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 >> 172.21.15.87:0/3278094051 conn(0x564517333100 msgr2=0x564517400780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 shutdown_connections 2024-06-06T07:50:01.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 1 -- 172.21.15.87:0/3278094051 wait complete. 2024-06-06T07:50:01.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.699+0000 7fd59a146b80 20 client.25465 trim_cache size 0 max 16384 2024-06-06T07:50:01.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 Processor -- start 2024-06-06T07:50:01.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 -- start start 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740c900 0x564517401440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881aece0 con 0x56451740bfe0 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881ac570 con 0x56451740c900 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd59a146b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881aa3c0 con 0x564517406820 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45972/0 (socket says 172.21.15.87:45972) 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd58d65f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:55092/0 (socket says 172.21.15.87:55092) 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.720+0000 7fd58ce5e700 1 -- 172.21.15.87:0/697203583 learned_addr learned my addr 172.21.15.87:0/697203583 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58de60700 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740c900 0x564517401440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 -- 172.21.15.87:0/697203583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 msgr2=0x5645174121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 -- 172.21.15.87:0/697203583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740c900 msgr2=0x564517401440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740c900 0x564517401440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 -- 172.21.15.87:0/697203583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645174108d0 con 0x56451740bfe0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58d65f700 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd578009230 tx=0x7fd5780045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8aaf3f61c44d5d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd572ffd700 1 -- 172.21.15.87:0/697203583 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578065070 con 0x56451740bfe0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd572ffd700 1 -- 172.21.15.87:0/697203583 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57809b820 con 0x56451740bfe0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.721+0000 7fd572ffd700 1 -- 172.21.15.87:0/697203583 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57801c040 con 0x56451740bfe0 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- 172.21.15.87:0/697203583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 msgr2=0x564517400f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd578009230 tx=0x7fd5780045d0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- 172.21.15.87:0/697203583 shutdown_connections 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56451740c900 0x564517401440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564517406820 0x5645174121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 --2- 172.21.15.87:0/697203583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517400f00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- 172.21.15.87:0/697203583 >> 172.21.15.87:0/697203583 conn(0x564517333100 msgr2=0x56451740f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- 172.21.15.87:0/697203583 shutdown_connections 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- 172.21.15.87:0/697203583 wait complete. 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.722+0000 7fd59a146b80 1 -- start start 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 Processor -- start 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517420fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740c900 0x56451741ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881aaa30 con 0x56451740bfe0 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881abea0 con 0x564517406820 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881ab100 con 0x56451740c900 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd59a146b80 10 client.0 _ll_register_callbacks cb 0x5645172ed630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd58ce5e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45978/0 (socket says 172.21.15.87:45978) 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd58ce5e700 1 -- 172.21.15.87:0/981477227 learned_addr learned my addr 172.21.15.87:0/981477227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd58d65f700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517420fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.723+0000 7fd58de60700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740c900 0x56451741ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 -- 172.21.15.87:0/981477227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740c900 msgr2=0x56451741ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740c900 0x56451741ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 -- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 msgr2=0x564517420fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517420fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 -- 172.21.15.87:0/981477227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451741e4a0 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd58ce5e700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e9c0 tx=0x7fd5780ccfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cf74b724b30d72 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578065030 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd5780b4040 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57809b820 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd578082050 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd59a146b80 10 client.16032 fetch_fsmap learned FSMap version 28 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564517493fa0 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57809b820 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.724+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fd5780cf2b0 con 0x56451740bfe0 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:01.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 20 client.16032 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564517493fa0 0x564517496390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 4 client.16032 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56451735f350 con 0x564517493fa0 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd58d65f700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564517493fa0 0x564517496390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd58d65f700 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564517493fa0 0x564517496390 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19adc1ead675aaec server_cookie=5044961b4ca4cf62 in_seq=0 out_seq=0 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.725+0000 7fd570ff9700 10 client.16032 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd570ff9700 1 -- 172.21.15.87:0/981477227 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56451735f350 con 0x564517493fa0 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd570ff9700 10 client.16032 handle_command_reply: tid=0 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 client.16032 shutdown 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 msgr2=0x564517421520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e9c0 tx=0x7fd5780ccfb0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 shutdown_connections 2024-06-06T07:50:01.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564517493fa0 0x564517496390 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd570ff9700 0 client.16032 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564517406820 0x564517420fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56451740c900 0x56451741ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 --2- 172.21.15.87:0/981477227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56451740bfe0 0x564517421520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 >> 172.21.15.87:0/981477227 conn(0x564517333100 msgr2=0x56451740fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.726+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 shutdown_connections 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd59a146b80 1 -- 172.21.15.87:0/981477227 wait complete. 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd59a146b80 20 client.16032 trim_cache size 0 max 16384 2024-06-06T07:50:01.734 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:50:01.735 INFO:teuthology.orchestra.run.smithi087.stdout: "return_code": 0, 2024-06-06T07:50:01.735 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_tag": "f71b208e-547d-452a-941c-5cbe3fcfcf79", 2024-06-06T07:50:01.735 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": "asynchronous" 2024-06-06T07:50:01.735 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 msgr2=0x7fd5740a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 0x7fd5740a5250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd584009750 tx=0x7fd584005e80 comp rx=0 tx=0).stop 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 msgr2=0x7fd58819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 0x7fd58819edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e9f0 tx=0x7fd57800ed00 comp rx=0 tx=0).stop 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 shutdown_connections 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588100a80 0x7fd58819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd5740a2d90 0x7fd5740a5250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd588108510 0x7fd588198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 --2- 172.21.15.87:0/731939327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd588101440 0x7fd58819edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 >> 172.21.15.87:0/731939327 conn(0x7fd588077a00 msgr2=0x7fd5880ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.727+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 shutdown_connections 2024-06-06T07:50:01.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:01.728+0000 7fd58f8c3700 1 -- 172.21.15.87:0/731939327 wait complete. 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.736+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d9782b3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.736+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.736+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.737+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.737+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.737+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.737+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d97836f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.057 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.737+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d978370680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.058 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.738+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d978370c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.058 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.738+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d978371180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.058 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 07:50:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T07:50:01.738+0000 7feb35688700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [3,head] /volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24812=0-4194304@2} caps={24812=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=24812 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d978370100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v2310>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v32>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v49>,<0x1/volumes v46>]//[3]","memoryvalue":"(4)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v2108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v25>,<0x10000000006/clients v15>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v18>,<0x100000001fa/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542 v24>,<0x10000000001/sv_1 v44>,<0x10000000000/qa v37>,<0x1/volumes v34>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: scrub summary: active paths [/] 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:02 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: scrub summary: active paths [/] 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:02 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:02.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T07:50:02.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:02.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: scrub summary: active paths [/] 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-06-06T07:50:02.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:02 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:02.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:50:02.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14269 15.99 MB/sec warmup 6 sec latency 399.923 ms 2024-06-06T07:50:02.886 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:03.291 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:03 smithi087 ceph-mon[32276]: scrub queued for path: / 2024-06-06T07:50:03.292 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:03 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.292 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:03 smithi087 ceph-mon[32276]: scrub summary: active paths [/] 2024-06-06T07:50:03.292 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:03 smithi087 ceph-mon[32276]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:03 smithi195 ceph-mon[37040]: scrub queued for path: / 2024-06-06T07:50:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:03 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:03 smithi195 ceph-mon[37040]: scrub summary: active paths [/] 2024-06-06T07:50:03.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:03 smithi195 ceph-mon[37040]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:03 smithi181 ceph-mon[36790]: scrub queued for path: / 2024-06-06T07:50:03.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:03 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:03 smithi181 ceph-mon[36790]: scrub summary: active paths [/] 2024-06-06T07:50:03.559 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:03 smithi181 ceph-mon[36790]: scrub summary: idle+waiting paths [/] 2024-06-06T07:50:03.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 13.71 MB/sec warmup 7 sec latency 814.119 ms 2024-06-06T07:50:04.272 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:04 smithi087 ceph-mon[32276]: pgmap v64: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 724 KiB/s wr, 212 op/s 2024-06-06T07:50:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:04 smithi195 ceph-mon[37040]: pgmap v64: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 724 KiB/s wr, 212 op/s 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 -- 172.21.15.87:0/3090262543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101380 msgr2=0x7fedec101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 --2- 172.21.15.87:0/3090262543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101380 0x7fedec101800 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fede0009990 tx=0x7fede0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 -- 172.21.15.87:0/3090262543 shutdown_connections 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 --2- 172.21.15.87:0/3090262543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec1073f0 0x7fedec1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 --2- 172.21.15.87:0/3090262543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec101d40 0x7fedec10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 --2- 172.21.15.87:0/3090262543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101380 0x7fedec101800 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.528+0000 7fedf253a700 1 -- 172.21.15.87:0/3090262543 >> 172.21.15.87:0/3090262543 conn(0x7fedec078300 msgr2=0x7fedec078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.529+0000 7fedf253a700 1 -- 172.21.15.87:0/3090262543 shutdown_connections 2024-06-06T07:50:04.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.529+0000 7fedf253a700 1 -- 172.21.15.87:0/3090262543 wait complete. 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.529+0000 7fedf253a700 1 Processor -- start 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.529+0000 7fedf253a700 1 -- start start 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec101380 0x7fedec19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec1073f0 0x7fedec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedec111690 con 0x7fedec101d40 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedec111510 con 0x7fedec101380 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf253a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedec111810 con 0x7fedec1073f0 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45998/0 (socket says 172.21.15.87:45998) 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 -- 172.21.15.87:0/2489146546 learned_addr learned my addr 172.21.15.87:0/2489146546 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:04.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedebfff700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec101380 0x7fedec19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec1073f0 0x7fedec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 -- 172.21.15.87:0/2489146546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec1073f0 msgr2=0x7fedec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec1073f0 0x7fedec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 -- 172.21.15.87:0/2489146546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec101380 msgr2=0x7fedec19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec101380 0x7fedec19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.530+0000 7fedeb7fe700 1 -- 172.21.15.87:0/2489146546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1990e0 con 0x7fedec101d40 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fede0000c00 tx=0x7fede00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fede97fa700 1 -- 172.21.15.87:0/2489146546 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0027070 con 0x7fedec101d40 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec199370 con 0x7fedec101d40 2024-06-06T07:50:04.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fede97fa700 1 -- 172.21.15.87:0/2489146546 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0003e40 con 0x7fedec101d40 2024-06-06T07:50:04.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fede97fa700 1 -- 172.21.15.87:0/2489146546 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0022d10 con 0x7fedec101d40 2024-06-06T07:50:04.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.531+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a5690 con 0x7fedec101d40 2024-06-06T07:50:04.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.533+0000 7fede97fa700 1 -- 172.21.15.87:0/2489146546 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fede002a070 con 0x7fedec101d40 2024-06-06T07:50:04.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.534+0000 7fede97fa700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 0x7fedd40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.534+0000 7fedebfff700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 0x7fedd40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.534+0000 7fede97fa700 1 -- 172.21.15.87:0/2489146546 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fede00d0490 con 0x7fedec101d40 2024-06-06T07:50:04.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.534+0000 7fedebfff700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 0x7fedd40a5130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fedec199ac0 tx=0x7feddc00a380 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 Processor -- start 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 -- start start 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc84880 0x55c07cc7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c07cb70f60 con 0x55c07cc83f60 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c07cbcf850 con 0x55c07cc7e7a0 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c07cb37460 con 0x55c07cc84880 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46004/0 (socket says 172.21.15.87:46004) 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.535+0000 7fedebfff700 1 -- 172.21.15.87:0/252634900 learned_addr learned my addr 172.21.15.87:0/252634900 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc84880 0x55c07cc7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 -- 172.21.15.87:0/252634900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc84880 msgr2=0x55c07cc7e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc84880 0x55c07cc7e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 -- 172.21.15.87:0/252634900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 msgr2=0x55c07cc7ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 -- 172.21.15.87:0/252634900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07cb1e8d0 con 0x55c07cc83f60 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedebfff700 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feddc004530 tx=0x7feddc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00a58270e1d2838 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedd27fc700 1 -- 172.21.15.87:0/252634900 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc016070 con 0x55c07cc83f60 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedd27fc700 1 -- 172.21.15.87:0/252634900 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc014070 con 0x55c07cc83f60 2024-06-06T07:50:04.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.536+0000 7fedd27fc700 1 -- 172.21.15.87:0/252634900 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc010e10 con 0x55c07cc83f60 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/252634900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 msgr2=0x55c07cc84340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feddc004530 tx=0x7feddc012000 comp rx=0 tx=0).stop 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/252634900 shutdown_connections 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc83f60 0x55c07cc84340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/252634900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc84880 0x55c07cc7e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/252634900 >> 172.21.15.87:0/252634900 conn(0x55c07cbab080 msgr2=0x55c07cc79220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/252634900 shutdown_connections 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/252634900 wait complete. 2024-06-06T07:50:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.537+0000 7fedfcdbdb80 1 -- start start 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 Processor -- start 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc84880 0x55c07cc87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c07cb6ffc0 con 0x55c07cc7e7a0 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c07cbcb810 con 0x55c07cc84880 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c07cc8b260 con 0x55c07cc83f60 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedfcdbdb80 10 client.0 _ll_register_callbacks cb 0x55c07cb655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46020/0 (socket says 172.21.15.87:46020) 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 -- 172.21.15.87:0/551054350 learned_addr learned my addr 172.21.15.87:0/551054350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc84880 0x55c07cc87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 -- 172.21.15.87:0/551054350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 msgr2=0x55c07cc87860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc87860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 -- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc84880 msgr2=0x55c07cc87da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc84880 0x55c07cc87da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 -- 172.21.15.87:0/551054350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07cb95750 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedebfff700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feddc00aa40 tx=0x7feddc010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77440a764b076c56 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.538+0000 7fedcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc034060 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7feddc010ad0 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc014070 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feddc03b050 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 10 client.16044 fetch_fsmap learned FSMap version 28 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c07cc959b0 con 0x55c07cc7e7a0 2024-06-06T07:50:04.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc014070 con 0x55c07cc7e7a0 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7feddc01d930 con 0x55c07cc7e7a0 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 20 client.16044 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cc97f30 0x55c07cc9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 4 client.16044 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c07cbdcd50 con 0x55c07cc97f30 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.539+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cc97f30 0x55c07cc9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.540+0000 7fedcb7fe700 10 client.16044 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.540+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cc97f30 0x55c07cc9a320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80d182bc72aff3c server_cookie=468c6e8dc12bac5d in_seq=0 out_seq=0 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedcb7fe700 1 -- 172.21.15.87:0/551054350 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c07cbdcd50 con 0x55c07cc97f30 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedcb7fe700 10 client.16044 handle_command_reply: tid=0 2024-06-06T07:50:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 client.16044 shutdown 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 msgr2=0x55c07cc8a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feddc00aa40 tx=0x7feddc010810 comp rx=0 tx=0).stop 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 shutdown_connections 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cc97f30 0x55c07cc9a320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc84880 0x55c07cc87da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc7e7a0 0x55c07cc8a560 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedcb7fe700 0 client.16044 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/551054350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 >> 172.21.15.87:0/551054350 conn(0x55c07cbab080 msgr2=0x55c07cc78700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 shutdown_connections 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/551054350 wait complete. 2024-06-06T07:50:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.541+0000 7fedfcdbdb80 20 client.16044 trim_cache size 0 max 16384 2024-06-06T07:50:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:04 smithi181 ceph-mon[36790]: pgmap v64: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 724 KiB/s wr, 212 op/s 2024-06-06T07:50:04.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.561+0000 7fedfcdbdb80 1 Processor -- start 2024-06-06T07:50:04.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.561+0000 7fedfcdbdb80 1 -- start start 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc86a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b0af0 con 0x55c07cc84880 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ae380 con 0x55c07cc7e7a0 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ac1d0 con 0x55c07cc83f60 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46036/0 (socket says 172.21.15.87:46036) 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 -- 172.21.15.87:0/2535515382 learned_addr learned my addr 172.21.15.87:0/2535515382 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedebfff700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc86a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 -- 172.21.15.87:0/2535515382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 msgr2=0x55c07cc86a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc86a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 -- 172.21.15.87:0/2535515382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 msgr2=0x55c07cc7e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 -- 172.21.15.87:0/2535515382 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07cc85df0 con 0x55c07cc84880 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedebfff700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc86a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fede4009f00 tx=0x7fede4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8a14eed4a2c426 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedd1ffb700 1 -- 172.21.15.87:0/2535515382 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4020420 con 0x55c07cc84880 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.562+0000 7fedd1ffb700 1 -- 172.21.15.87:0/2535515382 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede401b030 con 0x55c07cc84880 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedd1ffb700 1 -- 172.21.15.87:0/2535515382 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede40167f0 con 0x55c07cc84880 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/2535515382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 msgr2=0x55c07cc897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fede4009f00 tx=0x7fede4009f30 comp rx=0 tx=0).stop 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/2535515382 shutdown_connections 2024-06-06T07:50:04.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc7e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cc897a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/2535515382 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc86a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/2535515382 >> 172.21.15.87:0/2535515382 conn(0x55c07cbab080 msgr2=0x55c07cc89090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/2535515382 shutdown_connections 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.563+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/2535515382 wait complete. 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 -- start start 2024-06-06T07:50:04.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 Processor -- start 2024-06-06T07:50:04.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc9f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc9fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ac840 con 0x55c07cc84880 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1adcb0 con 0x55c07cc7e7a0 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1acf10 con 0x55c07cc83f60 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedfcdbdb80 10 client.0 _ll_register_callbacks cb 0x55c07cb655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedf0ad7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedf0ad7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46042/0 (socket says 172.21.15.87:46042) 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedf0ad7700 1 -- 172.21.15.87:0/1316280399 learned_addr learned my addr 172.21.15.87:0/1316280399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedebfff700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.564+0000 7fedeb7fe700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc9fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 -- 172.21.15.87:0/1316280399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 msgr2=0x55c07cc9fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc9fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 -- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 msgr2=0x55c07cc9f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:04.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc9f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 -- 172.21.15.87:0/1316280399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07cd01ab0 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedf0ad7700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fede4009690 tx=0x7fede40178c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73c599f59525658 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4009b80 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fede4004ad0 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede401b030 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fede4004cd0 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedfcdbdb80 10 client.16050 fetch_fsmap learned FSMap version 28 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c07cd02d30 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede401b030 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.565+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fede4020490 con 0x55c07cc84880 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:04.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 20 client.16050 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cd03380 0x55c07cd057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 4 client.16050 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c07cc85880 con 0x55c07cd03380 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedebfff700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cd03380 0x55c07cd057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedcb7fe700 10 client.16050 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedebfff700 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cd03380 0x55c07cd057a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73067dda1248ad3f server_cookie=f5353092993ecf81 in_seq=0 out_seq=0 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedcb7fe700 1 -- 172.21.15.87:0/1316280399 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c07cc85880 con 0x55c07cd03380 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedcb7fe700 10 client.16050 handle_command_reply: tid=0 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 1 client.16050 shutdown 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 msgr2=0x55c07cca0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.566+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fede4009690 tx=0x7fede40178c0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 shutdown_connections 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c07cd03380 0x55c07cd057a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c07cc7e7a0 0x55c07cc9f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c07cc84880 0x55c07cca0180 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 --2- 172.21.15.87:0/1316280399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c07cc83f60 0x55c07cc9fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedcb7fe700 0 client.16050 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 >> 172.21.15.87:0/1316280399 conn(0x55c07cbab080 msgr2=0x55c07cc89920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 shutdown_connections 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 1 -- 172.21.15.87:0/1316280399 wait complete. 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedfcdbdb80 20 client.16050 trim_cache size 0 max 16384 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T07:50:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": { 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "f71b208e-547d-452a-941c-5cbe3fcfcf79": { 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/", 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "tag": "f71b208e-547d-452a-941c-5cbe3fcfcf79", 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "options": "recursive,force" 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:50:04.574 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 msgr2=0x7fedd40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 0x7fedd40a5130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fedec199ac0 tx=0x7feddc00a380 comp rx=0 tx=0).stop 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 msgr2=0x7fedec19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.567+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fede0000c00 tx=0x7fede00047b0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 shutdown_connections 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedec101380 0x7fedec19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedd40a2c70 0x7fedd40a5130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedec1073f0 0x7fedec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 --2- 172.21.15.87:0/2489146546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedec101d40 0x7fedec19ed00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:04.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 >> 172.21.15.87:0/2489146546 conn(0x7fedec078300 msgr2=0x7fedec100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:04.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 shutdown_connections 2024-06-06T07:50:04.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:04.568+0000 7fedf253a700 1 -- 172.21.15.87:0/2489146546 wait complete. 2024-06-06T07:50:04.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 12.00 MB/sec warmup 8 sec latency 1814.173 ms 2024-06-06T07:50:05.425 INFO:tasks.cephfs.filesystem:scrub status for tag:f71b208e-547d-452a-941c-5cbe3fcfcf79 - {'path': '/', 'tag': 'f71b208e-547d-452a-941c-5cbe3fcfcf79', 'options': 'recursive,force'} 2024-06-06T07:50:05.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 10.67 MB/sec warmup 9 sec latency 2814.250 ms 2024-06-06T07:50:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:06 smithi195 ceph-mon[37040]: pgmap v65: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 723 KiB/s wr, 209 op/s 2024-06-06T07:50:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:06 smithi181 ceph-mon[36790]: pgmap v65: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 723 KiB/s wr, 209 op/s 2024-06-06T07:50:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:06 smithi087 ceph-mon[32276]: pgmap v65: 129 pgs: 129 active+clean; 8.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 723 KiB/s wr, 209 op/s 2024-06-06T07:50:06.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 9.60 MB/sec warmup 10 sec latency 3814.347 ms 2024-06-06T07:50:07.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:07 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:07.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:07 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:07 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T07:50:07.215+0000 7f61096e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:50:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:07 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:07.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 14285 8.73 MB/sec warmup 11 sec latency 4814.430 ms 2024-06-06T07:50:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:08 smithi195 ceph-mon[37040]: pgmap v66: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 854 KiB/s wr, 239 op/s 2024-06-06T07:50:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:08 smithi195 ceph-mon[37040]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:50:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:08 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:50:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:08 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:08 smithi181 ceph-mon[36790]: pgmap v66: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 854 KiB/s wr, 239 op/s 2024-06-06T07:50:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:08 smithi181 ceph-mon[36790]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:50:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:08 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:50:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:08 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:08 smithi087 ceph-mon[32276]: pgmap v66: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 854 KiB/s wr, 239 op/s 2024-06-06T07:50:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:08 smithi087 ceph-mon[32276]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:50:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:08 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] up:active 2024-06-06T07:50:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:08 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:50:08.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 15786 8.55 MB/sec warmup 12 sec latency 4993.674 ms 2024-06-06T07:50:09.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17812 8.85 MB/sec warmup 13 sec latency 497.625 ms 2024-06-06T07:50:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:10 smithi195 ceph-mon[37040]: pgmap v67: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:10 smithi181 ceph-mon[36790]: pgmap v67: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:10 smithi087 ceph-mon[32276]: pgmap v67: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:10.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17812 8.22 MB/sec warmup 14 sec latency 1497.723 ms 2024-06-06T07:50:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:11 smithi195 ceph-mon[37040]: scrub summary: idle 2024-06-06T07:50:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:11 smithi195 ceph-mon[37040]: scrub summary: idle 2024-06-06T07:50:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:11 smithi181 ceph-mon[36790]: scrub summary: idle 2024-06-06T07:50:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:11 smithi181 ceph-mon[36790]: scrub summary: idle 2024-06-06T07:50:11.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:11 smithi087 ceph-mon[32276]: scrub summary: idle 2024-06-06T07:50:11.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:11 smithi087 ceph-mon[32276]: scrub summary: idle 2024-06-06T07:50:11.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17812 7.67 MB/sec warmup 15 sec latency 2497.818 ms 2024-06-06T07:50:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:12 smithi195 ceph-mon[37040]: pgmap v68: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:12 smithi181 ceph-mon[36790]: pgmap v68: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:12 smithi087 ceph-mon[32276]: pgmap v68: 129 pgs: 129 active+clean; 9.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 854 KiB/s wr, 236 op/s 2024-06-06T07:50:12.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17812 7.19 MB/sec warmup 16 sec latency 3497.914 ms 2024-06-06T07:50:13.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17879 6.78 MB/sec warmup 17 sec latency 3675.575 ms 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 -- 172.21.15.87:0/1708499013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef88100110 msgr2=0x7fef881004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1708499013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef88100110 0x7fef881004f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009900 tx=0x7fef7c02a410 comp rx=0 tx=0).stop 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 -- 172.21.15.87:0/1708499013 shutdown_connections 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1708499013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88106cd0 0x7fef8810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1708499013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88100ac0 0x7fef881065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1708499013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef88100110 0x7fef881004f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 -- 172.21.15.87:0/1708499013 >> 172.21.15.87:0/1708499013 conn(0x7fef88076e30 msgr2=0x7fef880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 -- 172.21.15.87:0/1708499013 shutdown_connections 2024-06-06T07:50:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.433+0000 7fef8efa2640 1 -- 172.21.15.87:0/1708499013 wait complete. 2024-06-06T07:50:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.434+0000 7fef8efa2640 1 Processor -- start 2024-06-06T07:50:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.434+0000 7fef8efa2640 1 -- start start 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88100110 0x7fef88198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 0x7fef881987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef88111d80 con 0x7fef8819cb20 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef88111c00 con 0x7fef88100110 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8efa2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef88111f00 con 0x7fef88106cd0 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56260/0 (socket says 172.21.15.87:56260) 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 -- 172.21.15.87:0/1308433350 learned_addr learned my addr 172.21.15.87:0/1308433350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88100110 0x7fef88198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef87fff640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 0x7fef881987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 -- 172.21.15.87:0/1308433350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 msgr2=0x7fef881987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 0x7fef881987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 -- 172.21.15.87:0/1308433350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88100110 msgr2=0x7fef88198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88100110 0x7fef88198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 -- 172.21.15.87:0/1308433350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8819d610 con 0x7fef8819cb20 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef87fff640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 0x7fef881987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef8d53f640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fef8000cad0 tx=0x7fef8000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.435+0000 7fef85ffb640 1 -- 172.21.15.87:0/1308433350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8000ae20 con 0x7fef8819cb20 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.436+0000 7fef85ffb640 1 -- 172.21.15.87:0/1308433350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef80005020 con 0x7fef8819cb20 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.436+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8819d900 con 0x7fef8819cb20 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.436+0000 7fef85ffb640 1 -- 172.21.15.87:0/1308433350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8000f6f0 con 0x7fef8819cb20 2024-06-06T07:50:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.436+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef881a50a0 con 0x7fef8819cb20 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.438+0000 7fef85ffb640 1 -- 172.21.15.87:0/1308433350 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fef800051d0 con 0x7fef8819cb20 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 Processor -- start 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 -- start start 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 0x564a92cea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92cf5350 0x564a92cf7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a92c4e4f0 con 0x564a92ceaa90 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a92bec870 con 0x564a92cf5350 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef91d93740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a92c4a4b0 con 0x564a92cea170 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56266/0 (socket says 172.21.15.87:56266) 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 -- 172.21.15.87:0/3468680318 learned_addr learned my addr 172.21.15.87:0/3468680318 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef87fff640 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 0x564a92cea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef85ffb640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 0x7fef780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 -- 172.21.15.87:0/3468680318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 msgr2=0x564a92cea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 0x564a92cea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 -- 172.21.15.87:0/3468680318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92cf5350 msgr2=0x564a92cf7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92cf5350 0x564a92cf7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef8cd3e640 1 -- 172.21.15.87:0/3468680318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a92c089b0 con 0x564a92ceaa90 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.439+0000 7fef87fff640 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 0x564a92cea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef8d53f640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 0x7fef780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009900 tx=0x7fef7c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd820f38f59e5f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef85ffb640 1 -- 172.21.15.87:0/1308433350 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fef800cf870 con 0x7fef8819cb20 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef72ffd640 1 -- 172.21.15.87:0/3468680318 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c040070 con 0x564a92ceaa90 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef72ffd640 1 -- 172.21.15.87:0/3468680318 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c0319f0 con 0x564a92ceaa90 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.440+0000 7fef72ffd640 1 -- 172.21.15.87:0/3468680318 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c035040 con 0x564a92ceaa90 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- 172.21.15.87:0/3468680318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 msgr2=0x564a92cf4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009900 tx=0x7fef7c005e80 comp rx=0 tx=0).stop 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef8d53f640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 0x7fef780a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef800cc200 tx=0x7fef800cc190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- 172.21.15.87:0/3468680318 shutdown_connections 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92cf5350 0x564a92cf7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92ceaa90 0x564a92cf4e10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 --2- 172.21.15.87:0/3468680318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cea170 0x564a92cea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- 172.21.15.87:0/3468680318 >> 172.21.15.87:0/3468680318 conn(0x564a92ce6680 msgr2=0x564a92ce6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- 172.21.15.87:0/3468680318 shutdown_connections 2024-06-06T07:50:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- 172.21.15.87:0/3468680318 wait complete. 2024-06-06T07:50:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 -- start start 2024-06-06T07:50:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.441+0000 7fef91d93740 1 Processor -- start 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a92cf7f20 con 0x564a92cea170 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a92bd7ae0 con 0x564a92ceaa90 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a92cf80a0 con 0x564a92cf5350 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 10 client.0 _ll_register_callbacks cb 0x564a92c109b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56268/0 (socket says 172.21.15.87:56268) 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 -- 172.21.15.87:0/1345151584 learned_addr learned my addr 172.21.15.87:0/1345151584 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 -- 172.21.15.87:0/1345151584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 msgr2=0x564a92cf2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 -- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 msgr2=0x564a92cf2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cf2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 -- 172.21.15.87:0/1345151584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a92962210 con 0x564a92cea170 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef87fff640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fef740126d0 tx=0x7fef74012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627dc32fd33d6780 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef67fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7401f060 con 0x564a92cea170 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fef74010bb0 con 0x564a92cea170 2024-06-06T07:50:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.442+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c0319f0 con 0x564a92cea170 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef74005020 con 0x564a92cea170 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 10 client.16062 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a92a2e620 con 0x564a92cea170 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c0319f0 con 0x564a92cea170 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fef7400e440 con 0x564a92cea170 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 10 client.16062 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 20 client.16062 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d054a0 0x564a92d07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 4 client.16062 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a92c61f10 con 0x564a92d054a0 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.443+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d054a0 0x564a92d07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.444+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d054a0 0x564a92d07890 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711ea0ad5f14f6ee server_cookie=8cac1271b72fc525 in_seq=0 out_seq=0 2024-06-06T07:50:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.444+0000 7fef67fff640 10 client.16062 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef67fff640 1 -- 172.21.15.87:0/1345151584 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a92c61f10 con 0x564a92d054a0 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef67fff640 10 client.16062 handle_command_reply: tid=0 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 client.16062 shutdown 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 msgr2=0x564a92cf2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fef740126d0 tx=0x7fef74012b90 comp rx=0 tx=0).stop 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 shutdown_connections 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d054a0 0x564a92d07890 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef67fff640 0 client.16062 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 --2- 172.21.15.87:0/1345151584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cf2350 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 >> 172.21.15.87:0/1345151584 conn(0x564a92ce6680 msgr2=0x564a92cec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 shutdown_connections 2024-06-06T07:50:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 1 -- 172.21.15.87:0/1345151584 wait complete. 2024-06-06T07:50:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.445+0000 7fef91d93740 20 client.16062 trim_cache size 0 max 16384 2024-06-06T07:50:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 Processor -- start 2024-06-06T07:50:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 -- start start 2024-06-06T07:50:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92d0d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92d0d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef881b17e0 con 0x564a92cea170 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef881af0b0 con 0x564a92ceaa90 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.460+0000 7fef91d93740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef881acf00 con 0x564a92cf5350 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56272/0 (socket says 172.21.15.87:56272) 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 -- 172.21.15.87:0/1112973481 learned_addr learned my addr 172.21.15.87:0/1112973481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92d0d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 -- 172.21.15.87:0/1112973481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 msgr2=0x564a92d0d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92d0d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 -- 172.21.15.87:0/1112973481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 msgr2=0x564a92d0d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92d0d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 -- 172.21.15.87:0/1112973481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a92ce0b40 con 0x564a92cea170 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92d0d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef87fff640 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fef74005d00 tx=0x7fef74010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b35dbff0d25c455 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef727fc640 1 -- 172.21.15.87:0/1112973481 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400e3b0 con 0x564a92cea170 2024-06-06T07:50:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef727fc640 1 -- 172.21.15.87:0/1112973481 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7402f070 con 0x564a92cea170 2024-06-06T07:50:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.461+0000 7fef727fc640 1 -- 172.21.15.87:0/1112973481 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74004660 con 0x564a92cea170 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- 172.21.15.87:0/1112973481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 msgr2=0x564a92ce5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fef74005d00 tx=0x7fef74010870 comp rx=0 tx=0).stop 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- 172.21.15.87:0/1112973481 shutdown_connections 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92d0d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92d0d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 --2- 172.21.15.87:0/1112973481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92ce5f60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- 172.21.15.87:0/1112973481 >> 172.21.15.87:0/1112973481 conn(0x564a92ce6680 msgr2=0x564a92ced5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- 172.21.15.87:0/1112973481 shutdown_connections 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- 172.21.15.87:0/1112973481 wait complete. 2024-06-06T07:50:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 -- start start 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.462+0000 7fef91d93740 1 Processor -- start 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cfdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef881ad570 con 0x564a92cea170 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef881ae9e0 con 0x564a92ceaa90 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef881adc40 con 0x564a92cf5350 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef91d93740 10 client.0 _ll_register_callbacks cb 0x7fef740095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56278/0 (socket says 172.21.15.87:56278) 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 -- 172.21.15.87:0/4042989094 learned_addr learned my addr 172.21.15.87:0/4042989094 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cfdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 -- 172.21.15.87:0/4042989094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 msgr2=0x564a92cf8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 -- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 msgr2=0x564a92cfdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cfdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 -- 172.21.15.87:0/4042989094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a92cf9260 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cfdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef87fff640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef74006010 tx=0x7fef740109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e1f87d037c2b4a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.463+0000 7fef67fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740055d0 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fef74004660 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7402f070 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef74004860 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 10 client.16068 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a92d6fce0 con 0x564a92cea170 2024-06-06T07:50:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7402f070 con 0x564a92cea170 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fef7402ab00 con 0x564a92cea170 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 10 client.16068 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 20 client.16068 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d70340 0x564a92d72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 4 client.16068 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.464+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a92c5bc20 con 0x564a92d70340 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.465+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d70340 0x564a92d72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.465+0000 7fef67fff640 10 client.16068 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.465+0000 7fef8cd3e640 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d70340 0x564a92d72760 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a35c03c34e9fa88 server_cookie=453a54a0dc2eb7b in_seq=0 out_seq=0 2024-06-06T07:50:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:14 smithi195 ceph-mon[37040]: pgmap v69: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.530+0000 7fef67fff640 1 -- 172.21.15.87:0/4042989094 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564a92c5bc20 con 0x564a92d70340 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.530+0000 7fef67fff640 10 client.16068 handle_command_reply: tid=0 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.530+0000 7fef91d93740 1 client.16068 shutdown 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.530+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 msgr2=0x564a92cfd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.530+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef74006010 tx=0x7fef740109b0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 shutdown_connections 2024-06-06T07:50:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a92d70340 0x564a92d72760 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a92cf5350 0x564a92cf8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef67fff640 0 client.16068 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a92ceaa90 0x564a92cfdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 --2- 172.21.15.87:0/4042989094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a92cea170 0x564a92cfd5b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 >> 172.21.15.87:0/4042989094 conn(0x564a92ce6680 msgr2=0x564a92d00170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 shutdown_connections 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 1 -- 172.21.15.87:0/4042989094 wait complete. 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef91d93740 20 client.16068 trim_cache size 0 max 16384 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:50:14.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 msgr2=0x7fef780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 0x7fef780a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef800cc200 tx=0x7fef800cc190 comp rx=0 tx=0).stop 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 msgr2=0x7fef8819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fef8000cad0 tx=0x7fef8000cf90 comp rx=0 tx=0).stop 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 shutdown_connections 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fef780a2da0 0x7fef780a5260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef8819cb20 0x7fef8819cfd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef88106cd0 0x7fef881987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 --2- 172.21.15.87:0/1308433350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef88100110 0x7fef88198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.531+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 >> 172.21.15.87:0/1308433350 conn(0x7fef88076e30 msgr2=0x7fef88105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.532+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 shutdown_connections 2024-06-06T07:50:14.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:14.532+0000 7fef8efa2640 1 -- 172.21.15.87:0/1308433350 wait complete. 2024-06-06T07:50:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:14 smithi181 ceph-mon[36790]: pgmap v69: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-06-06T07:50:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:14 smithi087 ceph-mon[32276]: pgmap v69: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-06-06T07:50:14.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17879 6.40 MB/sec warmup 18 sec latency 1763.776 ms 2024-06-06T07:50:15.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17879 6.06 MB/sec warmup 19 sec latency 2763.814 ms 2024-06-06T07:50:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:16 smithi195 ceph-mon[37040]: pgmap v70: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 325 KiB/s wr, 57 op/s 2024-06-06T07:50:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:16 smithi181 ceph-mon[36790]: pgmap v70: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 325 KiB/s wr, 57 op/s 2024-06-06T07:50:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:16 smithi087 ceph-mon[32276]: pgmap v70: 129 pgs: 129 active+clean; 11 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 325 KiB/s wr, 57 op/s 2024-06-06T07:50:16.649 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17879 5.76 MB/sec warmup 20 sec latency 3763.874 ms 2024-06-06T07:50:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:17 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:50:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:17 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:50:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:17 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:50:17.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:17 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:50:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:17 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:50:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:17 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:50:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:17 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:50:17.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:17 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:50:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:17 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:50:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:17 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:50:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:17 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:50:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:17 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:50:17.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17879 5.49 MB/sec warmup 21 sec latency 4763.983 ms 2024-06-06T07:50:18.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:18 smithi195 ceph-mon[37040]: pgmap v71: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 537 KiB/s wr, 99 op/s 2024-06-06T07:50:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:18 smithi181 ceph-mon[36790]: pgmap v71: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 537 KiB/s wr, 99 op/s 2024-06-06T07:50:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:18 smithi087 ceph-mon[32276]: pgmap v71: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 537 KiB/s wr, 99 op/s 2024-06-06T07:50:18.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17898 5.24 MB/sec warmup 22 sec latency 4950.881 ms 2024-06-06T07:50:19.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17898 5.01 MB/sec warmup 23 sec latency 1798.656 ms 2024-06-06T07:50:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:20 smithi195 ceph-mon[37040]: pgmap v72: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:20 smithi181 ceph-mon[36790]: pgmap v72: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:20 smithi087 ceph-mon[32276]: pgmap v72: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:20.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17898 4.80 MB/sec warmup 24 sec latency 2798.775 ms 2024-06-06T07:50:21.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17898 4.61 MB/sec warmup 25 sec latency 3798.848 ms 2024-06-06T07:50:22.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:22 smithi195 ceph-mon[37040]: pgmap v73: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:22 smithi181 ceph-mon[36790]: pgmap v73: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:22.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:22 smithi087 ceph-mon[32276]: pgmap v73: 129 pgs: 129 active+clean; 9.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 406 KiB/s wr, 70 op/s 2024-06-06T07:50:22.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 17898 4.43 MB/sec warmup 26 sec latency 4798.932 ms 2024-06-06T07:50:23.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 19670 4.52 MB/sec warmup 27 sec latency 4977.360 ms 2024-06-06T07:50:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:24 smithi195 ceph-mon[37040]: pgmap v74: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 421 KiB/s wr, 77 op/s 2024-06-06T07:50:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:24 smithi181 ceph-mon[36790]: pgmap v74: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 421 KiB/s wr, 77 op/s 2024-06-06T07:50:24.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:24 smithi087 ceph-mon[32276]: pgmap v74: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 421 KiB/s wr, 77 op/s 2024-06-06T07:50:24.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21333 4.79 MB/sec warmup 28 sec latency 569.559 ms 2024-06-06T07:50:25.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21333 4.62 MB/sec warmup 29 sec latency 1569.655 ms 2024-06-06T07:50:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:26 smithi181 ceph-mon[36790]: pgmap v75: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 KiB/s wr, 50 op/s 2024-06-06T07:50:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:26 smithi087 ceph-mon[32276]: pgmap v75: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 KiB/s wr, 50 op/s 2024-06-06T07:50:26.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21333 4.47 MB/sec warmup 30 sec latency 2569.752 ms 2024-06-06T07:50:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:26 smithi195 ceph-mon[37040]: pgmap v75: 129 pgs: 129 active+clean; 9.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 KiB/s wr, 50 op/s 2024-06-06T07:50:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 -- 172.21.15.87:0/2943574705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc4105840 msgr2=0x7f3cc4105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/2943574705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc4105840 0x7f3cc4105c20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb802a410 comp rx=0 tx=0).stop 2024-06-06T07:50:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 -- 172.21.15.87:0/2943574705 shutdown_connections 2024-06-06T07:50:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/2943574705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc41015b0 0x7f3cc410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/2943574705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 0x7f3cc4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/2943574705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc4105840 0x7f3cc4105c20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 -- 172.21.15.87:0/2943574705 >> 172.21.15.87:0/2943574705 conn(0x7f3cc40766a0 msgr2=0x7f3cc4076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 -- 172.21.15.87:0/2943574705 shutdown_connections 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.628+0000 7f3ccacd4640 1 -- 172.21.15.87:0/2943574705 wait complete. 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 Processor -- start 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 -- start start 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 0x7f3cc4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc4105840 0x7f3cc41a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc4110c30 con 0x7f3cc41015b0 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc4110ab0 con 0x7f3cc4100bf0 2024-06-06T07:50:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3ccacd4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc4110db0 con 0x7f3cc4105840 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38574/0 (socket says 172.21.15.87:38574) 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 -- 172.21.15.87:0/4006053953 learned_addr learned my addr 172.21.15.87:0/4006053953 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 0x7f3cc4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc9271640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc4105840 0x7f3cc41a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 -- 172.21.15.87:0/4006053953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc4105840 msgr2=0x7f3cc41a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc4105840 0x7f3cc41a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 -- 172.21.15.87:0/4006053953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 msgr2=0x7f3cc4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 0x7f3cc4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.629+0000 7f3cc3fff640 1 -- 172.21.15.87:0/4006053953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc41a44d0 con 0x7f3cc41015b0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d950 tx=0x7f3cb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3cc1ffb640 1 -- 172.21.15.87:0/4006053953 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0017070 con 0x7f3cc41015b0 2024-06-06T07:50:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3cc1ffb640 1 -- 172.21.15.87:0/4006053953 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb0009c30 con 0x7f3cc41015b0 2024-06-06T07:50:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc41a46a0 con 0x7f3cc41015b0 2024-06-06T07:50:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3cc1ffb640 1 -- 172.21.15.87:0/4006053953 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000f6b0 con 0x7f3cc41015b0 2024-06-06T07:50:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.630+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc41a4bc0 con 0x7f3cc41015b0 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.632+0000 7f3cc1ffb640 1 -- 172.21.15.87:0/4006053953 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3cb00109d0 con 0x7f3cc41015b0 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 Processor -- start 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 -- start start 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc1ffb640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 0x7f3cb40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f2c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55afc0e904f0 con 0x55afc0f37350 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afc0e2e870 con 0x55afc0f2ca90 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55afc0e8c4b0 con 0x55afc0f2c170 2024-06-06T07:50:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc9271640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 0x7f3cb40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc1ffb640 1 -- 172.21.15.87:0/4006053953 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cb00cf470 con 0x7f3cc41015b0 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc9271640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc8a70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f2c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc9271640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38590/0 (socket says 172.21.15.87:38590) 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc9271640 1 -- 172.21.15.87:0/1809970589 learned_addr learned my addr 172.21.15.87:0/1809970589 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:42878/0 (socket says 172.21.15.87:42878) 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1809970589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 msgr2=0x55afc0f2c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f2c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.633+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1809970589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 msgr2=0x55afc0f39740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1809970589 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc0e4a9b0 con 0x55afc0f2ca90 2024-06-06T07:50:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3cc9271640 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3cc9271640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 0x7f3cb40a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00bfd0 tx=0x7f3cbc00bd40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00cc7f0 tx=0x7f3cb009af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86b63c1c319e670 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3caeffd640 1 -- 172.21.15.87:0/1809970589 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0020070 con 0x55afc0f2ca90 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3caeffd640 1 -- 172.21.15.87:0/1809970589 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb009bcc0 con 0x55afc0f2ca90 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.634+0000 7f3caeffd640 1 -- 172.21.15.87:0/1809970589 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb007a140 con 0x55afc0f2ca90 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1809970589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 msgr2=0x55afc0f36e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00cc7f0 tx=0x7f3cb009af20 comp rx=0 tx=0).stop 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1809970589 shutdown_connections 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f37350 0x55afc0f39740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f36e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1809970589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f2c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1809970589 >> 172.21.15.87:0/1809970589 conn(0x55afc0f28680 msgr2=0x55afc0f28a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1809970589 shutdown_connections 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1809970589 wait complete. 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 -- start start 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.635+0000 7f3ccd9e7740 1 Processor -- start 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f3b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55afc0f39f80 con 0x55afc0f2ca90 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afc0e19ae0 con 0x55afc0f37350 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55afc0f3a100 con 0x55afc0f2c170 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 10 client.0 _ll_register_callbacks cb 0x55afc0e529b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38598/0 (socket says 172.21.15.87:38598) 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1713827462 learned_addr learned my addr 172.21.15.87:0/1713827462 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc9271640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f3b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1713827462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 msgr2=0x55afc0f41440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 msgr2=0x55afc0f3b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f3b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 -- 172.21.15.87:0/1713827462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc0ba4210 con 0x55afc0f2ca90 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc9271640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f3b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0010fd0 tx=0x7f3cb000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1a03d9cd9398 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0020050 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cb0078790 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb009bcc0 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3cb009f970 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 10 client.16080 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.636+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afc0c70620 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb009bcc0 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3cb007a3e0 con 0x55afc0f2ca90 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 10 client.16080 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 20 client.16080 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0f4b840 0x55afc0f4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 4 client.16080 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc0ea3f10 con 0x55afc0f4b840 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0f4b840 0x55afc0f4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3ca3fff640 10 client.16080 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.637+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0f4b840 0x55afc0f4dc30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9566777c9ec5a447 server_cookie=b97b9b86de53d579 in_seq=0 out_seq=0 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ca3fff640 1 -- 172.21.15.87:0/1713827462 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55afc0ea3f10 con 0x55afc0f4b840 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ca3fff640 10 client.16080 handle_command_reply: tid=0 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 client.16080 shutdown 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 msgr2=0x55afc0f41980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0010fd0 tx=0x7f3cb000bf60 comp rx=0 tx=0).stop 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 shutdown_connections 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0f4b840 0x55afc0f4dc30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f3b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ca3fff640 0 client.16080 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f41980 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/1713827462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 >> 172.21.15.87:0/1713827462 conn(0x55afc0f28680 msgr2=0x55afc0f2e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 shutdown_connections 2024-06-06T07:50:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.638+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/1713827462 wait complete. 2024-06-06T07:50:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.639+0000 7f3ccd9e7740 20 client.16080 trim_cache size 0 max 16384 2024-06-06T07:50:27.650 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21333 4.32 MB/sec warmup 31 sec latency 3569.795 ms 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 Processor -- start 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 -- start start 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f35f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41af3d0 con 0x55afc0f2ca90 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41acca0 con 0x55afc0f37350 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41aaaf0 con 0x55afc0f2c170 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38606/0 (socket says 172.21.15.87:38606) 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 -- 172.21.15.87:0/869463168 learned_addr learned my addr 172.21.15.87:0/869463168 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc9271640 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f35f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 -- 172.21.15.87:0/869463168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 msgr2=0x55afc0f41810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 -- 172.21.15.87:0/869463168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 msgr2=0x55afc0f35f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f35f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 -- 172.21.15.87:0/869463168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc0f36d00 con 0x55afc0f2ca90 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0097f90 tx=0x7f3cb0099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917617c1fa4d84d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cae7fc640 1 -- 172.21.15.87:0/869463168 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0020070 con 0x55afc0f2ca90 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.654+0000 7f3cae7fc640 1 -- 172.21.15.87:0/869463168 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb009bcc0 con 0x55afc0f2ca90 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3cae7fc640 1 -- 172.21.15.87:0/869463168 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0097c30 con 0x55afc0f2ca90 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/869463168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 msgr2=0x55afc0f35a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0097f90 tx=0x7f3cb0099ad0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/869463168 shutdown_connections 2024-06-06T07:50:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f37350 0x55afc0f35f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2ca90 0x55afc0f35a10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/869463168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f2c170 0x55afc0f41810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/869463168 >> 172.21.15.87:0/869463168 conn(0x55afc0f28680 msgr2=0x55afc0f41100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/869463168 shutdown_connections 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/869463168 wait complete. 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.655+0000 7f3ccd9e7740 1 -- start start 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 Processor -- start 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f37350 0x55afc0fb4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ab160 con 0x55afc0f2c170 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ac5d0 con 0x55afc0f2ca90 2024-06-06T07:50:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ab830 con 0x55afc0f37350 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3ccd9e7740 10 client.0 _ll_register_callbacks cb 0x7f3cbc013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38610/0 (socket says 172.21.15.87:38610) 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 -- 172.21.15.87:0/3111040364 learned_addr learned my addr 172.21.15.87:0/3111040364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 -- 172.21.15.87:0/3111040364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f37350 msgr2=0x55afc0fb4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc9271640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f37350 0x55afc0fb4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f37350 0x55afc0fb4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 -- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 msgr2=0x55afc0f46fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f46fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc8a70640 1 -- 172.21.15.87:0/3111040364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc0fb46c0 con 0x55afc0f2c170 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.656+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f46fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3cc8a70640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb8031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2786c64c646f2375 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb804d030 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f3cb8004250 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb009bcc0 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3cb8057050 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 10 client.16086 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afc0fb5940 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb009bcc0 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3cb8048680 con 0x55afc0f2c170 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 10 client.16086 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 20 client.16086 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0fb61a0 0x55afc0fb85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 4 client.16086 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:50:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.657+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afc0e9dc20 con 0x55afc0fb61a0 2024-06-06T07:50:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.658+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0fb61a0 0x55afc0fb85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.658+0000 7f3ca3fff640 10 client.16086 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.658+0000 7f3cc3fff640 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0fb61a0 0x55afc0fb85c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7d440cb60b735b server_cookie=e2cc1aa600a660c2 in_seq=0 out_seq=0 2024-06-06T07:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ca3fff640 1 -- 172.21.15.87:0/3111040364 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55afc0e9dc20 con 0x55afc0fb61a0 2024-06-06T07:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ca3fff640 10 client.16086 handle_command_reply: tid=0 2024-06-06T07:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 client.16086 shutdown 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 msgr2=0x55afc0f46a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb8031960 comp rx=0 tx=0).stop 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 shutdown_connections 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55afc0fb61a0 0x55afc0fb85c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55afc0f37350 0x55afc0fb4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afc0f2ca90 0x55afc0f46fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 --2- 172.21.15.87:0/3111040364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55afc0f2c170 0x55afc0f46a60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ca3fff640 0 client.16086 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 >> 172.21.15.87:0/3111040364 conn(0x55afc0f28680 msgr2=0x55afc0f26f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 shutdown_connections 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 1 -- 172.21.15.87:0/3111040364 wait complete. 2024-06-06T07:50:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.699+0000 7f3ccd9e7740 20 client.16086 trim_cache size 0 max 16384 2024-06-06T07:50:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:50:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:50:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 msgr2=0x7f3cb40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 0x7f3cb40a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00bfd0 tx=0x7f3cbc00bd40 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 msgr2=0x7f3cc4199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d950 tx=0x7f3cb000de10 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 shutdown_connections 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3cb40a2da0 0x7f3cb40a5260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cc4105840 0x7f3cc41a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc41015b0 0x7f3cc4199450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 --2- 172.21.15.87:0/4006053953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4100bf0 0x7f3cc4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 >> 172.21.15.87:0/4006053953 conn(0x7f3cc40766a0 msgr2=0x7f3cc40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 shutdown_connections 2024-06-06T07:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:27.700+0000 7f3ccacd4640 1 -- 172.21.15.87:0/4006053953 wait complete. 2024-06-06T07:50:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:28 smithi181 ceph-mon[36790]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-06-06T07:50:28.561 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:28 smithi195 ceph-mon[37040]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-06-06T07:50:28.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:28 smithi087 ceph-mon[32276]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-06-06T07:50:28.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21466 4.20 MB/sec warmup 32 sec latency 3929.732 ms 2024-06-06T07:50:29.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21466 4.07 MB/sec warmup 33 sec latency 1557.450 ms 2024-06-06T07:50:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:30 smithi181 ceph-mon[36790]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:30 smithi087 ceph-mon[32276]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:30.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21466 3.95 MB/sec warmup 34 sec latency 2557.485 ms 2024-06-06T07:50:30.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:30 smithi195 ceph-mon[37040]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:31.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21466 3.84 MB/sec warmup 35 sec latency 3557.592 ms 2024-06-06T07:50:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:32 smithi181 ceph-mon[36790]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:32.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:32 smithi087 ceph-mon[32276]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:32.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 21466 3.73 MB/sec warmup 36 sec latency 4557.648 ms 2024-06-06T07:50:32.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:32 smithi195 ceph-mon[37040]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 202 KiB/s wr, 31 op/s 2024-06-06T07:50:33.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 22992 3.81 MB/sec warmup 37 sec latency 4736.791 ms 2024-06-06T07:50:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:34 smithi181 ceph-mon[36790]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-06-06T07:50:34.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:34 smithi087 ceph-mon[32276]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-06-06T07:50:34.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 25011 4.04 MB/sec warmup 38 sec latency 453.483 ms 2024-06-06T07:50:34.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:34 smithi195 ceph-mon[37040]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-06-06T07:50:35.426 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:50:35.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 25011 3.94 MB/sec warmup 39 sec latency 1453.531 ms 2024-06-06T07:50:35.676 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:36 smithi087 ceph-mon[32276]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.0 MiB/s wr, 124 op/s 2024-06-06T07:50:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:36 smithi181 ceph-mon[36790]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.0 MiB/s wr, 124 op/s 2024-06-06T07:50:36.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 25011 3.84 MB/sec warmup 40 sec latency 2453.643 ms 2024-06-06T07:50:36.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:36 smithi195 ceph-mon[37040]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.0 MiB/s wr, 124 op/s 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 -- 172.21.15.87:0/2514538741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06ac60 msgr2=0x7f9a7c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2514538741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c06b0e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0099c0 tx=0x7f9a6c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 -- 172.21.15.87:0/2514538741 shutdown_connections 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2514538741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06b620 0x7f9a7c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2514538741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 0x7f9a7c108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2514538741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c06b0e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 -- 172.21.15.87:0/2514538741 >> 172.21.15.87:0/2514538741 conn(0x7f9a7c0772a0 msgr2=0x7f9a7c0776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 -- 172.21.15.87:0/2514538741 shutdown_connections 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.520+0000 7f9a845a4700 1 -- 172.21.15.87:0/2514538741 wait complete. 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 Processor -- start 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 -- start start 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 0x7f9a7c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c112810 con 0x7f9a7c06b620 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c112690 con 0x7f9a7c06ac60 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a845a4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c112990 con 0x7f9a7c1085b0 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35016/0 (socket says 172.21.15.87:35016) 2024-06-06T07:50:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 -- 172.21.15.87:0/2788254642 learned_addr learned my addr 172.21.15.87:0/2788254642 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a82b41700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 0x7f9a7c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a82340700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 -- 172.21.15.87:0/2788254642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 msgr2=0x7f9a7c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 0x7f9a7c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 -- 172.21.15.87:0/2788254642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06ac60 msgr2=0x7f9a7c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.521+0000 7f9a81b3f700 1 -- 172.21.15.87:0/2788254642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c199170 con 0x7f9a7c06b620 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006010 tx=0x7f9a6c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a737fe700 1 -- 172.21.15.87:0/2788254642 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c026070 con 0x7f9a7c06b620 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a737fe700 1 -- 172.21.15.87:0/2788254642 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c004cd0 con 0x7f9a7c06b620 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c199400 con 0x7f9a7c06b620 2024-06-06T07:50:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a737fe700 1 -- 172.21.15.87:0/2788254642 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c003ba0 con 0x7f9a7c06b620 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.522+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c1014c0 con 0x7f9a7c06b620 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.524+0000 7f9a737fe700 1 -- 172.21.15.87:0/2788254642 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9a6c029070 con 0x7f9a7c06b620 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.525+0000 7f9a737fe700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 0x7f9a680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.525+0000 7f9a82340700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 0x7f9a680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.525+0000 7f9a737fe700 1 -- 172.21.15.87:0/2788254642 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9a6c0d0150 con 0x7f9a7c06b620 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.525+0000 7f9a82340700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 0x7f9a680a5250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c199b50 tx=0x7f9a7800b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.525+0000 7f9a8ee27b80 1 Processor -- start 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 -- start start 2024-06-06T07:50:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 0x55b042d40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d407a0 0x55b042d40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b042c32f60 con 0x55b042d45f60 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b042c91850 con 0x55b042d46880 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b042bf9460 con 0x55b042d407a0 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35022/0 (socket says 172.21.15.87:35022) 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 -- 172.21.15.87:0/2261275220 learned_addr learned my addr 172.21.15.87:0/2261275220 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 0x55b042d40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 -- 172.21.15.87:0/2261275220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d407a0 msgr2=0x55b042d40bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82b41700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d407a0 0x55b042d40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d407a0 0x55b042d40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 -- 172.21.15.87:0/2261275220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 msgr2=0x55b042d40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 0x55b042d40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 -- 172.21.15.87:0/2261275220 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b042be08d0 con 0x55b042d45f60 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 0x55b042d40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.526+0000 7f9a82340700 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005d70 tx=0x7f9a78013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2141dcc8513aa97 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a70ff9700 1 -- 172.21.15.87:0/2261275220 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78011ad0 con 0x55b042d45f60 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a70ff9700 1 -- 172.21.15.87:0/2261275220 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a78003e90 con 0x55b042d45f60 2024-06-06T07:50:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a70ff9700 1 -- 172.21.15.87:0/2261275220 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7801c610 con 0x55b042d45f60 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/2261275220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 msgr2=0x55b042d46340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005d70 tx=0x7f9a78013000 comp rx=0 tx=0).stop 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/2261275220 shutdown_connections 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d407a0 0x55b042d40bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d46340 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/2261275220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d46880 0x55b042d40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.527+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/2261275220 >> 172.21.15.87:0/2261275220 conn(0x55b042c6d080 msgr2=0x55b042d3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/2261275220 shutdown_connections 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/2261275220 wait complete. 2024-06-06T07:50:37.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- start start 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 Processor -- start 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d4c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b042c31fc0 con 0x55b042d45f60 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b042c8d810 con 0x55b042d407a0 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b042d4d260 con 0x55b042d46880 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a8ee27b80 10 client.0 _ll_register_callbacks cb 0x55b042c275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35036/0 (socket says 172.21.15.87:35036) 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a81b3f700 1 -- 172.21.15.87:0/4059833708 learned_addr learned my addr 172.21.15.87:0/4059833708 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.528+0000 7f9a82340700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d4c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a82b41700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 -- 172.21.15.87:0/4059833708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 msgr2=0x55b042d49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 -- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 msgr2=0x55b042d4c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d4c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 -- 172.21.15.87:0/4059833708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b042c57750 con 0x55b042d45f60 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c000c00 tx=0x7f9a6c09bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b93dffbe47aaf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0b5060 con 0x55b042d45f60 2024-06-06T07:50:37.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9a6c0b5200 con 0x55b042d45f60 2024-06-06T07:50:37.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a78003e90 con 0x55b042d45f60 2024-06-06T07:50:37.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c09c330 con 0x55b042d45f60 2024-06-06T07:50:37.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a8ee27b80 10 client.16098 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:37.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b042d579e0 con 0x55b042d45f60 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a78003e90 con 0x55b042d45f60 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a6c067260 con 0x55b042d45f60 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a8ee27b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.529+0000 7f9a8ee27b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a8ee27b80 20 client.16098 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042d59fa0 0x55b042d5c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a8ee27b80 4 client.16098 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b042c9ed50 con 0x55b042d59fa0 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a82340700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042d59fa0 0x55b042d5c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a82340700 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042d59fa0 0x55b042d5c3b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c106dfc450815124 server_cookie=ace4db6d1672f30b in_seq=0 out_seq=0 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.530+0000 7f9a65ffb700 10 client.16098 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:37.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a65ffb700 1 -- 172.21.15.87:0/4059833708 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b042c9ed50 con 0x55b042d59fa0 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a65ffb700 10 client.16098 handle_command_reply: tid=0 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 client.16098 shutdown 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 msgr2=0x55b042d498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c000c00 tx=0x7f9a6c09bd10 comp rx=0 tx=0).stop 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 shutdown_connections 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042d59fa0 0x55b042d5c3b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d4c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a65ffb700 0 client.16098 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/4059833708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d498f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 >> 172.21.15.87:0/4059833708 conn(0x55b042c6d080 msgr2=0x55b042d3a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 shutdown_connections 2024-06-06T07:50:37.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/4059833708 wait complete. 2024-06-06T07:50:37.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.531+0000 7f9a8ee27b80 20 client.16098 trim_cache size 0 max 16384 2024-06-06T07:50:37.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.565+0000 7f9a8ee27b80 1 Processor -- start 2024-06-06T07:50:37.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.565+0000 7f9a8ee27b80 1 -- start start 2024-06-06T07:50:37.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.565+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d45f60 0x55b042d47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1b2ef0 con 0x55b042d46880 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1b0780 con 0x55b042d407a0 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1ae5d0 con 0x55b042d45f60 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35042/0 (socket says 172.21.15.87:35042) 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 -- 172.21.15.87:0/3655788531 learned_addr learned my addr 172.21.15.87:0/3655788531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d45f60 0x55b042d47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82340700 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d49f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 -- 172.21.15.87:0/3655788531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d45f60 msgr2=0x55b042d47430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d45f60 0x55b042d47430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 -- 172.21.15.87:0/3655788531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 msgr2=0x55b042d49f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d49f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 -- 172.21.15.87:0/3655788531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b042d47e60 con 0x55b042d46880 2024-06-06T07:50:37.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a82b41700 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009860 tx=0x7f9a74009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3cf3afba539a9f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a67fff700 1 -- 172.21.15.87:0/3655788531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7401a030 con 0x55b042d46880 2024-06-06T07:50:37.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a67fff700 1 -- 172.21.15.87:0/3655788531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a74004bb0 con 0x55b042d46880 2024-06-06T07:50:37.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.566+0000 7f9a67fff700 1 -- 172.21.15.87:0/3655788531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74007d70 con 0x55b042d46880 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/3655788531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 msgr2=0x55b042d3ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009860 tx=0x7f9a74009b70 comp rx=0 tx=0).stop 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/3655788531 shutdown_connections 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d49f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d46880 0x55b042d3ae80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/3655788531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d45f60 0x55b042d47430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/3655788531 >> 172.21.15.87:0/3655788531 conn(0x55b042c6d080 msgr2=0x55b042d46d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/3655788531 shutdown_connections 2024-06-06T07:50:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.567+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/3655788531 wait complete. 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 -- start start 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 Processor -- start 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d4f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1aec40 con 0x55b042d45f60 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1b00b0 con 0x55b042d407a0 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1af310 con 0x55b042d46880 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a8ee27b80 10 client.0 _ll_register_callbacks cb 0x55b042c275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35054/0 (socket says 172.21.15.87:35054) 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 -- 172.21.15.87:0/1714378385 learned_addr learned my addr 172.21.15.87:0/1714378385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a82340700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a82b41700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d4f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 -- 172.21.15.87:0/1714378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 msgr2=0x55b042d4f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d4f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 -- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 msgr2=0x55b042d527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.568+0000 7f9a81b3f700 1 -- 172.21.15.87:0/1714378385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b042d4fd10 con 0x55b042d45f60 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a81b3f700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c09c2c0 tx=0x7f9a6c00b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c5a50f509c7024 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c005000 con 0x55b042d45f60 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9a6c0a4a80 con 0x55b042d45f60 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a74004bb0 con 0x55b042d45f60 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c0b5020 con 0x55b042d45f60 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 10 client.16104 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b042d4fd10 con 0x55b042d45f60 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a74004bb0 con 0x55b042d45f60 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a6c09b800 con 0x55b042d45f60 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 20 client.16104 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042dc5b40 0x55b042dc7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 4 client.16104 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.569+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b042c992d0 con 0x55b042dc5b40 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a82340700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042dc5b40 0x55b042dc7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a65ffb700 10 client.16104 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a82340700 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042dc5b40 0x55b042dc7f60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce9d6396438ae9d server_cookie=c5d2f26b210a2f56 in_seq=0 out_seq=0 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a65ffb700 1 -- 172.21.15.87:0/1714378385 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b042c992d0 con 0x55b042dc5b40 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a65ffb700 10 client.16104 handle_command_reply: tid=0 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 client.16104 shutdown 2024-06-06T07:50:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 msgr2=0x55b042d52d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c09c2c0 tx=0x7f9a6c00b580 comp rx=0 tx=0).stop 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 shutdown_connections 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b042dc5b40 0x55b042dc7f60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b042d407a0 0x55b042d527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b042d46880 0x55b042d4f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 --2- 172.21.15.87:0/1714378385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b042d45f60 0x55b042d52d20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 >> 172.21.15.87:0/1714378385 conn(0x55b042c6d080 msgr2=0x55b042d4b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a65ffb700 0 client.16104 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 shutdown_connections 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 1 -- 172.21.15.87:0/1714378385 wait complete. 2024-06-06T07:50:37.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.570+0000 7f9a8ee27b80 20 client.16104 trim_cache size 0 max 16384 2024-06-06T07:50:37.577 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:50:37.577 INFO:teuthology.orchestra.run.smithi087.stdout: "status": "no active scrubs running", 2024-06-06T07:50:37.577 INFO:teuthology.orchestra.run.smithi087.stdout: "scrubs": {} 2024-06-06T07:50:37.577 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 msgr2=0x7f9a680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 0x7f9a680a5250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c199b50 tx=0x7f9a7800b3c0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 msgr2=0x7f9a7c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006010 tx=0x7f9a6c009f80 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 shutdown_connections 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c06ac60 0x7f9a7c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9a680a2d90 0x7f9a680a5250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a7c1085b0 0x7f9a7c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 --2- 172.21.15.87:0/2788254642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a7c06b620 0x7f9a7c19ed90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 >> 172.21.15.87:0/2788254642 conn(0x7f9a7c0772a0 msgr2=0x7f9a7c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:37.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 shutdown_connections 2024-06-06T07:50:37.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:37.571+0000 7f9a845a4700 1 -- 172.21.15.87:0/2788254642 wait complete. 2024-06-06T07:50:37.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 25011 3.74 MB/sec warmup 41 sec latency 3453.780 ms 2024-06-06T07:50:38.383 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:50:38.383 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:50:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:38 smithi181 ceph-mon[36790]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-06-06T07:50:38.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:38 smithi087 ceph-mon[32276]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-06-06T07:50:38.635 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:38.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 26854 3.82 MB/sec warmup 42 sec latency 3630.688 ms 2024-06-06T07:50:38.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:38 smithi195 ceph-mon[37040]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-06-06T07:50:39.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28489 4.01 MB/sec warmup 43 sec latency 549.757 ms 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- 172.21.15.87:0/566741343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 msgr2=0x7f76c0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 --2- 172.21.15.87:0/566741343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c0108940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f76b00099c0 tx=0x7f76b0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- 172.21.15.87:0/566741343 shutdown_connections 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 --2- 172.21.15.87:0/566741343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01013a0 0x7f76c010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 --2- 172.21.15.87:0/566741343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c0108940 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 --2- 172.21.15.87:0/566741343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01009e0 0x7f76c0100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- 172.21.15.87:0/566741343 >> 172.21.15.87:0/566741343 conn(0x7f76c0077990 msgr2=0x7f76c00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- 172.21.15.87:0/566741343 shutdown_connections 2024-06-06T07:50:40.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- 172.21.15.87:0/566741343 wait complete. 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 Processor -- start 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.387+0000 7f76c5c47700 1 -- start start 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 0x7f76c019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01013a0 0x7f76c019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01127c0 con 0x7f76c0108560 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112640 con 0x7f76c01013a0 2024-06-06T07:50:40.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76c5c47700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112940 con 0x7f76c01009e0 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 0x7f76c019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 0x7f76c019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:60956/0 (socket says 172.21.15.87:60956) 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bf7fe700 1 -- 172.21.15.87:0/665076246 learned_addr learned my addr 172.21.15.87:0/665076246 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76beffd700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01013a0 0x7f76c019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 -- 172.21.15.87:0/665076246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 msgr2=0x7f76c019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 0x7f76c019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 -- 172.21.15.87:0/665076246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01013a0 msgr2=0x7f76c019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01013a0 0x7f76c019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 -- 172.21.15.87:0/665076246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019fdc0 con 0x7f76c0108560 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bffff700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c019f680 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ab40 tx=0x7f76b800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.388+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8017020 con 0x7f76c0108560 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.389+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c01a00b0 con 0x7f76c0108560 2024-06-06T07:50:40.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.389+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b8004bb0 con 0x7f76c0108560 2024-06-06T07:50:40.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.389+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8005280 con 0x7f76c0108560 2024-06-06T07:50:40.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.389+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a7930 con 0x7f76c0108560 2024-06-06T07:50:40.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.389+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0069180 con 0x7f76c0108560 2024-06-06T07:50:40.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.391+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f76b800c580 con 0x7f76c0108560 2024-06-06T07:50:40.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.391+0000 7f76bcff9700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 0x7f76a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.392+0000 7f76bf7fe700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 0x7f76a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.392+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f76b80cf370 con 0x7f76c0108560 2024-06-06T07:50:40.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.392+0000 7f76bf7fe700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 0x7f76a80a5250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009750 tx=0x7f76b0005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.395+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76b8021020 con 0x7f76c0108560 2024-06-06T07:50:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:40 smithi181 ceph-mon[36790]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:40 smithi087 ceph-mon[32276]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:40.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.638+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76c01a7b00 con 0x7f76c0108560 2024-06-06T07:50:40.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.638+0000 7f76bcff9700 1 -- 172.21.15.87:0/665076246 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11877 (secure 0 0 0) 0x7f76b809bc50 con 0x7f76c0108560 2024-06-06T07:50:40.639 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:50:40.639 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:50:06.260840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.640+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 msgr2=0x7f76a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.640+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 0x7f76a80a5250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009750 tx=0x7f76b0005960 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.640+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 msgr2=0x7f76c019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.640+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c019f680 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ab40 tx=0x7f76b800ae50 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 shutdown_connections 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76a80a2d90 0x7f76a80a5250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76c0108560 0x7f76c019f680 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c01009e0 0x7f76c019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 --2- 172.21.15.87:0/665076246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76c01013a0 0x7f76c019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 >> 172.21.15.87:0/665076246 conn(0x7f76c0077990 msgr2=0x7f76c00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 shutdown_connections 2024-06-06T07:50:40.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:40.641+0000 7f76c5c47700 1 -- 172.21.15.87:0/665076246 wait complete. 2024-06-06T07:50:40.642 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 30 2024-06-06T07:50:40.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28489 3.92 MB/sec warmup 44 sec latency 1549.800 ms 2024-06-06T07:50:40.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:40 smithi195 ceph-mon[37040]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 -- 172.21.15.87:0/1400904942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc448101290 msgr2=0x7fc44810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 --2- 172.21.15.87:0/1400904942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc448101290 0x7fc44810d750 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc440009cd0 tx=0x7fc44002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 -- 172.21.15.87:0/1400904942 shutdown_connections 2024-06-06T07:50:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 --2- 172.21.15.87:0/1400904942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc448101290 0x7fc44810d750 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 --2- 172.21.15.87:0/1400904942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4481008d0 0x7fc448100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 --2- 172.21.15.87:0/1400904942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc448106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 -- 172.21.15.87:0/1400904942 >> 172.21.15.87:0/1400904942 conn(0x7fc448077a20 msgr2=0x7fc448077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 -- 172.21.15.87:0/1400904942 shutdown_connections 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.787+0000 7fc44ea12640 1 -- 172.21.15.87:0/1400904942 wait complete. 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 Processor -- start 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 -- start start 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc448101290 0x7fc4481994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc4481a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc448110ca0 con 0x7fc4481008d0 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc448110b20 con 0x7fc448106890 2024-06-06T07:50:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc44ea12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc448110e20 con 0x7fc448101290 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35092/0 (socket says 172.21.15.87:35092) 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc447fff640 1 -- 172.21.15.87:0/2215107955 learned_addr learned my addr 172.21.15.87:0/2215107955 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.788+0000 7fc4477fe640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc448101290 0x7fc4481994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc4481a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 -- 172.21.15.87:0/2215107955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc448101290 msgr2=0x7fc4481994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc448101290 0x7fc4481994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 -- 172.21.15.87:0/2215107955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 msgr2=0x7fc4481a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc4481a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 -- 172.21.15.87:0/2215107955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4481a4540 con 0x7fc4481008d0 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc4481a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc447fff640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c890 tx=0x7fc43800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc4457fa640 1 -- 172.21.15.87:0/2215107955 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438004220 con 0x7fc4481008d0 2024-06-06T07:50:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc4457fa640 1 -- 172.21.15.87:0/2215107955 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4380043c0 con 0x7fc4481008d0 2024-06-06T07:50:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4481a4710 con 0x7fc4481008d0 2024-06-06T07:50:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc4457fa640 1 -- 172.21.15.87:0/2215107955 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438004a00 con 0x7fc4481008d0 2024-06-06T07:50:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.789+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481a4c00 con 0x7fc4481008d0 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.791+0000 7fc4457fa640 1 -- 172.21.15.87:0/2215107955 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc4380079b0 con 0x7fc4481008d0 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.791+0000 7fc4517f6740 1 Processor -- start 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.791+0000 7fc4517f6740 1 -- start start 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1bacb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1badcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a1ba2f4f0 con 0x564a1bacb990 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a1b9cd870 con 0x564a1bacb070 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4517f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a1ba2b4b0 con 0x564a1bada760 2024-06-06T07:50:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4457fa640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 0x7fc4300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35102/0 (socket says 172.21.15.87:35102) 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 -- 172.21.15.87:0/3443750018 learned_addr learned my addr 172.21.15.87:0/3443750018 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 0x7fc4300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4477fe640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1badcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc4457fa640 1 -- 172.21.15.87:0/2215107955 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc4380cf420 con 0x7fc4481008d0 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1bacb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 -- 172.21.15.87:0/3443750018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 msgr2=0x564a1badcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1badcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 -- 172.21.15.87:0/3443750018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 msgr2=0x564a1bacb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1bacb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 -- 172.21.15.87:0/3443750018 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1b9e99b0 con 0x564a1bacb990 2024-06-06T07:50:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1bacb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc447fff640 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc4380979d0 tx=0x7fc43801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a63d90178ea44b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 0x7fc4300a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc440005e30 tx=0x7fc440005da0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.792+0000 7fc42e7fc640 1 -- 172.21.15.87:0/3443750018 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43801d070 con 0x564a1bacb990 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc42e7fc640 1 -- 172.21.15.87:0/3443750018 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43809bc70 con 0x564a1bacb990 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc42e7fc640 1 -- 172.21.15.87:0/3443750018 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438097be0 con 0x564a1bacb990 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- 172.21.15.87:0/3443750018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 msgr2=0x564a1bada220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc4380979d0 tx=0x7fc43801b000 comp rx=0 tx=0).stop 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- 172.21.15.87:0/3443750018 shutdown_connections 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1badcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb990 0x564a1bada220 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 --2- 172.21.15.87:0/3443750018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1bacb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- 172.21.15.87:0/3443750018 >> 172.21.15.87:0/3443750018 conn(0x564a1bac7680 msgr2=0x564a1bac7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- 172.21.15.87:0/3443750018 shutdown_connections 2024-06-06T07:50:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- 172.21.15.87:0/3443750018 wait complete. 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.793+0000 7fc4517f6740 1 -- start start 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 Processor -- start 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bad7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1bad8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a1badd330 con 0x564a1bacb070 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a1b9b8ae0 con 0x564a1bacb990 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a1badd4b0 con 0x564a1bada760 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc4517f6740 10 client.0 _ll_register_callbacks cb 0x564a1b9f19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35116/0 (socket says 172.21.15.87:35116) 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 -- 172.21.15.87:0/1381316273 learned_addr learned my addr 172.21.15.87:0/1381316273 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 -- 172.21.15.87:0/1381316273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 msgr2=0x564a1bad8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1bad8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 -- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 msgr2=0x564a1bad7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bad7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.794+0000 7fc447fff640 1 -- 172.21.15.87:0/1381316273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1b743210 con 0x564a1bacb070 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc447fff640 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ccda0 tx=0x7fc438097e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b35873ec1d4238 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43801d070 con 0x564a1bacb070 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc4380cf170 con 0x564a1bacb070 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43809bc70 con 0x564a1bacb070 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc43809f920 con 0x564a1bacb070 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 10 client.16122 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a1b80f620 con 0x564a1bacb070 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43809bc70 con 0x564a1bacb070 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc438084b90 con 0x564a1bacb070 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 10 client.16122 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 20 client.16122 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1baea7f0 0x564a1baecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 4 client.16122 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a1ba42f10 con 0x564a1baea7f0 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.795+0000 7fc4477fe640 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1baea7f0 0x564a1baecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.796+0000 7fc4477fe640 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1baea7f0 0x564a1baecbe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c77145c89e97a2 server_cookie=e3cd57e3ed580ad in_seq=0 out_seq=0 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.796+0000 7fc4277fe640 10 client.16122 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4277fe640 1 -- 172.21.15.87:0/1381316273 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a1ba42f10 con 0x564a1baea7f0 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4277fe640 10 client.16122 handle_command_reply: tid=0 2024-06-06T07:50:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 client.16122 shutdown 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 msgr2=0x564a1bad77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ccda0 tx=0x7fc438097e90 comp rx=0 tx=0).stop 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 shutdown_connections 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1baea7f0 0x564a1baecbe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bada760 0x564a1bad8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bad7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4277fe640 0 client.16122 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 --2- 172.21.15.87:0/1381316273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bacb070 0x564a1bad77e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 >> 172.21.15.87:0/1381316273 conn(0x564a1bac7680 msgr2=0x564a1bacd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 shutdown_connections 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 1 -- 172.21.15.87:0/1381316273 wait complete. 2024-06-06T07:50:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.797+0000 7fc4517f6740 20 client.16122 trim_cache size 0 max 16384 2024-06-06T07:50:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 Processor -- start 2024-06-06T07:50:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 -- start start 2024-06-06T07:50:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1baf21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb990 0x564a1baf26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481af570 con 0x564a1bada760 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481ace40 con 0x564a1bacb070 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4517f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481aac90 con 0x564a1bacb990 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35130/0 (socket says 172.21.15.87:35130) 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2580736195 learned_addr learned my addr 172.21.15.87:0/2580736195 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc4477fe640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb990 0x564a1baf26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2580736195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb990 msgr2=0x564a1baf26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb990 0x564a1baf26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2580736195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 msgr2=0x564a1baf21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc447fff640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1baf21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1baf21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.812+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2580736195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1bae71b0 con 0x564a1bada760 2024-06-06T07:50:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc447fff640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1baf21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc440031b30 tx=0x7fc440040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ea83a135281995 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc42dffb640 1 -- 172.21.15.87:0/2580736195 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440031b60 con 0x564a1bada760 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc42dffb640 1 -- 172.21.15.87:0/2580736195 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc440036040 con 0x564a1bada760 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc42dffb640 1 -- 172.21.15.87:0/2580736195 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44003d420 con 0x564a1bada760 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 -- 172.21.15.87:0/2580736195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 msgr2=0x564a1bac5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc440031b30 tx=0x7fc440040070 comp rx=0 tx=0).stop 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 -- 172.21.15.87:0/2580736195 shutdown_connections 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bac5f50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb990 0x564a1baf26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2580736195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb070 0x564a1baf21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 -- 172.21.15.87:0/2580736195 >> 172.21.15.87:0/2580736195 conn(0x564a1bac7680 msgr2=0x564a1bad2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.813+0000 7fc4517f6740 1 -- 172.21.15.87:0/2580736195 shutdown_connections 2024-06-06T07:50:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 -- 172.21.15.87:0/2580736195 wait complete. 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 -- start start 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 Processor -- start 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb070 0x564a1baddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bade1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481ab300 con 0x564a1bada760 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481ac770 con 0x564a1bacb990 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481ab9d0 con 0x564a1bacb070 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4517f6740 10 client.0 _ll_register_callbacks cb 0x7fc4380058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc44cfaf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc44cfaf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35142/0 (socket says 172.21.15.87:35142) 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2445852858 learned_addr learned my addr 172.21.15.87:0/2445852858 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.814+0000 7fc4477fe640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bade1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc447fff640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb070 0x564a1baddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2445852858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb070 msgr2=0x564a1baddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb070 0x564a1baddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 msgr2=0x564a1bade1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bade1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 -- 172.21.15.87:0/2445852858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1bb536e0 con 0x564a1bada760 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4477fe640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bade1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc44cfaf640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc440031860 tx=0x7fc4400318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaad80b69f3162e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440003d40 con 0x564a1bada760 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc440031b60 con 0x564a1bada760 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc440036040 con 0x564a1bada760 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc440068050 con 0x564a1bada760 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4517f6740 10 client.16128 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a1bb54960 con 0x564a1bada760 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc440036040 con 0x564a1bada760 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.815+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc440063030 con 0x564a1bada760 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 10 client.16128 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 20 client.16128 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1bb55150 0x564a1bb57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 4 client.16128 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a1ba3cc20 con 0x564a1bb55150 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc447fff640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1bb55150 0x564a1bb57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc4277fe640 10 client.16128 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.816+0000 7fc447fff640 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1bb55150 0x564a1bb57570 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d804a428d15bdcec server_cookie=49318489ff243f91 in_seq=0 out_seq=0 2024-06-06T07:50:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4277fe640 1 -- 172.21.15.87:0/2445852858 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564a1ba3cc20 con 0x564a1bb55150 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4277fe640 10 client.16128 handle_command_reply: tid=0 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 client.16128 shutdown 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 msgr2=0x564a1bb53170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc440031860 tx=0x7fc4400318c0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 shutdown_connections 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a1bb55150 0x564a1bb57570 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1bada760 0x564a1bb53170 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1bacb990 0x564a1bade1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4277fe640 0 client.16128 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 --2- 172.21.15.87:0/2445852858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1bacb070 0x564a1baddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 >> 172.21.15.87:0/2445852858 conn(0x564a1bac7680 msgr2=0x564a1bac5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 shutdown_connections 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 1 -- 172.21.15.87:0/2445852858 wait complete. 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.900+0000 7fc4517f6740 20 client.16128 trim_cache size 0 max 16384 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:50:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 msgr2=0x7fc4300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 0x7fc4300a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc440005e30 tx=0x7fc440005da0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 msgr2=0x7fc448198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c890 tx=0x7fc43800cc90 comp rx=0 tx=0).stop 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 shutdown_connections 2024-06-06T07:50:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc4300a2da0 0x7fc4300a5260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc448106890 0x7fc4481a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc448101290 0x7fc4481994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 --2- 172.21.15.87:0/2215107955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4481008d0 0x7fc448198f80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 >> 172.21.15.87:0/2215107955 conn(0x7fc448077a20 msgr2=0x7fc4481032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 shutdown_connections 2024-06-06T07:50:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:40.901+0000 7fc44ea12640 1 -- 172.21.15.87:0/2215107955 wait complete. 2024-06-06T07:50:41.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:41 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/665076246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:41.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:50:41.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:41 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/665076246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:41.644 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:41.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28489 3.83 MB/sec warmup 45 sec latency 2549.828 ms 2024-06-06T07:50:41.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:41 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/665076246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:42 smithi181 ceph-mon[36790]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:42.594 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:42 smithi087 ceph-mon[32276]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:42.651 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28489 3.75 MB/sec warmup 46 sec latency 3549.910 ms 2024-06-06T07:50:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:42 smithi195 ceph-mon[37040]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.0 MiB/s wr, 129 op/s 2024-06-06T07:50:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/4185383637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc4108530 msgr2=0x7f7fc4108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/4185383637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc4108530 0x7f7fc4108910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009990 tx=0x7f7fc0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/4185383637 shutdown_connections 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/4185383637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406b5c0 0x7f7fc410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/4185383637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc4108530 0x7f7fc4108910 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/4185383637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc406ac00 0x7f7fc406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/4185383637 >> 172.21.15.87:0/4185383637 conn(0x7f7fc4077400 msgr2=0x7f7fc4077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/4185383637 shutdown_connections 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/4185383637 wait complete. 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.457+0000 7f7fcc6a0700 1 Processor -- start 2024-06-06T07:50:43.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 -- start start 2024-06-06T07:50:43.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406ac00 0x7f7fc4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4108530 0x7f7fc419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41127e0 con 0x7f7fc406b5c0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4112660 con 0x7f7fc406ac00 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcc6a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4112960 con 0x7f7fc4108530 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35160/0 (socket says 172.21.15.87:35160) 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3125705061 learned_addr learned my addr 172.21.15.87:0/3125705061 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406ac00 0x7f7fc4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4108530 0x7f7fc419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3125705061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4108530 msgr2=0x7f7fc419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4108530 0x7f7fc419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3125705061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406ac00 msgr2=0x7f7fc4198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406ac00 0x7f7fc4198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.458+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3125705061 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc419dd20 con 0x7f7fc406b5c0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400b530 tx=0x7f7fb400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fbb7fe700 1 -- 172.21.15.87:0/3125705061 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4015020 con 0x7f7fc406b5c0 2024-06-06T07:50:43.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fbb7fe700 1 -- 172.21.15.87:0/3125705061 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4011d10 con 0x7f7fc406b5c0 2024-06-06T07:50:43.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc419e010 con 0x7f7fc406b5c0 2024-06-06T07:50:43.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fbb7fe700 1 -- 172.21.15.87:0/3125705061 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400f2e0 con 0x7f7fc406b5c0 2024-06-06T07:50:43.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.459+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a5890 con 0x7f7fc406b5c0 2024-06-06T07:50:43.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.461+0000 7f7fbb7fe700 1 -- 172.21.15.87:0/3125705061 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7fb4018020 con 0x7f7fc406b5c0 2024-06-06T07:50:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.462+0000 7f7fbb7fe700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 0x7f7fb00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.462+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 0x7f7fb00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.462+0000 7f7fbb7fe700 1 -- 172.21.15.87:0/3125705061 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7fb40cedf0 con 0x7f7fc406b5c0 2024-06-06T07:50:43.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 0x7f7fb00a52d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000b5c0 tx=0x7f7fc0005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 Processor -- start 2024-06-06T07:50:43.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 -- start start 2024-06-06T07:50:43.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5600e702e810 con 0x5600e70dcf20 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5600e6fd6720 con 0x5600e70e2640 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.463+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600e702a7d0 con 0x5600e70e2f60 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35162/0 (socket says 172.21.15.87:35162) 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fca43c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:32800/0 (socket says 172.21.15.87:32800) 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 -- 172.21.15.87:0/8464290 learned_addr learned my addr 172.21.15.87:0/8464290 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 -- 172.21.15.87:0/8464290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 msgr2=0x5600e70dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 -- 172.21.15.87:0/8464290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 msgr2=0x5600e70e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 -- 172.21.15.87:0/8464290 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e6fccf50 con 0x5600e70dcf20 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6d6674ff10dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fb8ff9700 1 -- 172.21.15.87:0/8464290 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc004bb0 con 0x5600e70dcf20 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fb8ff9700 1 -- 172.21.15.87:0/8464290 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc004d50 con 0x5600e70dcf20 2024-06-06T07:50:43.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.464+0000 7f7fb8ff9700 1 -- 172.21.15.87:0/8464290 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc015480 con 0x5600e70dcf20 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/8464290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 msgr2=0x5600e70dd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc009c00 comp rx=0 tx=0).stop 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/8464290 shutdown_connections 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dd340 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/8464290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/8464290 >> 172.21.15.87:0/8464290 conn(0x5600e7007cc0 msgr2=0x5600e70d7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/8464290 shutdown_connections 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.465+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/8464290 wait complete. 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 -- start start 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 Processor -- start 2024-06-06T07:50:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5600e70e99d0 con 0x5600e70dcf20 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5600e6fcebf0 con 0x5600e70e2640 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600e70e9b50 con 0x5600e70e2f60 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fd6f23b80 10 client.0 _ll_register_callbacks cb 0x5600e6fc8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35166/0 (socket says 172.21.15.87:35166) 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 -- 172.21.15.87:0/4009756155 learned_addr learned my addr 172.21.15.87:0/4009756155 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 -- 172.21.15.87:0/4009756155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 msgr2=0x5600e70e64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 -- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 msgr2=0x5600e70e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 -- 172.21.15.87:0/4009756155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e6f7d8d0 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fca43c700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52699500d9c5f06c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.466+0000 7f7fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0024820 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7fc00249c0 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc004d50 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fc003e050 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 10 client.16140 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600e70f42e0 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc004d50 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc00173f0 con 0x5600e70dcf20 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 20 client.16140 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e70f6920 0x5600e70f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 4 client.16140 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.467+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600e703bd10 con 0x5600e70f6920 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.468+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e70f6920 0x5600e70f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.468+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e70f6920 0x5600e70f8dc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43ef54ce1b9539d4 server_cookie=328dff1b4697879b in_seq=0 out_seq=0 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.468+0000 7f7fadffb700 10 client.16140 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fadffb700 1 -- 172.21.15.87:0/4009756155 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600e703bd10 con 0x5600e70f6920 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fadffb700 10 client.16140 handle_command_reply: tid=0 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 client.16140 shutdown 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 msgr2=0x5600e70e8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc001a050 comp rx=0 tx=0).stop 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 shutdown_connections 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e70f6920 0x5600e70f8dc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70e8cb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/4009756155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2640 0x5600e70e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 >> 172.21.15.87:0/4009756155 conn(0x5600e7007cc0 msgr2=0x5600e70db9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fadffb700 0 client.16140 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:50:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 shutdown_connections 2024-06-06T07:50:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/4009756155 wait complete. 2024-06-06T07:50:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.469+0000 7f7fd6f23b80 20 client.16140 trim_cache size 0 max 16384 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 Processor -- start 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 -- start start 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2640 0x5600e70e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2f60 0x5600e70d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b0f80 con 0x5600e70dcf20 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ae810 con 0x5600e70e2f60 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.484+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ac660 con 0x5600e70e2640 2024-06-06T07:50:43.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35182/0 (socket says 172.21.15.87:35182) 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 -- 172.21.15.87:0/333883527 learned_addr learned my addr 172.21.15.87:0/333883527 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 -- 172.21.15.87:0/333883527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2640 msgr2=0x5600e70e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2640 0x5600e70e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2640 0x5600e70e3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 -- 172.21.15.87:0/333883527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2f60 msgr2=0x5600e70d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2f60 0x5600e70d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 -- 172.21.15.87:0/333883527 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e70d8500 con 0x5600e70dcf20 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7fca43c700 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00243e0 tx=0x7f7fc00037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7e7abcf35fec3e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7faffff700 1 -- 172.21.15.87:0/333883527 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00173f0 con 0x5600e70dcf20 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7faffff700 1 -- 172.21.15.87:0/333883527 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0035070 con 0x5600e70dcf20 2024-06-06T07:50:43.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.485+0000 7f7faffff700 1 -- 172.21.15.87:0/333883527 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc003a690 con 0x5600e70dcf20 2024-06-06T07:50:43.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/333883527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 msgr2=0x5600e70dc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00243e0 tx=0x7f7fc00037b0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/333883527 shutdown_connections 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70e2f60 0x5600e70d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70dcf20 0x5600e70dc9e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/333883527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2640 0x5600e70e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/333883527 >> 172.21.15.87:0/333883527 conn(0x5600e7007cc0 msgr2=0x5600e70d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/333883527 shutdown_connections 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/333883527 wait complete. 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- start start 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 Processor -- start 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70dcf20 0x5600e70ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41accd0 con 0x5600e70e2640 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ae140 con 0x5600e70dcf20 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.486+0000 7f7fd6f23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ad3a0 con 0x5600e70e2f60 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fd6f23b80 10 client.0 _ll_register_callbacks cb 0x5600e6fc8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35192/0 (socket says 172.21.15.87:35192) 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3040782238 learned_addr learned my addr 172.21.15.87:0/3040782238 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70dcf20 0x5600e70ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fcac3d700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3040782238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 msgr2=0x5600e70eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70dcf20 msgr2=0x5600e70ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70dcf20 0x5600e70ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 -- 172.21.15.87:0/3040782238 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e70eb200 con 0x5600e70e2640 2024-06-06T07:50:43.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fc9c3b700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4097450 tx=0x7f7fb409adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9fd2d199173667 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40b3030 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f7fb40105a0 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc0035070 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40ab050 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fd6f23b80 10 client.16146 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600e71614f0 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.487+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0035070 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7fb40118c0 con 0x5600e70e2640 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 20 client.16146 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:43.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e7161bb0 0x5600e7163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 4 client.16146 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 --> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600e7036290 con 0x5600e7161bb0 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e7161bb0 0x5600e7163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fca43c700 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e7161bb0 0x5600e7163fd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba3d9befdc9daf9a server_cookie=7556efc1e7d354af in_seq=0 out_seq=0 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.488+0000 7f7fadffb700 10 client.16146 ms_handle_connect on v2:172.21.15.181:6836/3077028235 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fadffb700 1 -- 172.21.15.87:0/3040782238 <== mds.2 v2:172.21.15.181:6836/3077028235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2538 (crc 0 0 0) 0x5600e7036290 con 0x5600e7161bb0 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fadffb700 10 client.16146 handle_command_reply: tid=0 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 client.16146 shutdown 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 msgr2=0x5600e70ea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4097450 tx=0x7f7fb409adc0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 shutdown_connections 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600e70dcf20 0x5600e70ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.181:6836/3077028235,v1:172.21.15.181:6837/3077028235] conn(0x5600e7161bb0 0x5600e7163fd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600e70e2f60 0x5600e70eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 --2- 172.21.15.87:0/3040782238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600e70e2640 0x5600e70ea6d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 >> 172.21.15.87:0/3040782238 conn(0x5600e7007cc0 msgr2=0x5600e70e1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fadffb700 0 client.16146 ms_handle_reset on v2:172.21.15.181:6836/3077028235 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 shutdown_connections 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 1 -- 172.21.15.87:0/3040782238 wait complete. 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.489+0000 7f7fd6f23b80 20 client.16146 trim_cache size 0 max 16384 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 32911579, 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628318, 2024-06-06T07:50:43.496 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 43353879, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628310, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 1036427688, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628311, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 1092659339, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628312, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 1229843501, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628314, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.498 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 1901789724, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628319, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 2018420713, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628320, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 2997056077, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628315, 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2024-06-06T07:50:43.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 3087001013, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628316, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 3088916915, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628313, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "damage_type": "backtrace", 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 3817306481, 2024-06-06T07:50:43.500 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 1099511628317, 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stdout: "path": "/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 msgr2=0x7f7fb00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 0x7f7fb00a52d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000b5c0 tx=0x7f7fc0005960 comp rx=0 tx=0).stop 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 msgr2=0x7f7fc4199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:43.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400b530 tx=0x7f7fb400b840 comp rx=0 tx=0).stop 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 shutdown_connections 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc406ac00 0x7f7fc4198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7fb00a2e10 0x7f7fb00a52d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4108530 0x7f7fc419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 --2- 172.21.15.87:0/3125705061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fc406b5c0 0x7f7fc4199250 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 >> 172.21.15.87:0/3125705061 conn(0x7f7fc4077400 msgr2=0x7f7fc4100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 shutdown_connections 2024-06-06T07:50:43.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:43.490+0000 7f7fcc6a0700 1 -- 172.21.15.87:0/3125705061 wait complete. 2024-06-06T07:50:43.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28505 3.67 MB/sec warmup 47 sec latency 3726.307 ms 2024-06-06T07:50:44.479 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:50:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:44 smithi181 ceph-mon[36790]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-06-06T07:50:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:44 smithi087 ceph-mon[32276]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-06-06T07:50:44.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28505 3.59 MB/sec warmup 48 sec latency 1809.257 ms 2024-06-06T07:50:44.723 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:44.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:44 smithi195 ceph-mon[37040]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-06-06T07:50:45.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28505 3.52 MB/sec warmup 49 sec latency 2809.316 ms 2024-06-06T07:50:46.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 -- 172.21.15.87:0/3650825881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102f00 msgr2=0x7f6fb410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 --2- 172.21.15.87:0/3650825881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102f00 0x7f6fb410f2c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009990 tx=0x7f6fac009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 -- 172.21.15.87:0/3650825881 shutdown_connections 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 --2- 172.21.15.87:0/3650825881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 0x7f6fb4108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 --2- 172.21.15.87:0/3650825881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102f00 0x7f6fb410f2c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 --2- 172.21.15.87:0/3650825881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102540 0x7f6fb41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 -- 172.21.15.87:0/3650825881 >> 172.21.15.87:0/3650825881 conn(0x7f6fb4079560 msgr2=0x7f6fb4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 -- 172.21.15.87:0/3650825881 shutdown_connections 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.365+0000 7f6fbb614700 1 -- 172.21.15.87:0/3650825881 wait complete. 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 Processor -- start 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 -- start start 2024-06-06T07:50:46.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102f00 0x7f6fb419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 0x7f6fb4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41127f0 con 0x7f6fb4102540 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4112670 con 0x7f6fb41085a0 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fbb614700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4112970 con 0x7f6fb4102f00 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.366+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35208/0 (socket says 172.21.15.87:35208) 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 -- 172.21.15.87:0/1721710184 learned_addr learned my addr 172.21.15.87:0/1721710184 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb9bb1700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 0x7f6fb4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102f00 0x7f6fb419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 -- 172.21.15.87:0/1721710184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102f00 msgr2=0x7f6fb419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102f00 0x7f6fb419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 -- 172.21.15.87:0/1721710184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 msgr2=0x7f6fb4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 0x7f6fb4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 -- 172.21.15.87:0/1721710184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb41990d0 con 0x7f6fb4102540 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007e20 tx=0x7f6fb000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6faa7fc700 1 -- 172.21.15.87:0/1721710184 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0016020 con 0x7f6fb4102540 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6faa7fc700 1 -- 172.21.15.87:0/1721710184 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fb000ae00 con 0x7f6fb4102540 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6faa7fc700 1 -- 172.21.15.87:0/1721710184 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011780 con 0x7f6fb4102540 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb41993c0 con 0x7f6fb4102540 2024-06-06T07:50:46.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.367+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41a5630 con 0x7f6fb4102540 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.369+0000 7f6faa7fc700 1 -- 172.21.15.87:0/1721710184 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6fb0019020 con 0x7f6fb4102540 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.370+0000 7f6faa7fc700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 0x7f6fa00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.370+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 0x7f6fa00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.370+0000 7f6faa7fc700 1 -- 172.21.15.87:0/1721710184 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6fb00cf4a0 con 0x7f6fb4102540 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 0x7f6fa00a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005ea0 tx=0x7f6fa4005e30 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 Processor -- start 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 -- start start 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d7249d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d72497bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561d72389f60 con 0x561d7249d880 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d723e8850 con 0x561d7249cf60 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.371+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d72350460 con 0x561d724977a0 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35224/0 (socket says 172.21.15.87:35224) 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 -- 172.21.15.87:0/2070942058 learned_addr learned my addr 172.21.15.87:0/2070942058 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb9bb1700 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d72497bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d7249d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 -- 172.21.15.87:0/2070942058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 msgr2=0x561d72497bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d72497bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 -- 172.21.15.87:0/2070942058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 msgr2=0x561d7249d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d7249d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 -- 172.21.15.87:0/2070942058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d723378d0 con 0x561d7249d880 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400a520 tx=0x7f6fa400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9c9567a5b82989 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6f9f7fe700 1 -- 172.21.15.87:0/2070942058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4021070 con 0x561d7249d880 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6f9f7fe700 1 -- 172.21.15.87:0/2070942058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4010cd0 con 0x561d7249d880 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.372+0000 7f6f9f7fe700 1 -- 172.21.15.87:0/2070942058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400f7b0 con 0x561d7249d880 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2070942058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 msgr2=0x561d72497260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400a520 tx=0x7f6fa400af90 comp rx=0 tx=0).stop 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2070942058 shutdown_connections 2024-06-06T07:50:46.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d7249d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d72497bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2070942058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249d880 0x561d72497260 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2070942058 >> 172.21.15.87:0/2070942058 conn(0x561d723c4080 msgr2=0x561d72492220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2070942058 shutdown_connections 2024-06-06T07:50:46.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.373+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2070942058 wait complete. 2024-06-06T07:50:46.376 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:46 smithi087 ceph-mon[32276]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 409 KiB/s wr, 105 op/s 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 -- start start 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 Processor -- start 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561d72388fc0 con 0x561d724977a0 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d723e4810 con 0x561d7249cf60 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d724a4260 con 0x561d7249d880 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fc5e97b80 10 client.0 _ll_register_callbacks cb 0x561d7237e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:46.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35232/0 (socket says 172.21.15.87:35232) 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2621847136 learned_addr learned my addr 172.21.15.87:0/2621847136 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.374+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb9bb1700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 msgr2=0x561d724a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2621847136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 msgr2=0x561d724a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2621847136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d723ae750 con 0x561d724977a0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007df0 tx=0x7f6fb0019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac25b3090d87a35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6f9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00b4060 con 0x561d724977a0 2024-06-06T07:50:46.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6fb00cf170 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa4010cd0 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fb009b7a0 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fc5e97b80 10 client.16158 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d724b2ee0 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.375+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4010cd0 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6fb0084ce0 con 0x561d724977a0 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 20 client.16158 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d724b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 4 client.16158 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d723f5d50 con 0x561d724b5450 2024-06-06T07:50:46.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d724b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d724b7880 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30f89bc55f93cd1f server_cookie=91755d641ab98ba7 in_seq=0 out_seq=0 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.376+0000 7f6f9cff9700 10 client.16158 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2621847136 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561d723f5d50 con 0x561d724b5450 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6f9cff9700 10 client.16158 handle_command_reply: tid=0 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 client.16158 shutdown 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 msgr2=0x561d724a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007df0 tx=0x7f6fb0019440 comp rx=0 tx=0).stop 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 shutdown_connections 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724a36d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d724b7880 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6f9cff9700 0 client.16158 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:50:46.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2621847136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.377+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 >> 172.21.15.87:0/2621847136 conn(0x561d723c4080 msgr2=0x561d72491700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.378+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 shutdown_connections 2024-06-06T07:50:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.378+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2621847136 wait complete. 2024-06-06T07:50:46.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.378+0000 7f6fc5e97b80 20 client.16158 trim_cache size 0 max 16384 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 Processor -- start 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 -- start start 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d724b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249d880 0x561d724a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41aeca0 con 0x561d7249cf60 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ac530 con 0x561d7249d880 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41aa380 con 0x561d724977a0 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35248/0 (socket says 172.21.15.87:35248) 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 -- 172.21.15.87:0/831831830 learned_addr learned my addr 172.21.15.87:0/831831830 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 -- 172.21.15.87:0/831831830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 msgr2=0x561d724b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d724b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d724b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 -- 172.21.15.87:0/831831830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249d880 msgr2=0x561d724a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249d880 0x561d724a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb8baf700 1 -- 172.21.15.87:0/831831830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7249e010 con 0x561d7249cf60 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.393+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d724b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.394+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400a520 tx=0x7f6fa4010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf3b2ae52e4c78b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.394+0000 7f6f9effd700 1 -- 172.21.15.87:0/831831830 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4021070 con 0x561d7249cf60 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.394+0000 7f6f9effd700 1 -- 172.21.15.87:0/831831830 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa402b070 con 0x561d7249cf60 2024-06-06T07:50:46.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.394+0000 7f6f9effd700 1 -- 172.21.15.87:0/831831830 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa401d450 con 0x561d7249cf60 2024-06-06T07:50:46.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/831831830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 msgr2=0x561d7249f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400a520 tx=0x7f6fa4010e80 comp rx=0 tx=0).stop 2024-06-06T07:50:46.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/831831830 shutdown_connections 2024-06-06T07:50:46.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249d880 0x561d724a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d724977a0 0x561d724b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/831831830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d7249cf60 0x561d7249f930 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/831831830 >> 172.21.15.87:0/831831830 conn(0x561d723c4080 msgr2=0x561d7249c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/831831830 shutdown_connections 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/831831830 wait complete. 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 -- start start 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.395+0000 7f6fc5e97b80 1 Processor -- start 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41aa9f0 con 0x561d724977a0 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41abe60 con 0x561d7249cf60 2024-06-06T07:50:46.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41ab0c0 con 0x561d7249d880 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fc5e97b80 10 client.0 _ll_register_callbacks cb 0x561d7237e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35264/0 (socket says 172.21.15.87:35264) 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2975558016 learned_addr learned my addr 172.21.15.87:0/2975558016 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb9bb1700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 msgr2=0x561d724bce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724bce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2975558016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 msgr2=0x561d724bc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724bc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 -- 172.21.15.87:0/2975558016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d724bd660 con 0x561d724977a0 2024-06-06T07:50:46.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb9bb1700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6fb93b0700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000f80 tx=0x7f6fb0012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254abe5807064e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.396+0000 7f6f9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00a4210 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6fb009adf0 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa402b070 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fb00b4020 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 10 client.16164 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d72520000 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa402b070 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6fb0084ce0 con 0x561d724977a0 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 10 client.16164 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:46.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 20 client.16164 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d725225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 4 client.16164 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.397+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 --> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d723f02d0 con 0x561d724b5450 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d725225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6f9cff9700 10 client.16164 ms_handle_connect on v2:172.21.15.195:6838/203076359 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fb8baf700 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d725225c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4712a94f18343f0 server_cookie=aa22f6fc92815632 in_seq=0 out_seq=0 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6f9cff9700 1 -- 172.21.15.87:0/2975558016 <== mds.1 v2:172.21.15.195:6838/203076359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d723f02d0 con 0x561d724b5450 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6f9cff9700 10 client.16164 handle_command_reply: tid=0 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 client.16164 shutdown 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 msgr2=0x561d724ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000f80 tx=0x7f6fb0012df0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 shutdown_connections 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d7249d880 0x561d724bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d724977a0 0x561d724ae0a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.195:6838/203076359,v1:172.21.15.195:6839/203076359] conn(0x561d724b5450 0x561d725225c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 --2- 172.21.15.87:0/2975558016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d7249cf60 0x561d724bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 >> 172.21.15.87:0/2975558016 conn(0x561d723c4080 msgr2=0x561d7249bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.398+0000 7f6f9cff9700 0 client.16164 ms_handle_reset on v2:172.21.15.195:6838/203076359 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 shutdown_connections 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fc5e97b80 1 -- 172.21.15.87:0/2975558016 wait complete. 2024-06-06T07:50:46.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fc5e97b80 20 client.16164 trim_cache size 0 max 16384 2024-06-06T07:50:46.405 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:50:46.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 msgr2=0x7f6fa00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 0x7f6fa00a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005ea0 tx=0x7f6fa4005e30 comp rx=0 tx=0).stop 2024-06-06T07:50:46.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 msgr2=0x7f6fb419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007e20 tx=0x7f6fb000a340 comp rx=0 tx=0).stop 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 shutdown_connections 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb41085a0 0x7f6fb4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fa00a2d90 0x7f6fa00a5250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fb4102540 0x7f6fb419e760 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 --2- 172.21.15.87:0/1721710184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fb4102f00 0x7f6fb419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 >> 172.21.15.87:0/1721710184 conn(0x7f6fb4079560 msgr2=0x7f6fb4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 shutdown_connections 2024-06-06T07:50:46.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:46.399+0000 7f6fbb614700 1 -- 172.21.15.87:0/1721710184 wait complete. 2024-06-06T07:50:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:46 smithi181 ceph-mon[36790]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 409 KiB/s wr, 105 op/s 2024-06-06T07:50:46.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28505 3.45 MB/sec warmup 50 sec latency 3809.354 ms 2024-06-06T07:50:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:46 smithi195 ceph-mon[37040]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 409 KiB/s wr, 105 op/s 2024-06-06T07:50:47.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:50:47.439 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:47.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28505 3.38 MB/sec warmup 51 sec latency 4809.442 ms 2024-06-06T07:50:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:48 smithi181 ceph-mon[36790]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 417 KiB/s wr, 121 op/s 2024-06-06T07:50:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:48 smithi087 ceph-mon[32276]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 417 KiB/s wr, 121 op/s 2024-06-06T07:50:48.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28575 3.32 MB/sec warmup 52 sec latency 4985.480 ms 2024-06-06T07:50:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:48 smithi195 ceph-mon[37040]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 417 KiB/s wr, 121 op/s 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.264+0000 7fd0159fc700 1 -- 172.21.15.87:0/1457568111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 msgr2=0x7fd0101077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.264+0000 7fd0159fc700 1 --2- 172.21.15.87:0/1457568111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd0101077e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd000009990 tx=0x7fd000009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 -- 172.21.15.87:0/1457568111 shutdown_connections 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 --2- 172.21.15.87:0/1457568111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101d50 0x7fd01010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 --2- 172.21.15.87:0/1457568111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd0101077e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 --2- 172.21.15.87:0/1457568111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101390 0x7fd010101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 -- 172.21.15.87:0/1457568111 >> 172.21.15.87:0/1457568111 conn(0x7fd010078340 msgr2=0x7fd010078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 -- 172.21.15.87:0/1457568111 shutdown_connections 2024-06-06T07:50:49.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 -- 172.21.15.87:0/1457568111 wait complete. 2024-06-06T07:50:49.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.265+0000 7fd0159fc700 1 Processor -- start 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 -- start start 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 0x7fd010103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101d50 0x7fd0101035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101116f0 con 0x7fd010107400 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd010111570 con 0x7fd010101d50 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd0159fc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd010111870 con 0x7fd010101390 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43094/0 (socket says 172.21.15.87:43094) 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 -- 172.21.15.87:0/3339093225 learned_addr learned my addr 172.21.15.87:0/3339093225 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00effd700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 0x7fd010103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101d50 0x7fd0101035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 -- 172.21.15.87:0/3339093225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 msgr2=0x7fd010103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 0x7fd010103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 -- 172.21.15.87:0/3339093225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101d50 msgr2=0x7fd0101035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101d50 0x7fd0101035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00f7fe700 1 -- 172.21.15.87:0/3339093225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0101a7190 con 0x7fd010107400 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.266+0000 7fd00effd700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 0x7fd010103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd00800ada0 tx=0x7fd00800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd0149fa700 1 -- 172.21.15.87:0/3339093225 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008017020 con 0x7fd010107400 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0101a7480 con 0x7fd010107400 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd0149fa700 1 -- 172.21.15.87:0/3339093225 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd008004bb0 con 0x7fd010107400 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd0149fa700 1 -- 172.21.15.87:0/3339093225 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008005520 con 0x7fd010107400 2024-06-06T07:50:49.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.267+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0101a7970 con 0x7fd010107400 2024-06-06T07:50:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.269+0000 7fd0149fa700 1 -- 172.21.15.87:0/3339093225 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd0080056c0 con 0x7fd010107400 2024-06-06T07:50:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.270+0000 7fd0149fa700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 0x7fcff80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.270+0000 7fd00effd700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 0x7fcff80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.270+0000 7fd0149fa700 1 -- 172.21.15.87:0/3339093225 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd0080cf020 con 0x7fd010107400 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.270+0000 7fd00effd700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 0x7fcff80a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd000006010 tx=0x7fd00000b540 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 Processor -- start 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 -- start start 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b37a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555563a83810 con 0x555563b31f20 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555563a2b720 con 0x555563b37640 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd02027fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555563a7f7d0 con 0x555563b37f60 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43102/0 (socket says 172.21.15.87:43102) 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 -- 172.21.15.87:0/841299539 learned_addr learned my addr 172.21.15.87:0/841299539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:49.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 -- 172.21.15.87:0/841299539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 msgr2=0x555563b319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00effd700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b37a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 -- 172.21.15.87:0/841299539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 msgr2=0x555563b37a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b37a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 -- 172.21.15.87:0/841299539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555563a21f50 con 0x555563b31f20 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00effd700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b37a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.271+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd008097bd0 tx=0x7fd00809b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1dfb27a231259e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fcff5ffb700 1 -- 172.21.15.87:0/841299539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080b4070 con 0x555563b31f20 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fcff5ffb700 1 -- 172.21.15.87:0/841299539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00809b200 con 0x555563b31f20 2024-06-06T07:50:49.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fcff5ffb700 1 -- 172.21.15.87:0/841299539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004e80 con 0x555563b31f20 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 -- 172.21.15.87:0/841299539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 msgr2=0x555563b32340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd008097bd0 tx=0x7fd00809b100 comp rx=0 tx=0).stop 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 -- 172.21.15.87:0/841299539 shutdown_connections 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b37a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b32340 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 --2- 172.21.15.87:0/841299539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 -- 172.21.15.87:0/841299539 >> 172.21.15.87:0/841299539 conn(0x555563a5ccc0 msgr2=0x555563b2c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 -- 172.21.15.87:0/841299539 shutdown_connections 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.272+0000 7fd02027fb80 1 -- 172.21.15.87:0/841299539 wait complete. 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 -- start start 2024-06-06T07:50:49.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 Processor -- start 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b3b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555563b3e9d0 con 0x555563b31f20 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555563a23bf0 con 0x555563b37f60 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555563b3eb50 con 0x555563b37640 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd02027fb80 10 client.0 _ll_register_callbacks cb 0x555563a1d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43104/0 (socket says 172.21.15.87:43104) 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd00effd700 1 -- 172.21.15.87:0/2498870053 learned_addr learned my addr 172.21.15.87:0/2498870053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.273+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 -- 172.21.15.87:0/2498870053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 msgr2=0x555563b3b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b3b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 -- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 msgr2=0x555563b3b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b3b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 -- 172.21.15.87:0/2498870053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555639d28d0 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd00effd700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd000009990 tx=0x7fd000009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436ca4779591fccd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000fe30 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd000003960 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00809b200 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd000003b60 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 10 client.16176 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555563b49340 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00809b200 con 0x555563b31f20 2024-06-06T07:50:49.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd000030780 con 0x555563b31f20 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.274+0000 7fd02027fb80 20 client.16176 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563b4b920 0x555563b4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fd02027fb80 4 client.16176 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555563a90d10 con 0x555563b4b920 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563b4b920 0x555563b4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fcfeeffd700 10 client.16176 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.275+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563b4b920 0x555563b4dd70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3668b4b54f40e3 server_cookie=86b01f620efa8e07 in_seq=0 out_seq=0 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fcfeeffd700 1 -- 172.21.15.87:0/2498870053 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555563a90d10 con 0x555563b4b920 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fcfeeffd700 10 client.16176 handle_command_reply: tid=0 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 client.16176 shutdown 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 msgr2=0x555563b3dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd000009990 tx=0x7fd000009ca0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 shutdown_connections 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563b4b920 0x555563b4dd70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fcfeeffd700 0 client.16176 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b3dd60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 --2- 172.21.15.87:0/2498870053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b3b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 >> 172.21.15.87:0/2498870053 conn(0x555563a5ccc0 msgr2=0x555563b309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 shutdown_connections 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 1 -- 172.21.15.87:0/2498870053 wait complete. 2024-06-06T07:50:49.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.276+0000 7fd02027fb80 20 client.16176 trim_cache size 0 max 16384 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 Processor -- start 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 -- start start 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b3160 con 0x555563b31f20 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b09f0 con 0x555563b37f60 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.291+0000 7fd02027fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101ae840 con 0x555563b37640 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43106/0 (socket says 172.21.15.87:43106) 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 -- 172.21.15.87:0/4266948711 learned_addr learned my addr 172.21.15.87:0/4266948711 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 -- 172.21.15.87:0/4266948711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 msgr2=0x555563b319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b319e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 -- 172.21.15.87:0/4266948711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 msgr2=0x555563b26690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 -- 172.21.15.87:0/4266948711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555563b4e3f0 con 0x555563b31f20 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fd00effd700 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd0000098e0 tx=0x7fd00000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7e8c2baa4fc805 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fcff57fa700 1 -- 172.21.15.87:0/4266948711 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000030410 con 0x555563b31f20 2024-06-06T07:50:49.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fcff57fa700 1 -- 172.21.15.87:0/4266948711 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd000028040 con 0x555563b31f20 2024-06-06T07:50:49.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.292+0000 7fcff57fa700 1 -- 172.21.15.87:0/4266948711 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000024790 con 0x555563b31f20 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- 172.21.15.87:0/4266948711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 msgr2=0x555563b47c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd0000098e0 tx=0x7fd00000f8a0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- 172.21.15.87:0/4266948711 shutdown_connections 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37f60 0x555563b26690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b47c70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 --2- 172.21.15.87:0/4266948711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37640 0x555563b319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- 172.21.15.87:0/4266948711 >> 172.21.15.87:0/4266948711 conn(0x555563a5ccc0 msgr2=0x555563b389b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- 172.21.15.87:0/4266948711 shutdown_connections 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- 172.21.15.87:0/4266948711 wait complete. 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 -- start start 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.293+0000 7fd02027fb80 1 Processor -- start 2024-06-06T07:50:49.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101aeeb0 con 0x555563b31f20 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b0320 con 0x555563b37640 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101af580 con 0x555563b37f60 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 10 client.0 _ll_register_callbacks cb 0x7fd00000bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43114/0 (socket says 172.21.15.87:43114) 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 -- 172.21.15.87:0/495546650 learned_addr learned my addr 172.21.15.87:0/495546650 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 -- 172.21.15.87:0/495546650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 msgr2=0x555563b531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 -- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 msgr2=0x555563b52cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b52cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 -- 172.21.15.87:0/495546650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555563b53730 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00f7fe700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd00effd700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd000009ce0 tx=0x7fd000005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7818a5b7d3eed29f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fcfeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000043040 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd0000306d0 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.294+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd000028040 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0000308d0 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 10 client.16182 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555563bb6340 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd000028040 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd000037020 con 0x555563b31f20 2024-06-06T07:50:49.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 20 client.16182 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563bb6930 0x555563bb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 4 client.16182 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555563a8b290 con 0x555563bb6930 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.295+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563bb6930 0x555563bb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd00e7fc700 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563bb6930 0x555563bb8d50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc70ee7f82c6dda server_cookie=a7b6ac3ef4f4c06f in_seq=0 out_seq=0 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fcfeeffd700 10 client.16182 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fcfeeffd700 1 -- 172.21.15.87:0/495546650 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555563a8b290 con 0x555563bb6930 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fcfeeffd700 10 client.16182 handle_command_reply: tid=0 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 client.16182 shutdown 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 msgr2=0x555563b44240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd000009ce0 tx=0x7fd000005810 comp rx=0 tx=0).stop 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 shutdown_connections 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555563bb6930 0x555563bb8d50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555563b37f60 0x555563b531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555563b31f20 0x555563b44240 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fcfeeffd700 0 client.16182 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 --2- 172.21.15.87:0/495546650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555563b37640 0x555563b52cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 >> 172.21.15.87:0/495546650 conn(0x555563a5ccc0 msgr2=0x555563b39970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 shutdown_connections 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 1 -- 172.21.15.87:0/495546650 wait complete. 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.296+0000 7fd02027fb80 20 client.16182 trim_cache size 0 max 16384 2024-06-06T07:50:49.301 INFO:teuthology.orchestra.run.smithi087.stdout:[] 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 msgr2=0x7fcff80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 0x7fcff80a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd000006010 tx=0x7fd00000b540 comp rx=0 tx=0).stop 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 msgr2=0x7fd010103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd00800ada0 tx=0x7fd00800c330 comp rx=0 tx=0).stop 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 shutdown_connections 2024-06-06T07:50:49.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcff80a2d90 0x7fcff80a5250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd010107400 0x7fd010103af0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd010101390 0x7fd010103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 --2- 172.21.15.87:0/3339093225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd010101d50 0x7fd0101035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 >> 172.21.15.87:0/3339093225 conn(0x7fd010078340 msgr2=0x7fd010100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 shutdown_connections 2024-06-06T07:50:49.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:49.297+0000 7fd0159fc700 1 -- 172.21.15.87:0/3339093225 wait complete. 2024-06-06T07:50:49.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28575 3.26 MB/sec warmup 53 sec latency 1764.251 ms 2024-06-06T07:50:50.048 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 32911579, 'ino': 1099511628318, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 43353879, 'ino': 1099511628310, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 1036427688, 'ino': 1099511628311, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 1092659339, 'ino': 1099511628312, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 1229843501, 'ino': 1099511628314, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 1901789724, 'ino': 1099511628319, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2018420713, 'ino': 1099511628320, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 2997056077, 'ino': 1099511628315, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 3087001013, 'ino': 1099511628316, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 3088916915, 'ino': 1099511628313, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 3817306481, 'ino': 1099511628317, 'path': '/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}] 2024-06-06T07:50:50.048 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-06-06T07:50:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:50 smithi181 ceph-mon[36790]: pgmap v87: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:50 smithi087 ceph-mon[32276]: pgmap v87: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:50.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28575 3.20 MB/sec warmup 54 sec latency 2764.335 ms 2024-06-06T07:50:50.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:50 smithi195 ceph-mon[37040]: pgmap v87: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:51.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28575 3.14 MB/sec warmup 55 sec latency 3764.441 ms 2024-06-06T07:50:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:52 smithi087 ceph-mon[32276]: pgmap v88: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:52.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 28575 3.08 MB/sec warmup 56 sec latency 4764.560 ms 2024-06-06T07:50:52.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:52 smithi195 ceph-mon[37040]: pgmap v88: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:52.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:52 smithi181 ceph-mon[36790]: pgmap v88: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 220 KiB/s wr, 93 op/s 2024-06-06T07:50:53.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 29637 3.09 MB/sec warmup 57 sec latency 4939.694 ms 2024-06-06T07:50:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 -- 172.21.15.87:0/2345653554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 msgr2=0x7f4608106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 --2- 172.21.15.87:0/2345653554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f4608106c70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4600009930 tx=0x7f460002a420 comp rx=0 tx=0).stop 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 -- 172.21.15.87:0/2345653554 shutdown_connections 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 --2- 172.21.15.87:0/2345653554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4608101290 0x7f460810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 --2- 172.21.15.87:0/2345653554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46081008d0 0x7f4608100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 --2- 172.21.15.87:0/2345653554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f4608106c70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 -- 172.21.15.87:0/2345653554 >> 172.21.15.87:0/2345653554 conn(0x7f4608077a20 msgr2=0x7f4608077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 -- 172.21.15.87:0/2345653554 shutdown_connections 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.992+0000 7f460cbc1640 1 -- 172.21.15.87:0/2345653554 wait complete. 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 Processor -- start 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 -- start start 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46081008d0 0x7f46081970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4608101290 0x7f4608197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4608110cb0 con 0x7f4608106890 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4608110b30 con 0x7f4608101290 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460cbc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4608110e30 con 0x7f46081008d0 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43136/0 (socket says 172.21.15.87:43136) 2024-06-06T07:50:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 -- 172.21.15.87:0/4068327975 learned_addr learned my addr 172.21.15.87:0/4068327975 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f460659c640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46081008d0 0x7f46081970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4605d9b640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4608101290 0x7f4608197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 -- 172.21.15.87:0/4068327975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46081008d0 msgr2=0x7f46081970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46081008d0 0x7f46081970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 -- 172.21.15.87:0/4068327975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4608101290 msgr2=0x7f4608197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4608101290 0x7f4608197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.993+0000 7f4606d9d640 1 -- 172.21.15.87:0/4068327975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460819c0e0 con 0x7f4608106890 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f4606d9d640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f45f80098a0 tx=0x7f45f8009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f45ff7fe640 1 -- 172.21.15.87:0/4068327975 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800d030 con 0x7f4608106890 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460819c3d0 con 0x7f4608106890 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f45ff7fe640 1 -- 172.21.15.87:0/4068327975 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f800d1d0 con 0x7f4608106890 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f45ff7fe640 1 -- 172.21.15.87:0/4068327975 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8003850 con 0x7f4608106890 2024-06-06T07:50:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.994+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4608069050 con 0x7f4608106890 2024-06-06T07:50:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f45ff7fe640 1 -- 172.21.15.87:0/4068327975 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f45f800c460 con 0x7f4608106890 2024-06-06T07:50:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 Processor -- start 2024-06-06T07:50:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 -- start start 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2708fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270908e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d2708624f0 con 0x55d270909350 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d270800870 con 0x55d2708fe170 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460f9ae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d27085e4b0 con 0x55d2708fea90 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f460659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270908e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.997+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43144/0 (socket says 172.21.15.87:43144) 2024-06-06T07:50:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 -- 172.21.15.87:0/113350558 learned_addr learned my addr 172.21.15.87:0/113350558 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2708fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f45ff7fe640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 0x7f45f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4606d9d640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 0x7f45f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 -- 172.21.15.87:0/113350558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 msgr2=0x55d270908e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270908e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 -- 172.21.15.87:0/113350558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 msgr2=0x55d2708fe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2708fe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 -- 172.21.15.87:0/113350558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27081c9b0 con 0x55d270909350 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f460659c640 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270908e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f45ff7fe640 1 -- 172.21.15.87:0/4068327975 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f45f80d0d80 con 0x7f4608106890 2024-06-06T07:50:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4605d9b640 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005f40 tx=0x7f45f0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c5843a3fc5f392 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f45fcff9640 1 -- 172.21.15.87:0/113350558 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000fc50 con 0x55d270909350 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f4606d9d640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 0x7f45f40a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ce1b0 tx=0x7f45f806ff40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f45fcff9640 1 -- 172.21.15.87:0/113350558 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0003e50 con 0x55d270909350 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.998+0000 7f45fcff9640 1 -- 172.21.15.87:0/113350558 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f001e3f0 con 0x55d270909350 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- 172.21.15.87:0/113350558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 msgr2=0x55d27090b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005f40 tx=0x7f45f0005bd0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- 172.21.15.87:0/113350558 shutdown_connections 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090b740 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270908e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 --2- 172.21.15.87:0/113350558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2708fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- 172.21.15.87:0/113350558 >> 172.21.15.87:0/113350558 conn(0x55d2708fa680 msgr2=0x55d2708faa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- 172.21.15.87:0/113350558 shutdown_connections 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- 172.21.15.87:0/113350558 wait complete. 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 -- start start 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:53.999+0000 7f460f9ae740 1 Processor -- start 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2709063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270906930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d27090bf80 con 0x55d270909350 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d2707ebae0 con 0x55d2708fe170 2024-06-06T07:50:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d27090c100 con 0x55d2708fea90 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 10 client.0 _ll_register_callbacks cb 0x55d2708249b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43146/0 (socket says 172.21.15.87:43146) 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270906930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f4605d9b640 1 -- 172.21.15.87:0/681000327 learned_addr learned my addr 172.21.15.87:0/681000327 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f4605d9b640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2709063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 -- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 msgr2=0x55d2709063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2709063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 -- 172.21.15.87:0/681000327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 msgr2=0x55d270906e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 -- 172.21.15.87:0/681000327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d270576210 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f4605d9b640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460659c640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270906930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f460002a940 tx=0x7f46000043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f21e0e0b61a355 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600035040 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f0003e50 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.000+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4600005450 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4600005650 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 10 client.25573 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d270642620 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0003e50 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4600003a00 con 0x55d2708fea90 2024-06-06T07:50:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 10 client.25573 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 20 client.25573 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270919610 0x55d27091ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 4 client.25573 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d270875f10 con 0x55d270919610 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.001+0000 7f4605d9b640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270919610 0x55d27091ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.002+0000 7f45edffb640 10 client.25573 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.002+0000 7f4605d9b640 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270919610 0x55d27091ba20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05b161c96a7bbd3 server_cookie=2833b3a113ddfc13 in_seq=0 out_seq=0 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.002+0000 7f45edffb640 1 -- 172.21.15.87:0/681000327 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d270875f10 con 0x55d270919610 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.002+0000 7f45edffb640 10 client.25573 handle_command_reply: tid=0 2024-06-06T07:50:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.002+0000 7f460f9ae740 1 client.25573 shutdown 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 msgr2=0x55d270906930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270906930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f460002a940 tx=0x7f46000043b0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 shutdown_connections 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270919610 0x55d27091ba20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270906e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f45edffb640 0 client.25573 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270906930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 --2- 172.21.15.87:0/681000327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d2709063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 >> 172.21.15.87:0/681000327 conn(0x55d2708fa680 msgr2=0x55d2709003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 shutdown_connections 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 1 -- 172.21.15.87:0/681000327 wait complete. 2024-06-06T07:50:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.003+0000 7f460f9ae740 20 client.25573 trim_cache size 0 max 16384 2024-06-06T07:50:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 Processor -- start 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 -- start start 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d270900b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270921330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f46081af510 con 0x55d270909350 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46081acde0 con 0x55d2708fe170 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460f9ae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46081aac30 con 0x55d2708fea90 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43152/0 (socket says 172.21.15.87:43152) 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 -- 172.21.15.87:0/1449312669 learned_addr learned my addr 172.21.15.87:0/1449312669 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f460659c640 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270921330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d270900b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 -- 172.21.15.87:0/1449312669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 msgr2=0x55d270921330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270921330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 -- 172.21.15.87:0/1449312669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 msgr2=0x55d270900b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d270900b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 -- 172.21.15.87:0/1449312669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2708ff420 con 0x55d270909350 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.018+0000 7f4605d9b640 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45f00044e0 tx=0x7f45f00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848c82e5b79b9a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f45effff640 1 -- 172.21.15.87:0/1449312669 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0032070 con 0x55d270909350 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f45effff640 1 -- 172.21.15.87:0/1449312669 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0033040 con 0x55d270909350 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f45effff640 1 -- 172.21.15.87:0/1449312669 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0011230 con 0x55d270909350 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- 172.21.15.87:0/1449312669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 msgr2=0x55d270921870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45f00044e0 tx=0x7f45f00045b0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- 172.21.15.87:0/1449312669 shutdown_connections 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d270921870 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fea90 0x55d270921330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 --2- 172.21.15.87:0/1449312669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fe170 0x55d270900b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- 172.21.15.87:0/1449312669 >> 172.21.15.87:0/1449312669 conn(0x55d2708fa680 msgr2=0x55d2708f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- 172.21.15.87:0/1449312669 shutdown_connections 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- 172.21.15.87:0/1449312669 wait complete. 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 -- start start 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.019+0000 7f460f9ae740 1 Processor -- start 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fe170 0x55d270911c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 0x55d27090cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ab2a0 con 0x55d270909350 2024-06-06T07:50:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ac710 con 0x55d2708fea90 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ab970 con 0x55d2708fe170 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460f9ae740 10 client.0 _ll_register_callbacks cb 0x7f4600009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 0x55d27090cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43160/0 (socket says 172.21.15.87:43160) 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f460659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 0x55d27090cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:52124/0 (socket says 172.21.15.87:52124) 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 -- 172.21.15.87:0/3527486779 learned_addr learned my addr 172.21.15.87:0/3527486779 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fe170 0x55d270911c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 -- 172.21.15.87:0/3527486779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fe170 msgr2=0x55d270911c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fe170 0x55d270911c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 -- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 msgr2=0x55d27090cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 0x55d27090cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 -- 172.21.15.87:0/3527486779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d270983300 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f4605d9b640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45f0002230 tx=0x7f45f0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d5b71ae27d299c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.020+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0032070 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f45f000fda0 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f0033040 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45f00049a0 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 10 client.16200 fetch_fsmap learned FSMap version 30 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2709844d0 con 0x55d270909350 2024-06-06T07:50:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0033040 con 0x55d270909350 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f45f001d030 con 0x55d270909350 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 10 client.16200 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 20 client.16200 populate_metadata read hostname 'smithi087' 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270984980 0x55d270986da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 4 client.16200 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d27086fc20 con 0x55d270984980 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.021+0000 7f460659c640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270984980 0x55d270986da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.022+0000 7f45edffb640 10 client.16200 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.022+0000 7f460659c640 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270984980 0x55d270986da0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0126466eeb4cea2 server_cookie=97b2933b3af868d9 in_seq=0 out_seq=0 2024-06-06T07:50:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f45edffb640 1 -- 172.21.15.87:0/3527486779 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d27086fc20 con 0x55d270984980 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f45edffb640 10 client.16200 handle_command_reply: tid=0 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 client.16200 shutdown 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 msgr2=0x55d27090d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45f0002230 tx=0x7f45f0009f10 comp rx=0 tx=0).stop 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 shutdown_connections 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d270984980 0x55d270986da0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d270909350 0x55d27090d730 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2708fea90 0x55d27090cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f45edffb640 0 client.16200 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 --2- 172.21.15.87:0/3527486779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2708fe170 0x55d270911c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 >> 172.21.15.87:0/3527486779 conn(0x55d2708fa680 msgr2=0x55d270906390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 shutdown_connections 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 1 -- 172.21.15.87:0/3527486779 wait complete. 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.069+0000 7f460f9ae740 20 client.16200 trim_cache size 0 max 16384 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:50:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:50:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 msgr2=0x7f45f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 0x7f45f40a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ce1b0 tx=0x7f45f806ff40 comp rx=0 tx=0).stop 2024-06-06T07:50:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 msgr2=0x7f460819b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f45f80098a0 tx=0x7f45f8009d60 comp rx=0 tx=0).stop 2024-06-06T07:50:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 shutdown_connections 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45f40a2da0 0x7f45f40a5260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4608106890 0x7f460819b9b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4608101290 0x7f4608197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 --2- 172.21.15.87:0/4068327975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46081008d0 0x7f46081970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 >> 172.21.15.87:0/4068327975 conn(0x7f4608077a20 msgr2=0x7f4608103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 shutdown_connections 2024-06-06T07:50:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:50:54.070+0000 7f460cbc1640 1 -- 172.21.15.87:0/4068327975 wait complete. 2024-06-06T07:50:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:54 smithi087 ceph-mon[32276]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 228 KiB/s wr, 98 op/s 2024-06-06T07:50:54.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 32520 3.33 MB/sec warmup 58 sec latency 22.473 ms 2024-06-06T07:50:54.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:54 smithi195 ceph-mon[37040]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 228 KiB/s wr, 98 op/s 2024-06-06T07:50:54.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:54 smithi181 ceph-mon[36790]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 228 KiB/s wr, 98 op/s 2024-06-06T07:50:55.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 35338 3.52 MB/sec warmup 59 sec latency 23.761 ms 2024-06-06T07:50:56.607 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:56 smithi181 ceph-mon[36790]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-06T07:50:56.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:56 smithi087 ceph-mon[32276]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-06T07:50:56.652 INFO:tasks.workunit.client.0.smithi087.stdout: 1 37424 3.63 MB/sec warmup 60 sec latency 15.904 ms 2024-06-06T07:50:56.755 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-06-06T07:50:56.755 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:50:56.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:56 smithi195 ceph-mon[37040]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-06T07:50:57.015 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:50:57.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 39875 3.80 MB/sec warmup 61 sec latency 23.046 ms 2024-06-06T07:50:58.584 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:58 smithi087 ceph-mon[32276]: pgmap v91: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-06T07:50:58.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 42974 4.04 MB/sec warmup 62 sec latency 7.138 ms 2024-06-06T07:50:58.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:58 smithi195 ceph-mon[37040]: pgmap v91: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-06T07:50:58.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:58 smithi181 ceph-mon[36790]: pgmap v91: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-06T07:50:58.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/4216882221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78103120 msgr2=0x7f5b78107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/4216882221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78103120 0x7f5b78107130 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0099c0 tx=0x7f5b6c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/4216882221 shutdown_connections 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/4216882221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78102770 0x7f5b78102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/4216882221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78107860 0x7f5b7810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/4216882221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78103120 0x7f5b78107130 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/4216882221 >> 172.21.15.87:0/4216882221 conn(0x7f5b78079600 msgr2=0x7f5b78079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.970+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/4216882221 shutdown_connections 2024-06-06T07:50:58.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.971+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/4216882221 wait complete. 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.971+0000 7f5b7e0f5700 1 Processor -- start 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.971+0000 7f5b7e0f5700 1 -- start start 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78103120 0x7f5b7819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78107860 0x7f5b78198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78112770 con 0x7f5b78102770 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781125f0 con 0x7f5b78107860 2024-06-06T07:50:58.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781128f0 con 0x7f5b78103120 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60082/0 (socket says 172.21.15.87:60082) 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 -- 172.21.15.87:0/3533237180 learned_addr learned my addr 172.21.15.87:0/3533237180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b77fff700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78107860 0x7f5b78198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b76ffd700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78103120 0x7f5b7819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 -- 172.21.15.87:0/3533237180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78103120 msgr2=0x7f5b7819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78103120 0x7f5b7819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 -- 172.21.15.87:0/3533237180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78107860 msgr2=0x7f5b78198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78107860 0x7f5b78198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 -- 172.21.15.87:0/3533237180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b781991b0 con 0x7f5b78102770 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b777fe700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e9a0 tx=0x7f5b6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:58.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800ca10 con 0x7f5b78102770 2024-06-06T07:50:58.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b781994a0 con 0x7f5b78102770 2024-06-06T07:50:58.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781a5710 con 0x7f5b78102770 2024-06-06T07:50:58.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6800cbb0 con 0x7f5b78102770 2024-06-06T07:50:58.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.972+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68018680 con 0x7f5b78102770 2024-06-06T07:50:58.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.973+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b78069180 con 0x7f5b78102770 2024-06-06T07:50:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.975+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b68010430 con 0x7f5b78102770 2024-06-06T07:50:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.976+0000 7f5b74ff9700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 0x7f5b600a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:50:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.976+0000 7f5b76ffd700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 0x7f5b600a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:50:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.976+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f5b680cfe90 con 0x7f5b78102770 2024-06-06T07:50:58.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.976+0000 7f5b76ffd700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 0x7f5b600a52d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b78100fb0 tx=0x7f5b6c005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:50:58.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:58.979+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b6809c6e0 con 0x7f5b78102770 2024-06-06T07:50:59.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.256+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b78199f30 con 0x7f5b78102770 2024-06-06T07:50:59.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.257+0000 7f5b74ff9700 1 -- 172.21.15.87:0/3533237180 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11877 (secure 0 0 0) 0x7f5b6809bfe0 con 0x7f5b78102770 2024-06-06T07:50:59.257 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:50:59.257 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:50:06.260840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 msgr2=0x7f5b600a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 0x7f5b600a52d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b78100fb0 tx=0x7f5b6c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 msgr2=0x7f5b7819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e9a0 tx=0x7f5b6800ecb0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 shutdown_connections 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b78107860 0x7f5b78198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5b600a2e10 0x7f5b600a52d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b78102770 0x7f5b7819e840 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.259+0000 7f5b7e0f5700 1 --2- 172.21.15.87:0/3533237180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b78103120 0x7f5b7819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.260+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 >> 172.21.15.87:0/3533237180 conn(0x7f5b78079600 msgr2=0x7f5b78101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.260+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 shutdown_connections 2024-06-06T07:50:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:50:59.260+0000 7f5b7e0f5700 1 -- 172.21.15.87:0/3533237180 wait complete. 2024-06-06T07:50:59.261 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 30 2024-06-06T07:50:59.592 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:50:59 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3533237180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:59.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 45371 4.15 MB/sec warmup 63 sec latency 35.919 ms 2024-06-06T07:50:59.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:50:59 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3533237180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:59.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:50:59 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3533237180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:50:59.935 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-06-06T07:50:59.936 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-06-06T07:51:00.186 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:51:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:00 smithi087 ceph-mon[32276]: pgmap v92: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:00.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 47162 4.23 MB/sec warmup 64 sec latency 41.440 ms 2024-06-06T07:51:00.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:00 smithi195 ceph-mon[37040]: pgmap v92: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:00.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:00 smithi181 ceph-mon[36790]: pgmap v92: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:01.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 48717 4.27 MB/sec warmup 65 sec latency 53.774 ms 2024-06-06T07:51:02.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 -- 172.21.15.87:0/4266939487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208107460 msgr2=0x7f4208107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 --2- 172.21.15.87:0/4266939487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208107460 0x7f4208107840 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4204009990 tx=0x7f4204009ca0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 -- 172.21.15.87:0/4266939487 shutdown_connections 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 --2- 172.21.15.87:0/4266939487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208101d70 0x7f420810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 --2- 172.21.15.87:0/4266939487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208107460 0x7f4208107840 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 --2- 172.21.15.87:0/4266939487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f4208101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 -- 172.21.15.87:0/4266939487 >> 172.21.15.87:0/4266939487 conn(0x7f42080783b0 msgr2=0x7f42080787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 -- 172.21.15.87:0/4266939487 shutdown_connections 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.137+0000 7f420ff58700 1 -- 172.21.15.87:0/4266939487 wait complete. 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 Processor -- start 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 -- start start 2024-06-06T07:51:02.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208101d70 0x7f420819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208107460 0x7f4208198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111680 con 0x7f4208101d70 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111500 con 0x7f42081013b0 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420ff58700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111800 con 0x7f4208107460 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208101d70 0x7f420819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:52206/0 (socket says 172.21.15.87:52206) 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420dcf4700 1 -- 172.21.15.87:0/1180442230 learned_addr learned my addr 172.21.15.87:0/1180442230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.138+0000 7f420e4f5700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208107460 0x7f4208198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 -- 172.21.15.87:0/1180442230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208107460 msgr2=0x7f4208198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208107460 0x7f4208198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 -- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208101d70 msgr2=0x7f420819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208101d70 0x7f420819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 -- 172.21.15.87:0/1180442230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42081990e0 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420dcf4700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f420400b2e0 tx=0x7f420400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f41feffd700 1 -- 172.21.15.87:0/1180442230 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204026070 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4208199370 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f41feffd700 1 -- 172.21.15.87:0/1180442230 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4204003d70 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f41feffd700 1 -- 172.21.15.87:0/1180442230 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204021410 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.139+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42081a5690 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.142+0000 7f41feffd700 1 -- 172.21.15.87:0/1180442230 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4204029070 con 0x7f42081013b0 2024-06-06T07:51:02.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.142+0000 7f41feffd700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 0x7f41f40a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.142+0000 7f420d4f3700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 0x7f41f40a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.142+0000 7f41feffd700 1 -- 172.21.15.87:0/1180442230 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f42040d0590 con 0x7f42081013b0 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f420d4f3700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 0x7f41f40a52f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4208078990 tx=0x7f41f8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 Processor -- start 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 -- start start 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56445b3119a0 con 0x56445b3ca0e0 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56445b2b98b0 con 0x56445b3c4870 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.143+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56445b30d960 con 0x56445b3caa00 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54514/0 (socket says 172.21.15.87:54514) 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60126/0 (socket says 172.21.15.87:60126) 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 -- 172.21.15.87:0/2831967685 learned_addr learned my addr 172.21.15.87:0/2831967685 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420e4f5700 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 -- 172.21.15.87:0/2831967685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 msgr2=0x56445b3c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 -- 172.21.15.87:0/2831967685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 msgr2=0x56445b3ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 -- 172.21.15.87:0/2831967685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445b2b00e0 con 0x56445b3caa00 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420dcf4700 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f420d4f3700 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011d50 tx=0x7f41f8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686e369815cd434d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f41f3fff700 1 -- 172.21.15.87:0/2831967685 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800eb20 con 0x56445b3caa00 2024-06-06T07:51:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f41f3fff700 1 -- 172.21.15.87:0/2831967685 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f800ecc0 con 0x56445b3caa00 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.144+0000 7f41f3fff700 1 -- 172.21.15.87:0/2831967685 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f801d770 con 0x56445b3caa00 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 -- 172.21.15.87:0/2831967685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 msgr2=0x56445b3c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011d50 tx=0x7f41f8014000 comp rx=0 tx=0).stop 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 -- 172.21.15.87:0/2831967685 shutdown_connections 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/2831967685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3c4330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 -- 172.21.15.87:0/2831967685 >> 172.21.15.87:0/2831967685 conn(0x56445b2eae50 msgr2=0x56445b3bb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 -- 172.21.15.87:0/2831967685 shutdown_connections 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.145+0000 7f421a7dbb80 1 -- 172.21.15.87:0/2831967685 wait complete. 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 -- start start 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 Processor -- start 2024-06-06T07:51:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56445b3d1470 con 0x56445b3ca0e0 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56445b2b1d80 con 0x56445b3c4870 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56445b3d15f0 con 0x56445b3caa00 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f421a7dbb80 10 client.0 _ll_register_callbacks cb 0x56445b2abab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60128/0 (socket says 172.21.15.87:60128) 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f420d4f3700 1 -- 172.21.15.87:0/3873054196 learned_addr learned my addr 172.21.15.87:0/3873054196 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.146+0000 7f420e4f5700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 -- 172.21.15.87:0/3873054196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 msgr2=0x56445b3cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 -- 172.21.15.87:0/3873054196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 msgr2=0x56445b3cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 -- 172.21.15.87:0/3873054196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445b260a60 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420d4f3700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d0790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42040cd910 tx=0x7f42040a3120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa746e67da29de19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f41f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040a0b20 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f800ecc0 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f420400f6f0 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42040a5350 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f421a7dbb80 10 client.25520 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56445b3dbda0 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.147+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f800ecc0 con 0x56445b3c4870 2024-06-06T07:51:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f420409ca60 con 0x56445b3c4870 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 10 client.25520 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 10 client.25520 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 20 client.25520 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b3de3a0 0x56445b3e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 4 client.25520 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445b31eea0 con 0x56445b3de3a0 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f420d4f3700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b3de3a0 0x56445b3e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f41f17fa700 10 client.25520 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.148+0000 7f420d4f3700 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b3de3a0 0x56445b3e0820 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3516ed4e14103c4d server_cookie=a5f5c0f9bacc2c0b in_seq=0 out_seq=0 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f41f17fa700 1 -- 172.21.15.87:0/3873054196 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56445b31eea0 con 0x56445b3de3a0 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f41f17fa700 10 client.25520 handle_command_reply: tid=0 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 client.25520 shutdown 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 msgr2=0x56445b3d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d0790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42040cd910 tx=0x7f42040a3120 comp rx=0 tx=0).stop 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 shutdown_connections 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b3de3a0 0x56445b3e0820 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d0790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3873054196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 >> 172.21.15.87:0/3873054196 conn(0x56445b2eae50 msgr2=0x56445b3bada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f41f17fa700 0 client.25520 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 shutdown_connections 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.149+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3873054196 wait complete. 2024-06-06T07:51:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.150+0000 7f421a7dbb80 20 client.25520 trim_cache size 0 max 16384 2024-06-06T07:51:02.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 Processor -- start 2024-06-06T07:51:02.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 -- start start 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3ca0e0 0x56445b3e6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42081aebe0 con 0x56445b3c4870 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42081ac470 con 0x56445b3ca0e0 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.171+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42081aa2c0 con 0x56445b3caa00 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60140/0 (socket says 172.21.15.87:60140) 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 -- 172.21.15.87:0/769160180 learned_addr learned my addr 172.21.15.87:0/769160180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420d4f3700 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3ca0e0 0x56445b3e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420e4f5700 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 -- 172.21.15.87:0/769160180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 msgr2=0x56445b3cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 -- 172.21.15.87:0/769160180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3ca0e0 msgr2=0x56445b3e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3ca0e0 0x56445b3e6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 -- 172.21.15.87:0/769160180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445b3c8540 con 0x56445b3c4870 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f420dcf4700 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4204005cb0 tx=0x7f4204087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ba96efab2f3d14 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f41f37fe700 1 -- 172.21.15.87:0/769160180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400f6f0 con 0x56445b3c4870 2024-06-06T07:51:02.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f41f37fe700 1 -- 172.21.15.87:0/769160180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420400f890 con 0x56445b3c4870 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.172+0000 7f41f37fe700 1 -- 172.21.15.87:0/769160180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420409c9d0 con 0x56445b3c4870 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- 172.21.15.87:0/769160180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 msgr2=0x56445b3e6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4204005cb0 tx=0x7f4204087050 comp rx=0 tx=0).stop 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- 172.21.15.87:0/769160180 shutdown_connections 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3c4870 0x56445b3e6130 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/769160180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3ca0e0 0x56445b3e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- 172.21.15.87:0/769160180 >> 172.21.15.87:0/769160180 conn(0x56445b2eae50 msgr2=0x56445b3ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- 172.21.15.87:0/769160180 shutdown_connections 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- 172.21.15.87:0/769160180 wait complete. 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.173+0000 7f421a7dbb80 1 -- start start 2024-06-06T07:51:02.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 Processor -- start 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42081aa930 con 0x56445b3ca0e0 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42081abda0 con 0x56445b3c4870 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42081ab000 con 0x56445b3caa00 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f421a7dbb80 10 client.0 _ll_register_callbacks cb 0x56445b2abab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420d4f3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3d6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60156/0 (socket says 172.21.15.87:60156) 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420d4f3700 1 -- 172.21.15.87:0/3925689995 learned_addr learned my addr 172.21.15.87:0/3925689995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420e4f5700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 -- 172.21.15.87:0/3925689995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 msgr2=0x56445b3d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 -- 172.21.15.87:0/3925689995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 msgr2=0x56445b3d6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:02.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3d6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.174+0000 7f420dcf4700 1 -- 172.21.15.87:0/3925689995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445b3d3f30 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f420dcf4700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d6a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56445b3cf090 tx=0x7f420409c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b479d7ce8523a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400f6f0 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420400f890 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f420409c9d0 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42040d03a0 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f421a7dbb80 10 client.25526 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56445b449a70 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420400f890 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.175+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f420409b630 con 0x56445b3c4870 2024-06-06T07:51:02.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 10 client.25526 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 10 client.25526 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 20 client.25526 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b449e00 0x56445b44c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 4 client.25526 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x56445b319420 con 0x56445b449e00 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f420d4f3700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b449e00 0x56445b44c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f41f17fa700 10 client.25526 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:02.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.176+0000 7f420d4f3700 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b449e00 0x56445b44c1f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9ee1385f306429 server_cookie=cd97df0fa50ecc5b in_seq=0 out_seq=0 2024-06-06T07:51:02.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.292+0000 7f41f17fa700 1 -- 172.21.15.87:0/3925689995 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56445b319420 con 0x56445b449e00 2024-06-06T07:51:02.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.292+0000 7f41f17fa700 10 client.25526 handle_command_reply: tid=0 2024-06-06T07:51:02.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.292+0000 7f421a7dbb80 1 client.25526 shutdown 2024-06-06T07:51:02.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.292+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 msgr2=0x56445b3d6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.292+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d6a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56445b3cf090 tx=0x7f420409c360 comp rx=0 tx=0).stop 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 shutdown_connections 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56445b449e00 0x56445b44c1f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56445b3c4870 0x56445b3d6a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56445b3caa00 0x56445b3d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 --2- 172.21.15.87:0/3925689995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56445b3ca0e0 0x56445b3d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 >> 172.21.15.87:0/3925689995 conn(0x56445b2eae50 msgr2=0x56445b3cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f41f17fa700 0 client.25526 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 shutdown_connections 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 1 -- 172.21.15.87:0/3925689995 wait complete. 2024-06-06T07:51:02.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.293+0000 7f421a7dbb80 20 client.25526 trim_cache size 0 max 16384 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 30, 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 6, 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "c2c9cdcd": { 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 6, 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:51:02.296 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:02.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:02.298 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:02.298 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:51:02.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 msgr2=0x7f41f40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 0x7f41f40a52f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4208078990 tx=0x7f41f8009260 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 msgr2=0x7f420819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f420400b2e0 tx=0x7f420400bb20 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 shutdown_connections 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42081013b0 0x7f420819e7c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41f40a2e30 0x7f41f40a52f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4208107460 0x7f4208198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 --2- 172.21.15.87:0/1180442230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4208101d70 0x7f420819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 >> 172.21.15.87:0/1180442230 conn(0x7f42080783b0 msgr2=0x7f4208100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 shutdown_connections 2024-06-06T07:51:02.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:02.294+0000 7f420ff58700 1 -- 172.21.15.87:0/1180442230 wait complete. 2024-06-06T07:51:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:02 smithi087 ceph-mon[32276]: pgmap v93: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:02.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.37 MB/sec warmup 66 sec latency 356.984 ms 2024-06-06T07:51:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:02 smithi195 ceph-mon[37040]: pgmap v93: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:02 smithi181 ceph-mon[36790]: pgmap v93: 129 pgs: 129 active+clean; 33 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-06T07:51:03.259 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c2c9cdcd', quiesce took 6.5 seconds. Will release after: 18.7 2024-06-06T07:51:03.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.31 MB/sec warmup 67 sec latency 1357.089 ms 2024-06-06T07:51:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:04 smithi087 ceph-mon[32276]: pgmap v94: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:51:04.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.25 MB/sec warmup 68 sec latency 2357.149 ms 2024-06-06T07:51:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:04 smithi195 ceph-mon[37040]: pgmap v94: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:51:04.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:04 smithi181 ceph-mon[36790]: pgmap v94: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:51:05.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.18 MB/sec warmup 69 sec latency 3357.224 ms 2024-06-06T07:51:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:06 smithi087 ceph-mon[32276]: pgmap v95: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:51:06.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.12 MB/sec warmup 70 sec latency 4357.315 ms 2024-06-06T07:51:06.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:06 smithi195 ceph-mon[37040]: pgmap v95: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:51:06.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:06 smithi181 ceph-mon[36790]: pgmap v95: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:51:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 -- 172.21.15.87:0/589474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f409010bf70 msgr2=0x7f409010e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 --2- 172.21.15.87:0/589474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f409010bf70 0x7f409010e410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4080009ca0 tx=0x7f408002a9e0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 -- 172.21.15.87:0/589474801 shutdown_connections 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 --2- 172.21.15.87:0/589474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f409010bf70 0x7f409010e410 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 --2- 172.21.15.87:0/589474801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4090103550 0x7f409010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 --2- 172.21.15.87:0/589474801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090102c30 0x7f4090103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 -- 172.21.15.87:0/589474801 >> 172.21.15.87:0/589474801 conn(0x7f4090077a20 msgr2=0x7f4090077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 -- 172.21.15.87:0/589474801 shutdown_connections 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.167+0000 7f40973a2640 1 -- 172.21.15.87:0/589474801 wait complete. 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 Processor -- start 2024-06-06T07:51:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 -- start start 2024-06-06T07:51:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090103550 0x7f4090100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409010bf70 0x7f4090101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4090111d70 con 0x7f4090102c30 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4090111bf0 con 0x7f4090103550 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f40973a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4090111ef0 con 0x7f409010bf70 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f409513e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f409513e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47130/0 (socket says 172.21.15.87:47130) 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f409513e640 1 -- 172.21.15.87:0/386230889 learned_addr learned my addr 172.21.15.87:0/386230889 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f409493d640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090103550 0x7f4090100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.168+0000 7f409593f640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409010bf70 0x7f4090101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 -- 172.21.15.87:0/386230889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409010bf70 msgr2=0x7f4090101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409010bf70 0x7f4090101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 -- 172.21.15.87:0/386230889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090103550 msgr2=0x7f4090100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090103550 0x7f4090100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 -- 172.21.15.87:0/386230889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40901a65d0 con 0x7f4090102c30 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f409513e640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f408800c920 tx=0x7f408800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f408e7fc640 1 -- 172.21.15.87:0/386230889 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088007bb0 con 0x7f4090102c30 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f408e7fc640 1 -- 172.21.15.87:0/386230889 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4088007d50 con 0x7f4090102c30 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a68c0 con 0x7f4090102c30 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f408e7fc640 1 -- 172.21.15.87:0/386230889 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880187d0 con 0x7f4090102c30 2024-06-06T07:51:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.169+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a6e60 con 0x7f4090102c30 2024-06-06T07:51:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.171+0000 7f408e7fc640 1 -- 172.21.15.87:0/386230889 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f408800f450 con 0x7f4090102c30 2024-06-06T07:51:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 Processor -- start 2024-06-06T07:51:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 -- start start 2024-06-06T07:51:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f6873264f0 con 0x55f6873cd350 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6872c4870 con 0x55f6873c2a90 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f409a192740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6873224b0 con 0x55f6873c2170 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.172+0000 7f408e7fc640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 0x7f40780a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47140/0 (socket says 172.21.15.87:47140) 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 -- 172.21.15.87:0/758515022 learned_addr learned my addr 172.21.15.87:0/758515022 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409593f640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 0x7f40780a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f408e7fc640 1 -- 172.21.15.87:0/386230889 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f40880cf8c0 con 0x7f4090102c30 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409593f640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409513e640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 -- 172.21.15.87:0/758515022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 msgr2=0x55f6873c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 -- 172.21.15.87:0/758515022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 msgr2=0x55f6873cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 -- 172.21.15.87:0/758515022 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6872e09b0 con 0x55f6873cd350 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409593f640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409593f640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 0x7f40780a5310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4080036550 tx=0x7f408003a040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f409493d640 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f408400ac20 tx=0x7f408400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ed2011bbfe8ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f40777fe640 1 -- 172.21.15.87:0/758515022 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408401d070 con 0x55f6873cd350 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f40777fe640 1 -- 172.21.15.87:0/758515022 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084018b20 con 0x55f6873cd350 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.173+0000 7f40777fe640 1 -- 172.21.15.87:0/758515022 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084010ac0 con 0x55f6873cd350 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- 172.21.15.87:0/758515022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 msgr2=0x55f6873cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f408400ac20 tx=0x7f408400f790 comp rx=0 tx=0).stop 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- 172.21.15.87:0/758515022 shutdown_connections 2024-06-06T07:51:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cf740 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 --2- 172.21.15.87:0/758515022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- 172.21.15.87:0/758515022 >> 172.21.15.87:0/758515022 conn(0x55f6873be680 msgr2=0x55f6873bea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- 172.21.15.87:0/758515022 shutdown_connections 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- 172.21.15.87:0/758515022 wait complete. 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.174+0000 7f409a192740 1 -- start start 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 Processor -- start 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f6873cff80 con 0x55f6873cd350 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6872afae0 con 0x55f6873c2a90 2024-06-06T07:51:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6873d0100 con 0x55f6873c2170 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409a192740 10 client.0 _ll_register_callbacks cb 0x55f6872e89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47142/0 (socket says 172.21.15.87:47142) 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 -- 172.21.15.87:0/1739582251 learned_addr learned my addr 172.21.15.87:0/1739582251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409513e640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409493d640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 -- 172.21.15.87:0/1739582251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 msgr2=0x55f6873ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 -- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 msgr2=0x55f6873ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.175+0000 7f409593f640 1 -- 172.21.15.87:0/1739582251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68703a210 con 0x55f6873cd350 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409593f640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4080035000 tx=0x7f408002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4cfb82bcf7de4c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408004d070 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4084018b20 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4080004040 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4080033070 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 10 client.16236 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f687106620 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084018b20 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f40800365d0 con 0x55f6873cd350 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 10 client.16236 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 20 client.16236 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.176+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f6873dd4d0 0x55f6873df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.177+0000 7f409a192740 4 client.16236 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.177+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f687339f10 con 0x55f6873dd4d0 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.177+0000 7f409513e640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f6873dd4d0 0x55f6873df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.177+0000 7f409513e640 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f6873dd4d0 0x55f6873df920 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01e11230b794315 server_cookie=529e4a6fe1da2fb8 in_seq=0 out_seq=0 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.177+0000 7f4074ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f4074ff9640 1 -- 172.21.15.87:0/1739582251 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f687339f10 con 0x55f6873dd4d0 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f4074ff9640 10 client.16236 handle_command_reply: tid=0 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 client.16236 shutdown 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 msgr2=0x55f6873cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4080035000 tx=0x7f408002a9e0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 shutdown_connections 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f6873dd4d0 0x55f6873df920 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873cad20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 --2- 172.21.15.87:0/1739582251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f4074ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 >> 172.21.15.87:0/1739582251 conn(0x55f6873be680 msgr2=0x55f6873c43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 shutdown_connections 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 1 -- 172.21.15.87:0/1739582251 wait complete. 2024-06-06T07:51:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.178+0000 7f409a192740 20 client.16236 trim_cache size 0 max 16384 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.193+0000 7f409a192740 1 Processor -- start 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 -- start start 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873e5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b38e0 con 0x55f6873cd350 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b11b0 con 0x55f6873c2a90 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409a192740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40901af000 con 0x55f6873c2170 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47158/0 (socket says 172.21.15.87:47158) 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 -- 172.21.15.87:0/4166027276 learned_addr learned my addr 172.21.15.87:0/4166027276 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409513e640 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 -- 172.21.15.87:0/4166027276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 msgr2=0x55f6873c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 -- 172.21.15.87:0/4166027276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 msgr2=0x55f6873e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 -- 172.21.15.87:0/4166027276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6873b8b40 con 0x55f6873cd350 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f409593f640 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f40800369a0 tx=0x7f4080009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5676ba62bfb3e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f4076ffd640 1 -- 172.21.15.87:0/4166027276 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408005d030 con 0x55f6873cd350 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.194+0000 7f4076ffd640 1 -- 172.21.15.87:0/4166027276 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4080036c10 con 0x55f6873cd350 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f4076ffd640 1 -- 172.21.15.87:0/4166027276 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408004d040 con 0x55f6873cd350 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- 172.21.15.87:0/4166027276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 msgr2=0x55f6873e5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f40800369a0 tx=0x7f4080009990 comp rx=0 tx=0).stop 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- 172.21.15.87:0/4166027276 shutdown_connections 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873cd350 0x55f6873e5770 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 --2- 172.21.15.87:0/4166027276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873c2170 0x55f6873c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- 172.21.15.87:0/4166027276 >> 172.21.15.87:0/4166027276 conn(0x55f6873be680 msgr2=0x55f6873bea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- 172.21.15.87:0/4166027276 shutdown_connections 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- 172.21.15.87:0/4166027276 wait complete. 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 -- start start 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.195+0000 7f409a192740 1 Processor -- start 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873cd350 0x55f6873d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40901af670 con 0x55f6873c2170 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b0ae0 con 0x55f6873c2a90 2024-06-06T07:51:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40901afd40 con 0x55f6873cd350 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409a192740 10 client.0 _ll_register_callbacks cb 0x7f4080009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47166/0 (socket says 172.21.15.87:47166) 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 -- 172.21.15.87:0/1763506678 learned_addr learned my addr 172.21.15.87:0/1763506678 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409493d640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 -- 172.21.15.87:0/1763506678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873cd350 msgr2=0x55f6873d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873cd350 0x55f6873d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 -- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 msgr2=0x55f6873d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 -- 172.21.15.87:0/1763506678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6873d14a0 con 0x55f6873c2170 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409493d640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f409513e640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f408809b710 tx=0x7f4088066a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269b3021127d7770 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.196+0000 7f4074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088066aa0 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f408800fa30 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4080036c10 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f408809ba10 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 10 client.16242 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f687447f00 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4080036c10 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4088098080 con 0x55f6873c2170 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 10 client.16242 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 20 client.16242 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f687448630 0x55f68744aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 4 client.16242 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:51:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.197+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f687333c20 con 0x55f687448630 2024-06-06T07:51:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.198+0000 7f409493d640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f687448630 0x55f68744aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.198+0000 7f409493d640 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f687448630 0x55f68744aa70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0972d5d0810f0c server_cookie=181cf8564b7466e4 in_seq=0 out_seq=0 2024-06-06T07:51:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.198+0000 7f4074ff9640 10 client.16242 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f4074ff9640 1 -- 172.21.15.87:0/1763506678 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f687333c20 con 0x55f687448630 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f4074ff9640 10 client.16242 handle_command_reply: tid=0 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 client.16242 shutdown 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 msgr2=0x55f6873d5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f408809b710 tx=0x7f4088066a30 comp rx=0 tx=0).stop 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 shutdown_connections 2024-06-06T07:51:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f687448630 0x55f68744aa70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6873cd350 0x55f6873d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6873c2a90 0x55f6873d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f4074ff9640 0 client.16242 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 --2- 172.21.15.87:0/1763506678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f6873c2170 0x55f6873d5730 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.309+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 >> 172.21.15.87:0/1763506678 conn(0x55f6873be680 msgr2=0x55f6873ca260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 shutdown_connections 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f409a192740 1 -- 172.21.15.87:0/1763506678 wait complete. 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f409a192740 20 client.16242 trim_cache size 0 max 16384 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:51:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 msgr2=0x7f40780a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 0x7f40780a5310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4080036550 tx=0x7f408003a040 comp rx=0 tx=0).stop 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 msgr2=0x7f4090100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f408800c920 tx=0x7f408800cd20 comp rx=0 tx=0).stop 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 shutdown_connections 2024-06-06T07:51:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f40780a2e50 0x7f40780a5310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409010bf70 0x7f4090101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4090103550 0x7f4090100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 --2- 172.21.15.87:0/386230889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4090102c30 0x7f4090100880 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 >> 172.21.15.87:0/386230889 conn(0x7f4090077a20 msgr2=0x7f40900fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 shutdown_connections 2024-06-06T07:51:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:07.310+0000 7f40973a2640 1 -- 172.21.15.87:0/386230889 wait complete. 2024-06-06T07:51:07.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.07 MB/sec warmup 71 sec latency 5357.414 ms 2024-06-06T07:51:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:08 smithi087 ceph-mon[32276]: pgmap v96: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-06-06T07:51:08.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 4.01 MB/sec warmup 72 sec latency 6357.520 ms 2024-06-06T07:51:08.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:08 smithi195 ceph-mon[37040]: pgmap v96: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-06-06T07:51:08.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:08 smithi181 ceph-mon[36790]: pgmap v96: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-06-06T07:51:09.653 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.95 MB/sec warmup 73 sec latency 7357.577 ms 2024-06-06T07:51:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:10 smithi087 ceph-mon[32276]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:10.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.90 MB/sec warmup 74 sec latency 8357.650 ms 2024-06-06T07:51:10.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:10 smithi195 ceph-mon[37040]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:10.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:10 smithi181 ceph-mon[36790]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:11.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.85 MB/sec warmup 75 sec latency 9357.688 ms 2024-06-06T07:51:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:12 smithi087 ceph-mon[32276]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:12.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.80 MB/sec warmup 76 sec latency 10357.786 ms 2024-06-06T07:51:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:12 smithi195 ceph-mon[37040]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:12 smithi181 ceph-mon[36790]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 KiB/s wr, 133 op/s 2024-06-06T07:51:13.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.75 MB/sec warmup 77 sec latency 11357.883 ms 2024-06-06T07:51:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:14 smithi087 ceph-mon[32276]: pgmap v99: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 210 op/s 2024-06-06T07:51:14.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.70 MB/sec warmup 78 sec latency 12357.940 ms 2024-06-06T07:51:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:14 smithi195 ceph-mon[37040]: pgmap v99: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 210 op/s 2024-06-06T07:51:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:14 smithi181 ceph-mon[36790]: pgmap v99: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 210 op/s 2024-06-06T07:51:15.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.65 MB/sec warmup 79 sec latency 13358.049 ms 2024-06-06T07:51:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:16 smithi087 ceph-mon[32276]: pgmap v100: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 313 KiB/s wr, 98 op/s 2024-06-06T07:51:16.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.61 MB/sec warmup 80 sec latency 14358.146 ms 2024-06-06T07:51:16.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:16 smithi195 ceph-mon[37040]: pgmap v100: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 313 KiB/s wr, 98 op/s 2024-06-06T07:51:16.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:16 smithi181 ceph-mon[36790]: pgmap v100: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 313 KiB/s wr, 98 op/s 2024-06-06T07:51:17.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:17 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:51:17.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.56 MB/sec warmup 81 sec latency 15358.181 ms 2024-06-06T07:51:17.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:17 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:51:17.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:17 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:51:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:18 smithi087 ceph-mon[32276]: pgmap v101: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 323 KiB/s wr, 102 op/s 2024-06-06T07:51:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:51:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:51:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:51:18.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.52 MB/sec warmup 82 sec latency 16358.262 ms 2024-06-06T07:51:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:18 smithi195 ceph-mon[37040]: pgmap v101: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 323 KiB/s wr, 102 op/s 2024-06-06T07:51:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:51:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:51:18.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:51:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:18 smithi181 ceph-mon[36790]: pgmap v101: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 323 KiB/s wr, 102 op/s 2024-06-06T07:51:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:51:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:51:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:51:19.655 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.48 MB/sec warmup 83 sec latency 17358.397 ms 2024-06-06T07:51:19.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:19 smithi195 ceph-mon[37040]: pgmap v102: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:19.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:19 smithi181 ceph-mon[36790]: pgmap v102: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:19.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:19 smithi087 ceph-mon[32276]: pgmap v102: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 -- 172.21.15.87:0/263968173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2254100bf0 msgr2=0x7f2254101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 --2- 172.21.15.87:0/263968173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2254100bf0 0x7f2254101070 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2250009900 tx=0x7f225002a410 comp rx=0 tx=0).stop 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 -- 172.21.15.87:0/263968173 shutdown_connections 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 --2- 172.21.15.87:0/263968173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22541015b0 0x7f225410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 --2- 172.21.15.87:0/263968173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2254100bf0 0x7f2254101070 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 --2- 172.21.15.87:0/263968173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254105840 0x7f2254105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 -- 172.21.15.87:0/263968173 >> 172.21.15.87:0/263968173 conn(0x7f22540766a0 msgr2=0x7f2254076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 -- 172.21.15.87:0/263968173 shutdown_connections 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.410+0000 7f225c093640 1 -- 172.21.15.87:0/263968173 wait complete. 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 Processor -- start 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 -- start start 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254100bf0 0x7f2254196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2254105840 0x7f225419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2254110c70 con 0x7f22541015b0 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2254110af0 con 0x7f2254100bf0 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225c093640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2254110df0 con 0x7f2254105840 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225962e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225962e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41480/0 (socket says 172.21.15.87:41480) 2024-06-06T07:51:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225962e640 1 -- 172.21.15.87:0/1879622300 learned_addr learned my addr 172.21.15.87:0/1879622300 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f2259e2f640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254100bf0 0x7f2254196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.411+0000 7f225a630640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2254105840 0x7f225419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 -- 172.21.15.87:0/1879622300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2254105840 msgr2=0x7f225419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2254105840 0x7f225419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 -- 172.21.15.87:0/1879622300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254100bf0 msgr2=0x7f2254196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254100bf0 0x7f2254196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 -- 172.21.15.87:0/1879622300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225419bfe0 con 0x7f22541015b0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225962e640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2250006010 tx=0x7f22500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f2246ffd640 1 -- 172.21.15.87:0/1879622300 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225003f030 con 0x7f22541015b0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f2246ffd640 1 -- 172.21.15.87:0/1879622300 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250003ce0 con 0x7f22541015b0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225419c270 con 0x7f22541015b0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f2246ffd640 1 -- 172.21.15.87:0/1879622300 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250035070 con 0x7f22541015b0 2024-06-06T07:51:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.412+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22541a3b60 con 0x7f22541015b0 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.414+0000 7f2246ffd640 1 -- 172.21.15.87:0/1879622300 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2250042030 con 0x7f22541015b0 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 Processor -- start 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 -- start start 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557792ea44f0 con 0x557792f400b0 2024-06-06T07:51:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557792e42870 con 0x557792f4d560 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225eda0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557792ea04b0 con 0x557792f409d0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f2246ffd640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 0x7f22400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225a630640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225a630640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41482/0 (socket says 172.21.15.87:41482) 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225a630640 1 -- 172.21.15.87:0/1286276238 learned_addr learned my addr 172.21.15.87:0/1286276238 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225a630640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 0x7f22400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f2259e2f640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f2246ffd640 1 -- 172.21.15.87:0/1879622300 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22500efe40 con 0x7f22541015b0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.415+0000 7f225962e640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 -- 172.21.15.87:0/1286276238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 msgr2=0x557792f4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 -- 172.21.15.87:0/1286276238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 msgr2=0x557792f4f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 -- 172.21.15.87:0/1286276238 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557792e5e9b0 con 0x557792f400b0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f2259e2f640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f224c012d00 tx=0x7f224c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e94315e0c209c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f225a630640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 0x7f22400a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f224c009fd0 tx=0x7f224c017000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f223ffff640 1 -- 172.21.15.87:0/1286276238 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c022070 con 0x557792f400b0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f223ffff640 1 -- 172.21.15.87:0/1286276238 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0124f0 con 0x557792f400b0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.416+0000 7f223ffff640 1 -- 172.21.15.87:0/1286276238 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c01e900 con 0x557792f400b0 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- 172.21.15.87:0/1286276238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 msgr2=0x557792f40490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f224c012d00 tx=0x7f224c010740 comp rx=0 tx=0).stop 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- 172.21.15.87:0/1286276238 shutdown_connections 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 --2- 172.21.15.87:0/1286276238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f40490 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- 172.21.15.87:0/1286276238 >> 172.21.15.87:0/1286276238 conn(0x557792f3c680 msgr2=0x557792f3ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- 172.21.15.87:0/1286276238 shutdown_connections 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- 172.21.15.87:0/1286276238 wait complete. 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 -- start start 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.417+0000 7f225eda0740 1 Processor -- start 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557792f500d0 con 0x557792f400b0 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557792e2dae0 con 0x557792f4d560 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557792f50250 con 0x557792f409d0 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225eda0740 10 client.0 _ll_register_callbacks cb 0x557792e669b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41498/0 (socket says 172.21.15.87:41498) 2024-06-06T07:51:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 -- 172.21.15.87:0/2006064768 learned_addr learned my addr 172.21.15.87:0/2006064768 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225962e640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225a630640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 -- 172.21.15.87:0/2006064768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 msgr2=0x557792f4aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 -- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 msgr2=0x557792f4b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 -- 172.21.15.87:0/2006064768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557792bb8210 con 0x557792f400b0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f225962e640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f224800df60 tx=0x7f224800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48022ac5ec84ffb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f223d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.418+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248005220 con 0x557792f400b0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22480053c0 con 0x557792f400b0 2024-06-06T07:51:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224c0124f0 con 0x557792f400b0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22480055c0 con 0x557792f400b0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557792c84620 con 0x557792f400b0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0124f0 con 0x557792f400b0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2248012a50 con 0x557792f400b0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 20 client.16254 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792f5d7e0 0x557792f5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 4 client.16254 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557792eb7f10 con 0x557792f5d7e0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.419+0000 7f225962e640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792f5d7e0 0x557792f5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.420+0000 7f225962e640 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792f5d7e0 0x557792f5fc30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f9985dbf1fe461 server_cookie=5fd249f84ea4f438 in_seq=0 out_seq=0 2024-06-06T07:51:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.420+0000 7f223d7fa640 10 client.16254 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.420+0000 7f223d7fa640 1 -- 172.21.15.87:0/2006064768 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557792eb7f10 con 0x557792f5d7e0 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.420+0000 7f223d7fa640 10 client.16254 handle_command_reply: tid=0 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 client.16254 shutdown 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 msgr2=0x557792f4a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f224800df60 tx=0x7f224800df90 comp rx=0 tx=0).stop 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 shutdown_connections 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792f5d7e0 0x557792f5fc30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f4b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f409d0 0x557792f4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f223d7fa640 0 client.16254 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 --2- 172.21.15.87:0/2006064768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f400b0 0x557792f4a590 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 >> 172.21.15.87:0/2006064768 conn(0x557792f3c680 msgr2=0x557792f42400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 shutdown_connections 2024-06-06T07:51:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 1 -- 172.21.15.87:0/2006064768 wait complete. 2024-06-06T07:51:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.421+0000 7f225eda0740 20 client.16254 trim_cache size 0 max 16384 2024-06-06T07:51:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 Processor -- start 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 -- start start 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f65540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f409d0 0x557792f65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22541af4a0 con 0x557792f4d560 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22541acd70 con 0x557792f409d0 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225eda0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22541aabc0 con 0x557792f400b0 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41500/0 (socket says 172.21.15.87:41500) 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 -- 172.21.15.87:0/3557960707 learned_addr learned my addr 172.21.15.87:0/3557960707 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f2259e2f640 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f65540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 -- 172.21.15.87:0/3557960707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 msgr2=0x557792f65540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f65540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 -- 172.21.15.87:0/3557960707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f409d0 msgr2=0x557792f65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.436+0000 7f225a630640 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f409d0 0x557792f65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225a630640 1 -- 172.21.15.87:0/3557960707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557792f3d7d0 con 0x557792f4d560 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f2259e2f640 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225a630640 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f224c009520 tx=0x7f224c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69195c6d5ba27434 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f223f7fe640 1 -- 172.21.15.87:0/3557960707 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c022040 con 0x557792f4d560 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f223f7fe640 1 -- 172.21.15.87:0/3557960707 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0127e0 con 0x557792f4d560 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f223f7fe640 1 -- 172.21.15.87:0/3557960707 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c00cda0 con 0x557792f4d560 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 -- 172.21.15.87:0/3557960707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 msgr2=0x557792f3af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f224c009520 tx=0x7f224c005a10 comp rx=0 tx=0).stop 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 -- 172.21.15.87:0/3557960707 shutdown_connections 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f4d560 0x557792f3af50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f409d0 0x557792f65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 --2- 172.21.15.87:0/3557960707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 -- 172.21.15.87:0/3557960707 >> 172.21.15.87:0/3557960707 conn(0x557792f3c680 msgr2=0x557792f3ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 -- 172.21.15.87:0/3557960707 shutdown_connections 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.437+0000 7f225eda0740 1 -- 172.21.15.87:0/3557960707 wait complete. 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 -- start start 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 Processor -- start 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22541ab230 con 0x557792f409d0 2024-06-06T07:51:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22541ac6a0 con 0x557792f4d560 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22541ab900 con 0x557792f400b0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225eda0740 10 client.0 _ll_register_callbacks cb 0x7f224800d460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225962e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225962e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41512/0 (socket says 172.21.15.87:41512) 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.438+0000 7f225962e640 1 -- 172.21.15.87:0/2749462103 learned_addr learned my addr 172.21.15.87:0/2749462103 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225a630640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 -- 172.21.15.87:0/2749462103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 msgr2=0x557792f55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 -- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 msgr2=0x557792f513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 -- 172.21.15.87:0/2749462103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557792f51ae0 con 0x557792f409d0 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225a630640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225962e640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f22500bbcf0 tx=0x7f225002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06907a19d468aea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250049040 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22500ba2f0 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224c0127e0 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22500bbff0 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 10 client.16260 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557792fc83f0 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0127e0 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f22500bc250 con 0x557792f409d0 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 20 client.16260 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.439+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792fc8b80 0x557792fcafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.440+0000 7f225eda0740 4 client.16260 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:51:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.440+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557792eb1c20 con 0x557792fc8b80 2024-06-06T07:51:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.440+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792fc8b80 0x557792fcafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.440+0000 7f2259e2f640 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792fc8b80 0x557792fcafa0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f597f20dc0146a server_cookie=9d6dbad8a8577ac1 in_seq=0 out_seq=0 2024-06-06T07:51:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.440+0000 7f223d7fa640 10 client.16260 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f223d7fa640 1 -- 172.21.15.87:0/2749462103 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557792eb1c20 con 0x557792fc8b80 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f223d7fa640 10 client.16260 handle_command_reply: tid=0 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 client.16260 shutdown 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 msgr2=0x557792f50eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f22500bbcf0 tx=0x7f225002a410 comp rx=0 tx=0).stop 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 shutdown_connections 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557792fc8b80 0x557792fcafa0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557792f4d560 0x557792f513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557792f409d0 0x557792f50eb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f223d7fa640 0 client.16260 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 --2- 172.21.15.87:0/2749462103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557792f400b0 0x557792f55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 >> 172.21.15.87:0/2749462103 conn(0x557792f3c680 msgr2=0x557792f4bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 shutdown_connections 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 1 -- 172.21.15.87:0/2749462103 wait complete. 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.464+0000 7f225eda0740 20 client.16260 trim_cache size 0 max 16384 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:51:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 msgr2=0x7f22400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 0x7f22400a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f224c009fd0 tx=0x7f224c017000 comp rx=0 tx=0).stop 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 msgr2=0x7f2254197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2250006010 tx=0x7f22500042f0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 shutdown_connections 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22400a2da0 0x7f22400a5260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2254105840 0x7f225419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22541015b0 0x7f2254197530 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 --2- 172.21.15.87:0/1879622300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2254100bf0 0x7f2254196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 >> 172.21.15.87:0/1879622300 conn(0x7f22540766a0 msgr2=0x7f22540fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 shutdown_connections 2024-06-06T07:51:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:20.465+0000 7f225c093640 1 -- 172.21.15.87:0/1879622300 wait complete. 2024-06-06T07:51:20.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.44 MB/sec warmup 84 sec latency 18358.523 ms 2024-06-06T07:51:21.654 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.40 MB/sec warmup 85 sec latency 19358.618 ms 2024-06-06T07:51:21.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:21 smithi195 ceph-mon[37040]: pgmap v103: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:21.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:21 smithi181 ceph-mon[36790]: pgmap v103: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:21.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:21 smithi087 ceph-mon[32276]: pgmap v103: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 204 KiB/s wr, 80 op/s 2024-06-06T07:51:21.961 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:51:22.213 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:51:22.655 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.36 MB/sec warmup 86 sec latency 20358.658 ms 2024-06-06T07:51:23.655 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.32 MB/sec warmup 87 sec latency 21358.728 ms 2024-06-06T07:51:23.680 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:23 smithi087 ceph-mon[32276]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 216 KiB/s wr, 91 op/s 2024-06-06T07:51:23.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:23 smithi181 ceph-mon[36790]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 216 KiB/s wr, 91 op/s 2024-06-06T07:51:24.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:23 smithi195 ceph-mon[37040]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 216 KiB/s wr, 91 op/s 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.110+0000 7fecebe88700 1 -- 172.21.15.87:0/4151734264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4108520 msgr2=0x7fece4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.110+0000 7fecebe88700 1 --2- 172.21.15.87:0/4151734264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4108520 0x7fece4108900 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fece0009990 tx=0x7fece0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.110+0000 7fecebe88700 1 -- 172.21.15.87:0/4151734264 shutdown_connections 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.110+0000 7fecebe88700 1 --2- 172.21.15.87:0/4151734264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.111+0000 7fecebe88700 1 --2- 172.21.15.87:0/4151734264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4108520 0x7fece4108900 secure :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fece0009990 tx=0x7fece0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.111+0000 7fecebe88700 1 --2- 172.21.15.87:0/4151734264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece4102510 0x7fece4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.111+0000 7fecebe88700 1 -- 172.21.15.87:0/4151734264 >> 172.21.15.87:0/4151734264 conn(0x7fece4079560 msgr2=0x7fece4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:24.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.115+0000 7fecebe88700 1 -- 172.21.15.87:0/4151734264 shutdown_connections 2024-06-06T07:51:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.115+0000 7fecebe88700 1 -- 172.21.15.87:0/4151734264 wait complete. 2024-06-06T07:51:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 Processor -- start 2024-06-06T07:51:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 -- start start 2024-06-06T07:51:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece406d350 0x7fece406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fece4110640 con 0x7fece4102510 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fece41104c0 con 0x7fece406d350 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fecebe88700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fece41107c0 con 0x7fece4102ed0 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fece9c24700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fece9c24700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41532/0 (socket says 172.21.15.87:41532) 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fece9c24700 1 -- 172.21.15.87:0/4189876263 learned_addr learned my addr 172.21.15.87:0/4189876263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.116+0000 7fece9423700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fecea425700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece406d350 0x7fece406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9423700 1 -- 172.21.15.87:0/4189876263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece406d350 msgr2=0x7fece406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9423700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece406d350 0x7fece406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9423700 1 -- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 msgr2=0x7fece4073d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9423700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9423700 1 -- 172.21.15.87:0/4189876263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece40718e0 con 0x7fece4102ed0 2024-06-06T07:51:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.118+0000 7fece9c24700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.119+0000 7fece9423700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece406ce10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecd400d750 tx=0x7fecd400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.120+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4014030 con 0x7fece4102ed0 2024-06-06T07:51:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.120+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004500 con 0x7fece4102ed0 2024-06-06T07:51:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.120+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd40054e0 con 0x7fece4102ed0 2024-06-06T07:51:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.120+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece4071bd0 con 0x7fece4102ed0 2024-06-06T07:51:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.120+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece4072000 con 0x7fece4102ed0 2024-06-06T07:51:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.123+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc8005330 con 0x7fece4102ed0 2024-06-06T07:51:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.125+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fecd4004020 con 0x7fece4102ed0 2024-06-06T07:51:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.126+0000 7fecdaffd700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 0x7fecd00a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.127+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fecd40ceea0 con 0x7fece4102ed0 2024-06-06T07:51:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.129+0000 7fece9c24700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 0x7fecd00a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.129+0000 7fece9c24700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 0x7fecd00a5450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fece0009750 tx=0x7fece0005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.135+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fecd409b690 con 0x7fece4102ed0 2024-06-06T07:51:24.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.406+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc80058f0 con 0x7fece4102ed0 2024-06-06T07:51:24.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.406+0000 7fecdaffd700 1 -- 172.21.15.87:0/4189876263 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11877 (secure 0 0 0) 0x7fecd409af90 con 0x7fece4102ed0 2024-06-06T07:51:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:51:24.407 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:50:06.260840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:51:24.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 msgr2=0x7fecd00a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 0x7fecd00a5450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fece0009750 tx=0x7fece0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 msgr2=0x7fece406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece406ce10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecd400d750 tx=0x7fecd400da60 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 shutdown_connections 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece406d350 0x7fece406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fecd00a2f90 0x7fecd00a5450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fece4102510 0x7fece4073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 --2- 172.21.15.87:0/4189876263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fece4102ed0 0x7fece406ce10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 >> 172.21.15.87:0/4189876263 conn(0x7fece4079560 msgr2=0x7fece4101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 shutdown_connections 2024-06-06T07:51:24.409 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:24.409+0000 7fecebe88700 1 -- 172.21.15.87:0/4189876263 wait complete. 2024-06-06T07:51:24.410 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 30 2024-06-06T07:51:24.655 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.28 MB/sec warmup 88 sec latency 22358.780 ms 2024-06-06T07:51:24.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:24 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4189876263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:51:24.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:24 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4189876263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:51:25.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:24 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4189876263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:51:25.113 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id c2c9cdcd --release --await' 2024-06-06T07:51:25.113 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id c2c9cdcd --release --await 2024-06-06T07:51:25.355 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:51:25.655 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.24 MB/sec warmup 89 sec latency 23358.834 ms 2024-06-06T07:51:25.754 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:25 smithi087 ceph-mon[32276]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-06T07:51:25.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:25 smithi181 ceph-mon[36790]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-06T07:51:26.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:25 smithi195 ceph-mon[37040]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-06T07:51:26.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50046 3.21 MB/sec warmup 90 sec latency 24359.810 ms 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 -- 172.21.15.87:0/3079671143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e0102740 msgr2=0x7f61e0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 --2- 172.21.15.87:0/3079671143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e0102740 0x7f61e0102b20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f61d40099c0 tx=0x7f61d4009cd0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 -- 172.21.15.87:0/3079671143 shutdown_connections 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 --2- 172.21.15.87:0/3079671143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0107840 0x7f61e010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 --2- 172.21.15.87:0/3079671143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e0102740 0x7f61e0102b20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 --2- 172.21.15.87:0/3079671143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e01030f0 0x7f61e0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 -- 172.21.15.87:0/3079671143 >> 172.21.15.87:0/3079671143 conn(0x7f61e00795f0 msgr2=0x7f61e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 -- 172.21.15.87:0/3079671143 shutdown_connections 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.252+0000 7f61e7209700 1 -- 172.21.15.87:0/3079671143 wait complete. 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 Processor -- start 2024-06-06T07:51:27.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 -- start start 2024-06-06T07:51:27.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0102740 0x7f61e019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 0x7f61e0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0112760 con 0x7f61e01030f0 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01125e0 con 0x7f61e0107840 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61e7209700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01128e0 con 0x7f61e0102740 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49630/0 (socket says 172.21.15.87:49630) 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.253+0000 7f61dffff700 1 -- 172.21.15.87:0/1562572069 learned_addr learned my addr 172.21.15.87:0/1562572069 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61e57a6700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 0x7f61e0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0102740 0x7f61e019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 -- 172.21.15.87:0/1562572069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0102740 msgr2=0x7f61e019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0102740 0x7f61e019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 -- 172.21.15.87:0/1562572069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 msgr2=0x7f61e0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 0x7f61e0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 -- 172.21.15.87:0/1562572069 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e01990c0 con 0x7f61e01030f0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61e57a6700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 0x7f61e0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61dffff700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00c9b0 tx=0x7f61cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61ddffb700 1 -- 172.21.15.87:0/1562572069 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc007b00 con 0x7f61e01030f0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61ddffb700 1 -- 172.21.15.87:0/1562572069 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc007ca0 con 0x7f61e01030f0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01993b0 con 0x7f61e01030f0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61ddffb700 1 -- 172.21.15.87:0/1562572069 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc018730 con 0x7f61e01030f0 2024-06-06T07:51:27.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.254+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a5670 con 0x7f61e01030f0 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.257+0000 7f61ddffb700 1 -- 172.21.15.87:0/1562572069 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f61cc0188d0 con 0x7f61e01030f0 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.257+0000 7f61ddffb700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 0x7f61d00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.257+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 0x7f61d00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.257+0000 7f61ddffb700 1 -- 172.21.15.87:0/1562572069 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f61cc0cf830 con 0x7f61e01030f0 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 0x7f61d00a5250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 Processor -- start 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 -- start start 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f82070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0e74060 con 0x55a1e0f87500 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0ed2950 con 0x55a1e0f87e20 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0e3a560 con 0x55a1e0f81c50 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61e4fa5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61e4fa5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49644/0 (socket says 172.21.15.87:49644) 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.258+0000 7f61e4fa5700 1 -- 172.21.15.87:0/2149899663 learned_addr learned my addr 172.21.15.87:0/2149899663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e57a6700 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f82070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61dffff700 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 -- 172.21.15.87:0/2149899663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 msgr2=0x55a1e0f82070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f82070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 -- 172.21.15.87:0/2149899663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 msgr2=0x55a1e0f81710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f81710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 -- 172.21.15.87:0/2149899663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e0e219d0 con 0x55a1e0f87500 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f61d4015000 tx=0x7f61d4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed811bc9388da8db server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61caffd700 1 -- 172.21.15.87:0/2149899663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d400bda0 con 0x55a1e0f87500 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61caffd700 1 -- 172.21.15.87:0/2149899663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d400fd50 con 0x55a1e0f87500 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.259+0000 7f61caffd700 1 -- 172.21.15.87:0/2149899663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4024d80 con 0x55a1e0f87500 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2149899663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 msgr2=0x55a1e0f878e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f61d4015000 tx=0x7f61d4020000 comp rx=0 tx=0).stop 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2149899663 shutdown_connections 2024-06-06T07:51:27.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f82070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f878e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2149899663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f81710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2149899663 >> 172.21.15.87:0/2149899663 conn(0x55a1e0eae180 msgr2=0x55a1e0f7ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2149899663 shutdown_connections 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2149899663 wait complete. 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.260+0000 7f61f1a8cb80 1 -- start start 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 Processor -- start 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0e730c0 con 0x55a1e0f87500 2024-06-06T07:51:27.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0ece910 con 0x55a1e0f87e20 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0f8e740 con 0x55a1e0f81c50 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61f1a8cb80 10 client.0 _ll_register_callbacks cb 0x55a1e0e686b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49656/0 (socket says 172.21.15.87:49656) 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 -- 172.21.15.87:0/2655255757 learned_addr learned my addr 172.21.15.87:0/2655255757 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61e57a6700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 -- 172.21.15.87:0/2655255757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 msgr2=0x55a1e0f8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 -- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 msgr2=0x55a1e0f8b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f8b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61dffff700 1 -- 172.21.15.87:0/2655255757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e0e98850 con 0x55a1e0f87500 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.261+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61dffff700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0074a0 tx=0x7f61cc09c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d764dad4df3c40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0b3070 con 0x55a1e0f87500 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f61cc09bdf0 con 0x55a1e0f87500 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d400fd50 con 0x55a1e0f87500 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61cc09fd30 con 0x55a1e0f87500 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 10 client.16278 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1e0f98e60 con 0x55a1e0f87500 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d400fd50 con 0x55a1e0f87500 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f61cc0ba140 con 0x55a1e0f87500 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 10 client.16278 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 20 client.16278 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e0f9b3f0 0x55a1e0f9d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 4 client.16278 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e0edfe50 con 0x55a1e0f9b3f0 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.262+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e0f9b3f0 0x55a1e0f9d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.263+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e0f9b3f0 0x55a1e0f9d820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9532742ff20c8a75 server_cookie=a6233e810f2ecbe5 in_seq=0 out_seq=0 2024-06-06T07:51:27.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.263+0000 7f61bffff700 10 client.16278 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:27.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61bffff700 1 -- 172.21.15.87:0/2655255757 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1e0edfe50 con 0x55a1e0f9b3f0 2024-06-06T07:51:27.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61bffff700 10 client.16278 handle_command_reply: tid=0 2024-06-06T07:51:27.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 client.16278 shutdown 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 msgr2=0x55a1e0f8add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0074a0 tx=0x7f61cc09c130 comp rx=0 tx=0).stop 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 shutdown_connections 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e0f9b3f0 0x55a1e0f9d820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87e20 0x55a1e0f8b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f81c50 0x55a1e0f8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/2655255757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0f8add0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 >> 172.21.15.87:0/2655255757 conn(0x55a1e0eae180 msgr2=0x55a1e0f7b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61bffff700 0 client.16278 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 shutdown_connections 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/2655255757 wait complete. 2024-06-06T07:51:27.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.264+0000 7f61f1a8cb80 20 client.16278 trim_cache size 0 max 16384 2024-06-06T07:51:27.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.283+0000 7f61f1a8cb80 1 Processor -- start 2024-06-06T07:51:27.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.283+0000 7f61f1a8cb80 1 -- start start 2024-06-06T07:51:27.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 0x55a1e0f85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0f88910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01b0d30 con 0x55a1e0f81c50 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ae5c0 con 0x55a1e0f87500 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ac410 con 0x55a1e0f87e20 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49660/0 (socket says 172.21.15.87:49660) 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 -- 172.21.15.87:0/1050043480 learned_addr learned my addr 172.21.15.87:0/1050043480 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e57a6700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0f88910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61dffff700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 0x55a1e0f85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 -- 172.21.15.87:0/1050043480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 msgr2=0x55a1e0f88910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0f88910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 -- 172.21.15.87:0/1050043480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 msgr2=0x55a1e0f85050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 0x55a1e0f85050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 -- 172.21.15.87:0/1050043480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e0f8aba0 con 0x55a1e0f81c50 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61dffff700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 0x55a1e0f85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009d10 tx=0x7f61d400b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423eaa05b71f40a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61ca7fc700 1 -- 172.21.15.87:0/1050043480 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d403c070 con 0x55a1e0f81c50 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61ca7fc700 1 -- 172.21.15.87:0/1050043480 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4024df0 con 0x55a1e0f81c50 2024-06-06T07:51:27.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.284+0000 7f61ca7fc700 1 -- 172.21.15.87:0/1050043480 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4038420 con 0x55a1e0f81c50 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/1050043480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 msgr2=0x55a1e0f81710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009d10 tx=0x7f61d400b580 comp rx=0 tx=0).stop 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/1050043480 shutdown_connections 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0f88910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f81c50 0x55a1e0f81710 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/1050043480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f87500 0x55a1e0f85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/1050043480 >> 172.21.15.87:0/1050043480 conn(0x55a1e0eae180 msgr2=0x55a1e0f88200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/1050043480 shutdown_connections 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/1050043480 wait complete. 2024-06-06T07:51:27.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 -- start start 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.285+0000 7f61f1a8cb80 1 Processor -- start 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 0x55a1e0fa2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0fa2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aca80 con 0x55a1e0f87500 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01adef0 con 0x55a1e0f81c50 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ad150 con 0x55a1e0f87e20 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61f1a8cb80 10 client.0 _ll_register_callbacks cb 0x55a1e0e686b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49664/0 (socket says 172.21.15.87:49664) 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 -- 172.21.15.87:0/3249106868 learned_addr learned my addr 172.21.15.87:0/3249106868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 0x55a1e0fa2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61e57a6700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0fa2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 -- 172.21.15.87:0/3249106868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 msgr2=0x55a1e0fa2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0fa2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 -- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 msgr2=0x55a1e0fa2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 0x55a1e0fa2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 -- 172.21.15.87:0/3249106868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e100a400 con 0x55a1e0f87500 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 0x55a1e0fa2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61e57a6700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0fa2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61dffff700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0a40c0 tx=0x7f61cc00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe229bc9abbd3d35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.286+0000 7f61bffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0b3070 con 0x55a1e0f87500 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f61cc0cf560 con 0x55a1e0f87500 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4024df0 con 0x55a1e0f87500 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61cc0ba170 con 0x55a1e0f87500 2024-06-06T07:51:27.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 10 client.16284 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1e100b6b0 con 0x55a1e0f87500 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4024df0 con 0x55a1e0f87500 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f61cc0b4c10 con 0x55a1e0f87500 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 10 client.16284 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 20 client.16284 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e100bb70 0x55a1e100df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 4 client.16284 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2c9cdcd", "release": true, "await": true}] 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.287+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2c9cdcd", "release": true, "await": true}) -- 0x55a1e0eda3d0 con 0x55a1e100bb70 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.288+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e100bb70 0x55a1e100df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.288+0000 7f61e4fa5700 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e100bb70 0x55a1e100df60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec5b8e672c8cab7 server_cookie=107d305d53f02618 in_seq=0 out_seq=0 2024-06-06T07:51:27.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.288+0000 7f61bffff700 10 client.16284 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61bffff700 1 -- 172.21.15.87:0/3249106868 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a1e0eda3d0 con 0x55a1e100bb70 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61bffff700 10 client.16284 handle_command_reply: tid=0 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 client.16284 shutdown 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 msgr2=0x55a1e0fa2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0a40c0 tx=0x7f61cc00a5f0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 shutdown_connections 2024-06-06T07:51:27.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a1e100bb70 0x55a1e100df60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1e0f81c50 0x55a1e0fa2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a1e0f87e20 0x55a1e0fa2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 --2- 172.21.15.87:0/3249106868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a1e0f87500 0x55a1e0fa2680 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 >> 172.21.15.87:0/3249106868 conn(0x55a1e0eae180 msgr2=0x55a1e0f7b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61bffff700 0 client.16284 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 shutdown_connections 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 1 -- 172.21.15.87:0/3249106868 wait complete. 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.298+0000 7f61f1a8cb80 20 client.16284 trim_cache size 0 max 16384 2024-06-06T07:51:27.299 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 30, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 8, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "c2c9cdcd": { 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 8, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:51:27.300 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:51:27.301 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:51:27.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 msgr2=0x7f61d00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 0x7f61d00a5250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4005fb0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 msgr2=0x7f61e019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:27.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00c9b0 tx=0x7f61cc00ccc0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 shutdown_connections 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61e0107840 0x7f61e0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61d00a2d90 0x7f61d00a5250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0102740 0x7f61e019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 --2- 172.21.15.87:0/1562572069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61e01030f0 0x7f61e019ece0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 >> 172.21.15.87:0/1562572069 conn(0x7f61e00795f0 msgr2=0x7f61e0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 shutdown_connections 2024-06-06T07:51:27.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:51:27.299+0000 7f61e7209700 1 -- 172.21.15.87:0/1562572069 wait complete. 2024-06-06T07:51:27.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 50356 3.17 MB/sec warmup 91 sec latency 25004.787 ms 2024-06-06T07:51:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:27 smithi181 ceph-mon[36790]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.4 MiB/s wr, 43 op/s 2024-06-06T07:51:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:27 smithi087 ceph-mon[32276]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.4 MiB/s wr, 43 op/s 2024-06-06T07:51:28.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:27 smithi195 ceph-mon[37040]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.4 MiB/s wr, 43 op/s 2024-06-06T07:51:28.163 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c2c9cdcd', total seconds elapsed: 31.4 2024-06-06T07:51:28.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 53361 3.33 MB/sec warmup 92 sec latency 8.694 ms 2024-06-06T07:51:29.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 55192 3.37 MB/sec warmup 93 sec latency 24.341 ms 2024-06-06T07:51:29.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:29 smithi181 ceph-mon[36790]: pgmap v107: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:29.811 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:29 smithi195 ceph-mon[37040]: pgmap v107: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:29.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:29 smithi087 ceph-mon[32276]: pgmap v107: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:30.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 57503 3.48 MB/sec warmup 94 sec latency 15.567 ms 2024-06-06T07:51:31.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 60568 3.64 MB/sec warmup 95 sec latency 20.775 ms 2024-06-06T07:51:31.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:31 smithi181 ceph-mon[36790]: pgmap v108: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:31.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:31 smithi087 ceph-mon[32276]: pgmap v108: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:32.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:31 smithi195 ceph-mon[37040]: pgmap v108: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-06-06T07:51:32.656 INFO:tasks.workunit.client.0.smithi087.stdout: 1 62891 3.69 MB/sec warmup 96 sec latency 23.578 ms 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 -- 172.21.15.87:0/22479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb80101290 msgr2=0x7ffb8010d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 --2- 172.21.15.87:0/22479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb80101290 0x7ffb8010d750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009cd0 tx=0x7ffb7802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 -- 172.21.15.87:0/22479062 shutdown_connections 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 --2- 172.21.15.87:0/22479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb80101290 0x7ffb8010d750 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 --2- 172.21.15.87:0/22479062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb801008d0 0x7ffb80100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 --2- 172.21.15.87:0/22479062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80106890 0x7ffb80106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 -- 172.21.15.87:0/22479062 >> 172.21.15.87:0/22479062 conn(0x7ffb80077a20 msgr2=0x7ffb80077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 -- 172.21.15.87:0/22479062 shutdown_connections 2024-06-06T07:51:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.559+0000 7ffb8564d640 1 -- 172.21.15.87:0/22479062 wait complete. 2024-06-06T07:51:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 Processor -- start 2024-06-06T07:51:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 -- start start 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80101290 0x7ffb80199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80106890 0x7ffb801a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80110ca0 con 0x7ffb801008d0 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80110b20 con 0x7ffb80106890 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb8564d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80110e20 con 0x7ffb80101290 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49690/0 (socket says 172.21.15.87:49690) 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 -- 172.21.15.87:0/3941454521 learned_addr learned my addr 172.21.15.87:0/3941454521 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80101290 0x7ffb80199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80106890 0x7ffb801a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 -- 172.21.15.87:0/3941454521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80101290 msgr2=0x7ffb80199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80101290 0x7ffb80199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 -- 172.21.15.87:0/3941454521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80106890 msgr2=0x7ffb801a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80106890 0x7ffb801a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.560+0000 7ffb7effd640 1 -- 172.21.15.87:0/3941454521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801a45a0 con 0x7ffb801008d0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000e9f0 tx=0x7ffb7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb67fff640 1 -- 172.21.15.87:0/3941454521 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000cd80 con 0x7ffb801008d0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb67fff640 1 -- 172.21.15.87:0/3941454521 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb70005500 con 0x7ffb801008d0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb67fff640 1 -- 172.21.15.87:0/3941454521 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70010690 con 0x7ffb801008d0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801a4770 con 0x7ffb801008d0 2024-06-06T07:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.561+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb801a4c90 con 0x7ffb801008d0 2024-06-06T07:51:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.563+0000 7ffb67fff640 1 -- 172.21.15.87:0/3941454521 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ffb700046c0 con 0x7ffb801008d0 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.563+0000 7ffb88436740 1 Processor -- start 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.563+0000 7ffb88436740 1 -- start start 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8da90 0x55b3a5b97e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5af14f0 con 0x55b3a5b8d170 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5a8f870 con 0x55b3a5b8da90 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb88436740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5aed4b0 con 0x55b3a5b98350 2024-06-06T07:51:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8da90 0x55b3a5b97e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49694/0 (socket says 172.21.15.87:49694) 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 -- 172.21.15.87:0/4101803489 learned_addr learned my addr 172.21.15.87:0/4101803489 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb67fff640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 0x7ffb680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 0x7ffb680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/4101803489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8da90 msgr2=0x55b3a5b97e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8da90 0x55b3a5b97e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/4101803489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 msgr2=0x55b3a5b8d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb67fff640 1 -- 172.21.15.87:0/3941454521 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffb700cfb10 con 0x7ffb801008d0 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/4101803489 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a5aab9b0 con 0x55b3a5b98350 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b9a740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009bd0 tx=0x7ffb74011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af308c14fbb9fdf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 0x7ffb680a52e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009a60 tx=0x7ffb7802a8e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.564+0000 7ffb657fa640 1 -- 172.21.15.87:0/4101803489 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74004230 con 0x55b3a5b98350 2024-06-06T07:51:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb657fa640 1 -- 172.21.15.87:0/4101803489 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb74009e40 con 0x55b3a5b98350 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb657fa640 1 -- 172.21.15.87:0/4101803489 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74004db0 con 0x55b3a5b98350 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 -- 172.21.15.87:0/4101803489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 msgr2=0x55b3a5b9a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b9a740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009bd0 tx=0x7ffb74011810 comp rx=0 tx=0).stop 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 -- 172.21.15.87:0/4101803489 shutdown_connections 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b9a740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8da90 0x55b3a5b97e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 --2- 172.21.15.87:0/4101803489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8d170 0x55b3a5b8d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 -- 172.21.15.87:0/4101803489 >> 172.21.15.87:0/4101803489 conn(0x55b3a5b89680 msgr2=0x55b3a5b89a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 -- 172.21.15.87:0/4101803489 shutdown_connections 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.565+0000 7ffb88436740 1 -- 172.21.15.87:0/4101803489 wait complete. 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 -- start start 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 Processor -- start 2024-06-06T07:51:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5b952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b95d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5b9af80 con 0x55b3a5b8da90 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5a7aae0 con 0x55b3a5b8d170 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5b9b100 con 0x55b3a5b98350 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb88436740 10 client.0 _ll_register_callbacks cb 0x55b3a5ab39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49708/0 (socket says 172.21.15.87:49708) 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3333918115 learned_addr learned my addr 172.21.15.87:0/3333918115 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5b952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b95d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3333918115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 msgr2=0x55b3a5b95d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b95d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 msgr2=0x55b3a5b952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5b952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3333918115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a5805210 con 0x55b3a5b8da90 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5b952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b95d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffb74005890 tx=0x7ffb74003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662fa73dec78336 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.566+0000 7ffb62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74027020 con 0x55b3a5b8da90 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffb74003c10 con 0x55b3a5b8da90 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3a5b04f10 con 0x55b3a5b8da90 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb74032050 con 0x55b3a5b8da90 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 10 client.16296 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3a58d1620 con 0x55b3a5b8da90 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b3a5b04f10 con 0x55b3a5b8da90 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffb74018070 con 0x55b3a5b8da90 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 10 client.16296 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 20 client.16296 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5ba83f0 0x55b3a5baa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 4 client.16296 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a5afec20 con 0x55b3a5ba83f0 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.567+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5ba83f0 0x55b3a5baa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.568+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5ba83f0 0x55b3a5baa7e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a09fed7844c3cbe server_cookie=5fecaafb55e98831 in_seq=0 out_seq=0 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.568+0000 7ffb62ff5640 10 client.16296 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3333918115 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b3a5afec20 con 0x55b3a5ba83f0 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb62ff5640 10 client.16296 handle_command_reply: tid=0 2024-06-06T07:51:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 client.16296 shutdown 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 msgr2=0x55b3a5b95800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffb74005890 tx=0x7ffb74003820 comp rx=0 tx=0).stop 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 shutdown_connections 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5ba83f0 0x55b3a5baa7e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b95d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5b95800 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb62ff5640 0 client.16296 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 --2- 172.21.15.87:0/3333918115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5b952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 >> 172.21.15.87:0/3333918115 conn(0x55b3a5b89680 msgr2=0x55b3a5b8f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 shutdown_connections 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 1 -- 172.21.15.87:0/3333918115 wait complete. 2024-06-06T07:51:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.569+0000 7ffb88436740 20 client.16296 trim_cache size 0 max 16384 2024-06-06T07:51:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 Processor -- start 2024-06-06T07:51:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 -- start start 2024-06-06T07:51:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5bb0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 0x55b3a5bb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801b1740 con 0x55b3a5b98350 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801af010 con 0x55b3a5b8d170 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.586+0000 7ffb88436740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801ace60 con 0x55b3a5b8da90 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5bb0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 0x55b3a5bb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49718/0 (socket says 172.21.15.87:49718) 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7f7fe640 1 -- 172.21.15.87:0/56902036 learned_addr learned my addr 172.21.15.87:0/56902036 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 -- 172.21.15.87:0/56902036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 msgr2=0x55b3a5bb0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 0x55b3a5bb0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 -- 172.21.15.87:0/56902036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 msgr2=0x55b3a5b8eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 -- 172.21.15.87:0/56902036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a5baab50 con 0x55b3a5b8d170 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 0x55b3a5bb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb7effd640 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5bb0040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb700cce90 tx=0x7ffb70004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f615c21844a07324 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb64ff9640 1 -- 172.21.15.87:0/56902036 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb700d4040 con 0x55b3a5b8d170 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb64ff9640 1 -- 172.21.15.87:0/56902036 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb700b1070 con 0x55b3a5b8d170 2024-06-06T07:51:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.587+0000 7ffb64ff9640 1 -- 172.21.15.87:0/56902036 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70099ff0 con 0x55b3a5b8d170 2024-06-06T07:51:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- 172.21.15.87:0/56902036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 msgr2=0x55b3a5bb0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5bb0040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb700cce90 tx=0x7ffb70004a40 comp rx=0 tx=0).stop 2024-06-06T07:51:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- 172.21.15.87:0/56902036 shutdown_connections 2024-06-06T07:51:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b98350 0x55b3a5b8eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b8da90 0x55b3a5bb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 --2- 172.21.15.87:0/56902036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5bb0040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- 172.21.15.87:0/56902036 >> 172.21.15.87:0/56902036 conn(0x55b3a5b89680 msgr2=0x55b3a5b90590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- 172.21.15.87:0/56902036 shutdown_connections 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- 172.21.15.87:0/56902036 wait complete. 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 -- start start 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.588+0000 7ffb88436740 1 Processor -- start 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5ba0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b91940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801ad4d0 con 0x55b3a5b8da90 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801ae940 con 0x55b3a5b8d170 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801adba0 con 0x55b3a5b98350 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 10 client.0 _ll_register_callbacks cb 0x7ffb70005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49734/0 (socket says 172.21.15.87:49734) 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3724000228 learned_addr learned my addr 172.21.15.87:0/3724000228 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5ba0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7f7fe640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b91940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3724000228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 msgr2=0x55b3a5b91940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b91940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 msgr2=0x55b3a5ba0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5ba0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 -- 172.21.15.87:0/3724000228 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a5b91e80 con 0x55b3a5b8da90 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb7e7fc640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffb74011d10 tx=0x7ffb74011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00165a5576e7831 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74027070 con 0x55b3a5b8da90 2024-06-06T07:51:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffb74003a90 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.589+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb700b1070 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb7401e040 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 10 client.16302 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3a5c127a0 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb700b1070 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffb74018040 con 0x55b3a5b8da90 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 10 client.16302 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 20 client.16302 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5c12e60 0x55b3a5c15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 4 client.16302 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3a5af8f70 con 0x55b3a5c12e60 2024-06-06T07:51:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.590+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5c12e60 0x55b3a5c15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.591+0000 7ffb62ff5640 10 client.16302 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.591+0000 7ffb7effd640 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5c12e60 0x55b3a5c15280 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e3238330d7b5ac server_cookie=2630e904116111a1 in_seq=0 out_seq=0 2024-06-06T07:51:33.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 64938 3.78 MB/sec warmup 97 sec latency 41.442 ms 2024-06-06T07:51:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb62ff5640 1 -- 172.21.15.87:0/3724000228 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b3a5af8f70 con 0x55b3a5c12e60 2024-06-06T07:51:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb62ff5640 10 client.16302 handle_command_reply: tid=0 2024-06-06T07:51:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 client.16302 shutdown 2024-06-06T07:51:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 msgr2=0x55b3a5ba0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffb74011d10 tx=0x7ffb74011eb0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 shutdown_connections 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3a5c12e60 0x55b3a5c15280 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3a5b98350 0x55b3a5b91940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb62ff5640 0 client.16302 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3a5b8da90 0x55b3a5ba0890 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 --2- 172.21.15.87:0/3724000228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3a5b8d170 0x55b3a5ba0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 >> 172.21.15.87:0/3724000228 conn(0x55b3a5b89680 msgr2=0x55b3a5b95260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 shutdown_connections 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 1 -- 172.21.15.87:0/3724000228 wait complete. 2024-06-06T07:51:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.711+0000 7ffb88436740 20 client.16302 trim_cache size 0 max 16384 2024-06-06T07:51:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:51:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:51:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:51:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 msgr2=0x7ffb680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 0x7ffb680a52e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009a60 tx=0x7ffb7802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 msgr2=0x7ffb80198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000e9f0 tx=0x7ffb7000edf0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 shutdown_connections 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffb680a2e20 0x7ffb680a52e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80106890 0x7ffb801a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb80101290 0x7ffb80199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 --2- 172.21.15.87:0/3941454521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb801008d0 0x7ffb80198fe0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 >> 172.21.15.87:0/3941454521 conn(0x7ffb80077a20 msgr2=0x7ffb80103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 shutdown_connections 2024-06-06T07:51:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:33.712+0000 7ffb8564d640 1 -- 172.21.15.87:0/3941454521 wait complete. 2024-06-06T07:51:33.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:33 smithi181 ceph-mon[36790]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-06-06T07:51:33.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:33 smithi087 ceph-mon[32276]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-06-06T07:51:34.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:33 smithi195 ceph-mon[37040]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-06-06T07:51:34.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 67812 3.92 MB/sec warmup 98 sec latency 48.868 ms 2024-06-06T07:51:35.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 70781 4.03 MB/sec warmup 99 sec latency 14.857 ms 2024-06-06T07:51:35.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:35 smithi181 ceph-mon[36790]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-06-06T07:51:35.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:35 smithi087 ceph-mon[32276]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-06-06T07:51:36.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:35 smithi195 ceph-mon[37040]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-06-06T07:51:36.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 72858 4.09 MB/sec warmup 100 sec latency 25.096 ms 2024-06-06T07:51:37.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 75276 4.19 MB/sec warmup 101 sec latency 16.896 ms 2024-06-06T07:51:37.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:37 smithi181 ceph-mon[36790]: pgmap v111: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-06-06T07:51:37.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:37 smithi087 ceph-mon[32276]: pgmap v111: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-06-06T07:51:38.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:37 smithi195 ceph-mon[37040]: pgmap v111: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-06-06T07:51:38.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 78322 4.33 MB/sec warmup 102 sec latency 18.013 ms 2024-06-06T07:51:39.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 80985 4.40 MB/sec warmup 103 sec latency 21.953 ms 2024-06-06T07:51:39.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:39 smithi181 ceph-mon[36790]: pgmap v112: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:39.863 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:39 smithi087 ceph-mon[32276]: pgmap v112: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:40.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:39 smithi195 ceph-mon[37040]: pgmap v112: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:40.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 83108 4.48 MB/sec warmup 104 sec latency 15.711 ms 2024-06-06T07:51:41.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 85785 4.57 MB/sec warmup 105 sec latency 15.880 ms 2024-06-06T07:51:41.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:41 smithi181 ceph-mon[36790]: pgmap v113: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:41.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:41 smithi087 ceph-mon[32276]: pgmap v113: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:42.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:41 smithi195 ceph-mon[37040]: pgmap v113: 129 pgs: 129 active+clean; 38 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-06-06T07:51:42.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 88675 4.69 MB/sec warmup 106 sec latency 90.135 ms 2024-06-06T07:51:43.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 90867 4.72 MB/sec warmup 107 sec latency 15.232 ms 2024-06-06T07:51:43.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:43 smithi181 ceph-mon[36790]: pgmap v114: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 381 op/s 2024-06-06T07:51:43.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:43 smithi087 ceph-mon[32276]: pgmap v114: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 381 op/s 2024-06-06T07:51:44.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:43 smithi195 ceph-mon[37040]: pgmap v114: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 381 op/s 2024-06-06T07:51:44.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 93072 4.80 MB/sec warmup 108 sec latency 15.688 ms 2024-06-06T07:51:45.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 96142 4.93 MB/sec warmup 109 sec latency 10.157 ms 2024-06-06T07:51:45.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:45 smithi181 ceph-mon[36790]: pgmap v115: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-06-06T07:51:45.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:45 smithi087 ceph-mon[32276]: pgmap v115: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-06-06T07:51:46.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:45 smithi195 ceph-mon[37040]: pgmap v115: 129 pgs: 129 active+clean; 45 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-06-06T07:51:46.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 98695 4.99 MB/sec warmup 110 sec latency 25.792 ms 2024-06-06T07:51:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 -- 172.21.15.87:0/106708606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa190102380 msgr2=0x7fa19010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 --2- 172.21.15.87:0/106708606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa190102380 0x7fa19010e890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa188009c10 tx=0x7fa18802a6e0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 -- 172.21.15.87:0/106708606 shutdown_connections 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 --2- 172.21.15.87:0/106708606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa190102380 0x7fa19010e890 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 --2- 172.21.15.87:0/106708606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901019c0 0x7fa190101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 --2- 172.21.15.87:0/106708606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1901079d0 0x7fa190107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 -- 172.21.15.87:0/106708606 >> 172.21.15.87:0/106708606 conn(0x7fa1900fd450 msgr2=0x7fa1900ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 -- 172.21.15.87:0/106708606 shutdown_connections 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.808+0000 7fa196aba640 1 -- 172.21.15.87:0/106708606 wait complete. 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 Processor -- start 2024-06-06T07:51:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 -- start start 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa190102380 0x7fa1901986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901079d0 0x7fa19019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111d90 con 0x7fa1901019c0 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111c10 con 0x7fa190102380 2024-06-06T07:51:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa196aba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111f10 con 0x7fa1901079d0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53674/0 (socket says 172.21.15.87:53674) 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 -- 172.21.15.87:0/2231532605 learned_addr learned my addr 172.21.15.87:0/2231532605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa18ffff640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa190102380 0x7fa1901986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa195057640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901079d0 0x7fa19019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 -- 172.21.15.87:0/2231532605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901079d0 msgr2=0x7fa19019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901079d0 0x7fa19019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 -- 172.21.15.87:0/2231532605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa190102380 msgr2=0x7fa1901986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa190102380 0x7fa1901986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.809+0000 7fa194856640 1 -- 172.21.15.87:0/2231532605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19019d160 con 0x7fa1901019c0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa194856640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa18400e9f0 tx=0x7fa18400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa18dffb640 1 -- 172.21.15.87:0/2231532605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18400ccc0 con 0x7fa1901019c0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19019d450 con 0x7fa1901019c0 2024-06-06T07:51:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa18dffb640 1 -- 172.21.15.87:0/2231532605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa184010910 con 0x7fa1901019c0 2024-06-06T07:51:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa18dffb640 1 -- 172.21.15.87:0/2231532605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840188a0 con 0x7fa1901019c0 2024-06-06T07:51:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.810+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a4ce0 con 0x7fa1901019c0 2024-06-06T07:51:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.812+0000 7fa18dffb640 1 -- 172.21.15.87:0/2231532605 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa184010ac0 con 0x7fa1901019c0 2024-06-06T07:51:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 Processor -- start 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 -- start start 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd447e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 0x55acfd44a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55acfd3a14f0 con 0x55acfd448350 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55acfd33f870 con 0x55acfd43d170 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa1997c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acfd39d4b0 con 0x55acfd43da90 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 0x55acfd44a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa195057640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa195057640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54700/0 (socket says 172.21.15.87:54700) 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa195057640 1 -- 172.21.15.87:0/2570142088 learned_addr learned my addr 172.21.15.87:0/2570142088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa194856640 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd447e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa18ffff640 1 -- 172.21.15.87:0/2570142088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 msgr2=0x55acfd447e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa18ffff640 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd447e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa18ffff640 1 -- 172.21.15.87:0/2570142088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 msgr2=0x55acfd43d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.813+0000 7fa18ffff640 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa18ffff640 1 -- 172.21.15.87:0/2570142088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfd35b9b0 con 0x55acfd448350 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa195057640 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa18dffb640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 0x7fa1800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa195057640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 0x7fa1800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa18ffff640 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 0x55acfd44a740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa17c005fd0 tx=0x7fa17c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f54dbeebb4b087 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa18dffb640 1 -- 172.21.15.87:0/2231532605 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fa1840cfb70 con 0x7fa1901019c0 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa17affd640 1 -- 172.21.15.87:0/2570142088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c01e070 con 0x55acfd448350 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa17affd640 1 -- 172.21.15.87:0/2570142088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c0035f0 con 0x55acfd448350 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.814+0000 7fa17affd640 1 -- 172.21.15.87:0/2570142088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c019690 con 0x55acfd448350 2024-06-06T07:51:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- 172.21.15.87:0/2570142088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 msgr2=0x55acfd44a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa195057640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 0x7fa1800a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1880092f0 tx=0x7fa188009280 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 0x55acfd44a740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa17c005fd0 tx=0x7fa17c011810 comp rx=0 tx=0).stop 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- 172.21.15.87:0/2570142088 shutdown_connections 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd448350 0x55acfd44a740 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd447e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 --2- 172.21.15.87:0/2570142088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43d170 0x55acfd43d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- 172.21.15.87:0/2570142088 >> 172.21.15.87:0/2570142088 conn(0x55acfd439680 msgr2=0x55acfd439a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- 172.21.15.87:0/2570142088 shutdown_connections 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- 172.21.15.87:0/2570142088 wait complete. 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 -- start start 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.815+0000 7fa1997c6740 1 Processor -- start 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd445770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd445cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55acfd44af80 con 0x55acfd43d170 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55acfd32aae0 con 0x55acfd448350 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acfd44b100 con 0x55acfd43da90 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa1997c6740 10 client.0 _ll_register_callbacks cb 0x55acfd3639b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53696/0 (socket says 172.21.15.87:53696) 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 -- 172.21.15.87:0/1004666194 learned_addr learned my addr 172.21.15.87:0/1004666194 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa18ffff640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd445770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa195057640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd445cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 -- 172.21.15.87:0/1004666194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 msgr2=0x55acfd445770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd445770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 -- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 msgr2=0x55acfd445cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd445cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 -- 172.21.15.87:0/1004666194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfd0b5210 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa195057640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd445cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa194856640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c6a0 tx=0x7fa18409bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f620a675daa24eb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.816+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840cd380 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fa18409b310 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17c0035f0 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa18409bcc0 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 10 client.16314 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acfd181620 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c0035f0 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa18409ab80 con 0x55acfd43d170 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 10 client.16314 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 20 client.16314 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd458450 0x55acfd45a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 4 client.16314 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acfd3b4f10 con 0x55acfd458450 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.817+0000 7fa18ffff640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd458450 0x55acfd45a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.818+0000 7fa16ffff640 10 client.16314 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.818+0000 7fa18ffff640 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd458450 0x55acfd45a860 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305f87435dd558ee server_cookie=cb735d79d831d472 in_seq=0 out_seq=0 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa16ffff640 1 -- 172.21.15.87:0/1004666194 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55acfd3b4f10 con 0x55acfd458450 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa16ffff640 10 client.16314 handle_command_reply: tid=0 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 client.16314 shutdown 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 msgr2=0x55acfd445230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c6a0 tx=0x7fa18409bb50 comp rx=0 tx=0).stop 2024-06-06T07:51:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 shutdown_connections 2024-06-06T07:51:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd458450 0x55acfd45a860 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd445cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd445770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 --2- 172.21.15.87:0/1004666194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd445230 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa16ffff640 0 client.16314 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 >> 172.21.15.87:0/1004666194 conn(0x55acfd439680 msgr2=0x55acfd43f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 shutdown_connections 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 1 -- 172.21.15.87:0/1004666194 wait complete. 2024-06-06T07:51:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.819+0000 7fa1997c6740 20 client.16314 trim_cache size 0 max 16384 2024-06-06T07:51:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.834+0000 7fa1997c6740 1 Processor -- start 2024-06-06T07:51:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.834+0000 7fa1997c6740 1 -- start start 2024-06-06T07:51:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd4606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd442490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901b1760 con 0x55acfd43d170 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901af030 con 0x55acfd448350 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa1997c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901ace80 con 0x55acfd43da90 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53712/0 (socket says 172.21.15.87:53712) 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 -- 172.21.15.87:0/3135803808 learned_addr learned my addr 172.21.15.87:0/3135803808 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa195057640 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd442490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa18ffff640 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd4606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 -- 172.21.15.87:0/3135803808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 msgr2=0x55acfd4606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd4606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 -- 172.21.15.87:0/3135803808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 msgr2=0x55acfd442490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd442490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 -- 172.21.15.87:0/3135803808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfd443780 con 0x55acfd43d170 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa194856640 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1840b6040 tx=0x7fa18409bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36108e1e1983296d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa17a7fc640 1 -- 172.21.15.87:0/3135803808 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18409b620 con 0x55acfd43d170 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa17a7fc640 1 -- 172.21.15.87:0/3135803808 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1840a4930 con 0x55acfd43d170 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.835+0000 7fa17a7fc640 1 -- 172.21.15.87:0/3135803808 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840a1280 con 0x55acfd43d170 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- 172.21.15.87:0/3135803808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 msgr2=0x55acfd460170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1840b6040 tx=0x7fa18409bbc0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- 172.21.15.87:0/3135803808 shutdown_connections 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd448350 0x55acfd442490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd43da90 0x55acfd4606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3135803808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd460170 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- 172.21.15.87:0/3135803808 >> 172.21.15.87:0/3135803808 conn(0x55acfd439680 msgr2=0x55acfd443b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- 172.21.15.87:0/3135803808 shutdown_connections 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- 172.21.15.87:0/3135803808 wait complete. 2024-06-06T07:51:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.836+0000 7fa1997c6740 1 -- start start 2024-06-06T07:51:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 Processor -- start 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43da90 0x55acfd4bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd448350 0x55acfd44b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901ad4f0 con 0x55acfd43d170 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901ae960 con 0x55acfd43da90 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901adbc0 con 0x55acfd448350 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa1997c6740 10 client.0 _ll_register_callbacks cb 0x7fa18400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53724/0 (socket says 172.21.15.87:53724) 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 -- 172.21.15.87:0/3560778894 learned_addr learned my addr 172.21.15.87:0/3560778894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43da90 0x55acfd4bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa195057640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd448350 0x55acfd44b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 -- 172.21.15.87:0/3560778894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd448350 msgr2=0x55acfd44b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd448350 0x55acfd44b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 -- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43da90 msgr2=0x55acfd4bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43da90 0x55acfd4bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.837+0000 7fa194856640 1 -- 172.21.15.87:0/3560778894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfd44c020 con 0x55acfd43d170 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa194856640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa18409b9c0 tx=0x7fa1840a01f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a85f27b35ceada7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18407f060 con 0x55acfd43d170 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fa184010630 con 0x55acfd43d170 2024-06-06T07:51:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1840a4930 con 0x55acfd43d170 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa18409b620 con 0x55acfd43d170 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 10 client.16320 fetch_fsmap learned FSMap version 30 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acfd44c020 con 0x55acfd43d170 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1840a4930 con 0x55acfd43d170 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa1840a1320 con 0x55acfd43d170 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 10 client.16320 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 20 client.16320 populate_metadata read hostname 'smithi087' 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd4c2f90 0x55acfd4c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 4 client.16320 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.838+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acfd3aec20 con 0x55acfd4c2f90 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.839+0000 7fa18ffff640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd4c2f90 0x55acfd4c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.839+0000 7fa18ffff640 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd4c2f90 0x55acfd4c53b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18047f42be6c6ca8 server_cookie=871f0c6e72587996 in_seq=0 out_seq=0 2024-06-06T07:51:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:46.839+0000 7fa16ffff640 10 client.16320 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa16ffff640 1 -- 172.21.15.87:0/3560778894 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55acfd3aec20 con 0x55acfd4c2f90 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa16ffff640 10 client.16320 handle_command_reply: tid=0 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 client.16320 shutdown 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 msgr2=0x55acfd4bb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa18409b9c0 tx=0x7fa1840a01f0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 shutdown_connections 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55acfd4c2f90 0x55acfd4c53b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acfd448350 0x55acfd44b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa16ffff640 0 client.16320 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acfd43da90 0x55acfd4bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 --2- 172.21.15.87:0/3560778894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55acfd43d170 0x55acfd4bb4b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 >> 172.21.15.87:0/3560778894 conn(0x55acfd439680 msgr2=0x55acfd43eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 shutdown_connections 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 1 -- 172.21.15.87:0/3560778894 wait complete. 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.037+0000 7fa1997c6740 20 client.16320 trim_cache size 0 max 16384 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:51:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:51:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 msgr2=0x7fa1800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 0x7fa1800a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1880092f0 tx=0x7fa188009280 comp rx=0 tx=0).stop 2024-06-06T07:51:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 msgr2=0x7fa190198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:51:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa18400e9f0 tx=0x7fa18400edf0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 shutdown_connections 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1800a2da0 0x7fa1800a5260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1901079d0 0x7fa19019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa190102380 0x7fa1901986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 --2- 172.21.15.87:0/2231532605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1901019c0 0x7fa190198170 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 >> 172.21.15.87:0/2231532605 conn(0x7fa1900fd450 msgr2=0x7fa1900fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 shutdown_connections 2024-06-06T07:51:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:51:47.038+0000 7fa196aba640 1 -- 172.21.15.87:0/2231532605 wait complete. 2024-06-06T07:51:47.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 100274 5.02 MB/sec warmup 111 sec latency 123.353 ms 2024-06-06T07:51:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:47 smithi181 ceph-mon[36790]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-06-06T07:51:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:47 smithi087 ceph-mon[32276]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-06-06T07:51:48.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:47 smithi195 ceph-mon[37040]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-06-06T07:51:48.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 103283 5.14 MB/sec warmup 112 sec latency 9.896 ms 2024-06-06T07:51:49.657 INFO:tasks.workunit.client.0.smithi087.stdout: 1 105931 5.20 MB/sec warmup 113 sec latency 20.265 ms 2024-06-06T07:51:49.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:49 smithi181 ceph-mon[36790]: pgmap v117: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:49.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:49 smithi087 ceph-mon[32276]: pgmap v117: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:50.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:49 smithi195 ceph-mon[37040]: pgmap v117: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:50.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 108226 5.26 MB/sec warmup 114 sec latency 14.721 ms 2024-06-06T07:51:51.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 110752 5.34 MB/sec warmup 115 sec latency 15.356 ms 2024-06-06T07:51:51.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:51 smithi181 ceph-mon[36790]: pgmap v118: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:51.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:51 smithi087 ceph-mon[32276]: pgmap v118: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:52.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:51 smithi195 ceph-mon[37040]: pgmap v118: 129 pgs: 129 active+clean; 53 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-06-06T07:51:52.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 113813 5.45 MB/sec warmup 116 sec latency 16.860 ms 2024-06-06T07:51:53.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 116462 5.52 MB/sec warmup 117 sec latency 14.673 ms 2024-06-06T07:51:53.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:53 smithi181 ceph-mon[36790]: pgmap v119: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-06-06T07:51:53.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:53 smithi087 ceph-mon[32276]: pgmap v119: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-06-06T07:51:54.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:53 smithi195 ceph-mon[37040]: pgmap v119: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-06-06T07:51:54.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 118321 5.54 MB/sec warmup 118 sec latency 18.608 ms 2024-06-06T07:51:55.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 121168 5.64 MB/sec warmup 119 sec latency 12.034 ms 2024-06-06T07:51:56.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:55 smithi195 ceph-mon[37040]: pgmap v120: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-06-06T07:51:56.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:55 smithi181 ceph-mon[36790]: pgmap v120: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-06-06T07:51:56.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:55 smithi087 ceph-mon[32276]: pgmap v120: 129 pgs: 129 active+clean; 47 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 311 op/s 2024-06-06T07:51:57.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:56 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:51:57.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:56 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:51:57.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:56 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:51:57.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 125881 6.93 MB/sec execute 1 sec latency 90.553 ms 2024-06-06T07:51:57.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:57 smithi087 ceph-mon[32276]: pgmap v121: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-06-06T07:51:58.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:57 smithi195 ceph-mon[37040]: pgmap v121: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-06-06T07:51:58.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:57 smithi181 ceph-mon[36790]: pgmap v121: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-06-06T07:51:58.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 128393 10.44 MB/sec execute 2 sec latency 19.072 ms 2024-06-06T07:51:59.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 131391 12.85 MB/sec execute 3 sec latency 14.817 ms 2024-06-06T07:52:00.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:51:59 smithi195 ceph-mon[37040]: pgmap v122: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:00.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:51:59 smithi181 ceph-mon[36790]: pgmap v122: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:00.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:51:59 smithi087 ceph-mon[32276]: pgmap v122: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 -- 172.21.15.87:0/1010919174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb101079c0 msgr2=0x7feb10107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 --2- 172.21.15.87:0/1010919174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb101079c0 0x7feb10107da0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feb00009930 tx=0x7feb0002a420 comp rx=0 tx=0).stop 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 -- 172.21.15.87:0/1010919174 shutdown_connections 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 --2- 172.21.15.87:0/1010919174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10102370 0x7feb1010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 --2- 172.21.15.87:0/1010919174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101019b0 0x7feb10101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 --2- 172.21.15.87:0/1010919174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb101079c0 0x7feb10107da0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 -- 172.21.15.87:0/1010919174 >> 172.21.15.87:0/1010919174 conn(0x7feb100fd4a0 msgr2=0x7feb100ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.135+0000 7feb15ca5640 1 -- 172.21.15.87:0/1010919174 shutdown_connections 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 -- 172.21.15.87:0/1010919174 wait complete. 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 Processor -- start 2024-06-06T07:52:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 -- start start 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101019b0 0x7feb1019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101079c0 0x7feb1019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111da0 con 0x7feb10102370 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111c20 con 0x7feb101019b0 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.136+0000 7feb15ca5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111f20 con 0x7feb101079c0 2024-06-06T07:52:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45182/0 (socket says 172.21.15.87:45182) 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 -- 172.21.15.87:0/3494715333 learned_addr learned my addr 172.21.15.87:0/3494715333 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101019b0 0x7feb1019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0ffff640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101079c0 0x7feb1019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 -- 172.21.15.87:0/3494715333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101079c0 msgr2=0x7feb1019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101079c0 0x7feb1019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 -- 172.21.15.87:0/3494715333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101019b0 msgr2=0x7feb1019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101019b0 0x7feb1019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 -- 172.21.15.87:0/3494715333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1019f410 con 0x7feb10102370 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0effd640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0cff9640 1 -- 172.21.15.87:0/3494715333 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800cd80 con 0x7feb10102370 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1019f700 con 0x7feb10102370 2024-06-06T07:52:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0cff9640 1 -- 172.21.15.87:0/3494715333 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8010910 con 0x7feb10102370 2024-06-06T07:52:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.137+0000 7feb0cff9640 1 -- 172.21.15.87:0/3494715333 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80188f0 con 0x7feb10102370 2024-06-06T07:52:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.138+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a6f90 con 0x7feb10102370 2024-06-06T07:52:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb0cff9640 1 -- 172.21.15.87:0/3494715333 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feaf8010ac0 con 0x7feb10102370 2024-06-06T07:52:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb0cff9640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 0x7feafc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 0x7feafc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb0cff9640 1 -- 172.21.15.87:0/3494715333 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7feaf80cfbd0 con 0x7feb10102370 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb189b4740 1 Processor -- start 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.140+0000 7feb189b4740 1 -- start start 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 0x7feafc0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb00009900 tx=0x7feb000058d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b0f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5600c6a664f0 con 0x5600c6b02a90 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5600c6a04870 con 0x5600c6b02170 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb189b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600c6a624b0 con 0x5600c6b0d350 2024-06-06T07:52:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45194/0 (socket says 172.21.15.87:45194) 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 -- 172.21.15.87:0/3178735818 learned_addr learned my addr 172.21.15.87:0/3178735818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 -- 172.21.15.87:0/3178735818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 msgr2=0x5600c6b0f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0ffff640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b0f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b0f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 -- 172.21.15.87:0/3178735818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 msgr2=0x5600c6b02550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b02550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 -- 172.21.15.87:0/3178735818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c6a209b0 con 0x5600c6b02a90 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0ffff640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b0f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb0effd640 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feaf80987a0 tx=0x7feaf80a00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c17844c2787739 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb05ffb640 1 -- 172.21.15.87:0/3178735818 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8098440 con 0x5600c6b02a90 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb05ffb640 1 -- 172.21.15.87:0/3178735818 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf809b220 con 0x5600c6b02a90 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.141+0000 7feb05ffb640 1 -- 172.21.15.87:0/3178735818 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf809ac90 con 0x5600c6b02a90 2024-06-06T07:52:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 -- 172.21.15.87:0/3178735818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 msgr2=0x5600c6b0ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feaf80987a0 tx=0x7feaf80a00d0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 -- 172.21.15.87:0/3178735818 shutdown_connections 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b0f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0ce10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 --2- 172.21.15.87:0/3178735818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b02550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 -- 172.21.15.87:0/3178735818 >> 172.21.15.87:0/3178735818 conn(0x5600c6afe680 msgr2=0x5600c6afea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 -- 172.21.15.87:0/3178735818 shutdown_connections 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.142+0000 7feb189b4740 1 -- 172.21.15.87:0/3178735818 wait complete. 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 -- start start 2024-06-06T07:52:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 Processor -- start 2024-06-06T07:52:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02170 0x5600c6b0a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b0ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5600c6b0ff80 con 0x5600c6b02a90 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5600c69efae0 con 0x5600c6b0d350 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600c6b10100 con 0x5600c6b02170 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb189b4740 10 client.0 _ll_register_callbacks cb 0x5600c6a289b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45208/0 (socket says 172.21.15.87:45208) 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0effd640 1 -- 172.21.15.87:0/2436544532 learned_addr learned my addr 172.21.15.87:0/2436544532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0ffff640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b0ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02170 0x5600c6b0a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0effd640 1 -- 172.21.15.87:0/2436544532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02170 msgr2=0x5600c6b0a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.143+0000 7feb0effd640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02170 0x5600c6b0a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb0effd640 1 -- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 msgr2=0x5600c6b0ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb0effd640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b0ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb0effd640 1 -- 172.21.15.87:0/2436544532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c677a210 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb0effd640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cd550 tx=0x7feaf80ccf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1145aa0724250e64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80a9060 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7feaf80a4be0 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf809b220 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feaf8097e90 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 10 client.16332 fetch_fsmap learned FSMap version 31 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5600c6846620 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf809b220 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7feaf80b3070 con 0x5600c6b02a90 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 10 client.16332 fetch_fsmap finished waiting for FSMap version 31 2024-06-06T07:52:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 20 client.16332 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b1d310 0x5600c6b1f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 4 client.16332 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.144+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600c6a79f10 con 0x5600c6b1d310 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.145+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b1d310 0x5600c6b1f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.145+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b1d310 0x5600c6b1f700 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49891323666c04ca server_cookie=714552cfb76bdae9 in_seq=0 out_seq=0 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.145+0000 7feaeeffd640 10 client.16332 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feaeeffd640 1 -- 172.21.15.87:0/2436544532 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600c6a79f10 con 0x5600c6b1d310 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feaeeffd640 10 client.16332 handle_command_reply: tid=0 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 client.16332 shutdown 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 msgr2=0x5600c6b0a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cd550 tx=0x7feaf80ccf70 comp rx=0 tx=0).stop 2024-06-06T07:52:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 shutdown_connections 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b1d310 0x5600c6b1f700 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b0ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b0a820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feaeeffd640 0 client.16332 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 --2- 172.21.15.87:0/2436544532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02170 0x5600c6b0a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.146+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 >> 172.21.15.87:0/2436544532 conn(0x5600c6afe680 msgr2=0x5600c6b04400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.147+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 shutdown_connections 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.147+0000 7feb189b4740 1 -- 172.21.15.87:0/2436544532 wait complete. 2024-06-06T07:52:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.147+0000 7feb189b4740 20 client.16332 trim_cache size 0 max 16384 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 Processor -- start 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 -- start start 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02a90 0x5600c6b22720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb101b3aa0 con 0x5600c6b02170 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb101b1370 con 0x5600c6b0d350 2024-06-06T07:52:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb189b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb101af1c0 con 0x5600c6b02a90 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45220/0 (socket says 172.21.15.87:45220) 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 -- 172.21.15.87:0/3887646816 learned_addr learned my addr 172.21.15.87:0/3887646816 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0effd640 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02a90 0x5600c6b22720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0ffff640 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 -- 172.21.15.87:0/3887646816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02a90 msgr2=0x5600c6b22720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02a90 0x5600c6b22720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 -- 172.21.15.87:0/3887646816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 msgr2=0x5600c6b22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.163+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.164+0000 7feb0f7fe640 1 -- 172.21.15.87:0/3887646816 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c6aff000 con 0x5600c6b02170 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.164+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb00005ec0 tx=0x7feb00040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5fd0950770e38d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.164+0000 7feb057fa640 1 -- 172.21.15.87:0/3887646816 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00058070 con 0x5600c6b02170 2024-06-06T07:52:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.164+0000 7feb057fa640 1 -- 172.21.15.87:0/3887646816 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000044d0 con 0x5600c6b02170 2024-06-06T07:52:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.164+0000 7feb057fa640 1 -- 172.21.15.87:0/3887646816 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00035040 con 0x5600c6b02170 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- 172.21.15.87:0/3887646816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 msgr2=0x5600c6b04b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb00005ec0 tx=0x7feb00040000 comp rx=0 tx=0).stop 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- 172.21.15.87:0/3887646816 shutdown_connections 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b0d350 0x5600c6b22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b02a90 0x5600c6b22720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 --2- 172.21.15.87:0/3887646816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02170 0x5600c6b04b30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- 172.21.15.87:0/3887646816 >> 172.21.15.87:0/3887646816 conn(0x5600c6afe680 msgr2=0x5600c6b0ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- 172.21.15.87:0/3887646816 shutdown_connections 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- 172.21.15.87:0/3887646816 wait complete. 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 -- start start 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.165+0000 7feb189b4740 1 Processor -- start 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b22f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b20870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb101af830 con 0x5600c6b02a90 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb101b0ca0 con 0x5600c6b02170 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb101aff00 con 0x5600c6b0d350 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 10 client.0 _ll_register_callbacks cb 0x7feb00040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45234/0 (socket says 172.21.15.87:45234) 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 -- 172.21.15.87:0/1920902443 learned_addr learned my addr 172.21.15.87:0/1920902443 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b22f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0ffff640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b20870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 -- 172.21.15.87:0/1920902443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 msgr2=0x5600c6b20870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b20870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 -- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 msgr2=0x5600c6b22f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b22f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 -- 172.21.15.87:0/1920902443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c6b210a0 con 0x5600c6b02a90 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b22f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb0effd640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feaf8098230 tx=0x7feaf80106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3116201a02d34b2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feaeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b8040 con 0x5600c6b02a90 2024-06-06T07:52:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7feaf8086bb0 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.166+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb000044d0 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feaf8098070 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 10 client.16338 fetch_fsmap learned FSMap version 31 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5600c6b87d30 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000044d0 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7feaf80b3040 con 0x5600c6b02a90 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 10 client.16338 fetch_fsmap finished waiting for FSMap version 31 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 20 client.16338 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b88310 0x5600c6b8a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 4 client.16338 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600c6a73c20 con 0x5600c6b88310 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b88310 0x5600c6b8a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feb0f7fe640 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b88310 0x5600c6b8a730 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac0ce2be53d8068 server_cookie=211cd430d2eaccc8 in_seq=0 out_seq=0 2024-06-06T07:52:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.167+0000 7feaeeffd640 10 client.16338 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.389+0000 7feaeeffd640 1 -- 172.21.15.87:0/1920902443 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5600c6a73c20 con 0x5600c6b88310 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.389+0000 7feaeeffd640 10 client.16338 handle_command_reply: tid=0 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.389+0000 7feb189b4740 1 client.16338 shutdown 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 msgr2=0x5600c6b202d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feaf8098230 tx=0x7feaf80106e0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 shutdown_connections 2024-06-06T07:52:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5600c6b88310 0x5600c6b8a730 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600c6b0d350 0x5600c6b20870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600c6b02a90 0x5600c6b202d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 --2- 172.21.15.87:0/1920902443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600c6b02170 0x5600c6b22f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feaeeffd640 0 client.16338 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 >> 172.21.15.87:0/1920902443 conn(0x5600c6afe680 msgr2=0x5600c6b18350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 shutdown_connections 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 1 -- 172.21.15.87:0/1920902443 wait complete. 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.390+0000 7feb189b4740 20 client.16338 trim_cache size 0 max 16384 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 msgr2=0x7feafc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 0x7feafc0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb00009900 tx=0x7feb000058d0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 msgr2=0x7feb1019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 shutdown_connections 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7feafc0a2da0 0x7feafc0a5260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb101079c0 0x7feb1019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb10102370 0x7feb1019a960 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 --2- 172.21.15.87:0/3494715333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101019b0 0x7feb1019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 >> 172.21.15.87:0/3494715333 conn(0x7feb100fd4a0 msgr2=0x7feb1010d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 shutdown_connections 2024-06-06T07:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:00.391+0000 7feb15ca5640 1 -- 172.21.15.87:0/3494715333 wait complete. 2024-06-06T07:52:00.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 132293 10.28 MB/sec execute 4 sec latency 117.301 ms 2024-06-06T07:52:01.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 133746 9.52 MB/sec execute 5 sec latency 79.309 ms 2024-06-06T07:52:02.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:01 smithi195 ceph-mon[37040]: pgmap v123: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:02.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:01 smithi181 ceph-mon[36790]: pgmap v123: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:02.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:01 smithi087 ceph-mon[32276]: pgmap v123: 129 pgs: 129 active+clean; 55 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-06-06T07:52:02.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 135813 10.00 MB/sec execute 6 sec latency 15.349 ms 2024-06-06T07:52:03.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 138594 11.12 MB/sec execute 7 sec latency 64.984 ms 2024-06-06T07:52:04.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:03 smithi195 ceph-mon[37040]: pgmap v124: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 589 op/s 2024-06-06T07:52:04.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:03 smithi181 ceph-mon[36790]: pgmap v124: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 589 op/s 2024-06-06T07:52:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:03 smithi087 ceph-mon[32276]: pgmap v124: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 589 op/s 2024-06-06T07:52:04.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 140797 10.66 MB/sec execute 8 sec latency 16.312 ms 2024-06-06T07:52:05.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 142904 10.91 MB/sec execute 9 sec latency 21.090 ms 2024-06-06T07:52:06.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:05 smithi195 ceph-mon[37040]: pgmap v125: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 368 op/s 2024-06-06T07:52:06.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:05 smithi181 ceph-mon[36790]: pgmap v125: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 368 op/s 2024-06-06T07:52:06.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:05 smithi087 ceph-mon[32276]: pgmap v125: 129 pgs: 129 active+clean; 48 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 368 op/s 2024-06-06T07:52:06.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 145856 11.64 MB/sec execute 10 sec latency 20.076 ms 2024-06-06T07:52:07.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 148633 11.78 MB/sec execute 11 sec latency 26.140 ms 2024-06-06T07:52:08.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:07 smithi195 ceph-mon[37040]: pgmap v126: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 493 op/s 2024-06-06T07:52:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:07 smithi181 ceph-mon[36790]: pgmap v126: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 493 op/s 2024-06-06T07:52:08.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:07 smithi087 ceph-mon[32276]: pgmap v126: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 493 op/s 2024-06-06T07:52:08.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 150519 11.72 MB/sec execute 12 sec latency 46.245 ms 2024-06-06T07:52:09.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 152939 11.89 MB/sec execute 13 sec latency 63.104 ms 2024-06-06T07:52:10.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:09 smithi195 ceph-mon[37040]: pgmap v127: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:10.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:09 smithi181 ceph-mon[36790]: pgmap v127: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:09 smithi087 ceph-mon[32276]: pgmap v127: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:10.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 155477 11.94 MB/sec execute 14 sec latency 19.123 ms 2024-06-06T07:52:11.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 157263 11.72 MB/sec execute 15 sec latency 15.792 ms 2024-06-06T07:52:12.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:11 smithi195 ceph-mon[37040]: pgmap v128: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:12.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:11 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:52:12.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:11 smithi181 ceph-mon[36790]: pgmap v128: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:12.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:11 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:52:12.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:11 smithi087 ceph-mon[32276]: pgmap v128: 129 pgs: 129 active+clean; 54 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-06-06T07:52:12.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:11 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:52:12.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 160076 12.05 MB/sec execute 16 sec latency 16.960 ms 2024-06-06T07:52:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 -- 172.21.15.87:0/1327296101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb81079c0 msgr2=0x7f4fb8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 --2- 172.21.15.87:0/1327296101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb81079c0 0x7f4fb8107da0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009930 tx=0x7f4fa802a420 comp rx=0 tx=0).stop 2024-06-06T07:52:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 -- 172.21.15.87:0/1327296101 shutdown_connections 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 --2- 172.21.15.87:0/1327296101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb8102370 0x7f4fb810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 --2- 172.21.15.87:0/1327296101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 0x7f4fb8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 --2- 172.21.15.87:0/1327296101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb81079c0 0x7f4fb8107da0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 -- 172.21.15.87:0/1327296101 >> 172.21.15.87:0/1327296101 conn(0x7f4fb8078e30 msgr2=0x7f4fb8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 -- 172.21.15.87:0/1327296101 shutdown_connections 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.485+0000 7f4fbe522640 1 -- 172.21.15.87:0/1327296101 wait complete. 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 Processor -- start 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 -- start start 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 0x7f4fb81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb81079c0 0x7f4fb819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111dd0 con 0x7f4fb8102370 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111c50 con 0x7f4fb81019b0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbe522640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111f50 con 0x7f4fb81079c0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39602/0 (socket says 172.21.15.87:39602) 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2623037851 learned_addr learned my addr 172.21.15.87:0/2623037851 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 0x7f4fb81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.486+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb81079c0 0x7f4fb819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2623037851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb81079c0 msgr2=0x7f4fb819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb81079c0 0x7f4fb819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2623037851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 msgr2=0x7f4fb81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 0x7f4fb81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2623037851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb819d190 con 0x7f4fb8102370 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00d950 tx=0x7f4fac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb57fa640 1 -- 172.21.15.87:0/2623037851 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac017030 con 0x7f4fb8102370 2024-06-06T07:52:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb57fa640 1 -- 172.21.15.87:0/2623037851 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fac004020 con 0x7f4fb8102370 2024-06-06T07:52:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb819d480 con 0x7f4fb8102370 2024-06-06T07:52:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fb57fa640 1 -- 172.21.15.87:0/2623037851 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0047a0 con 0x7f4fb8102370 2024-06-06T07:52:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.487+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb81a4d10 con 0x7f4fb8102370 2024-06-06T07:52:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fb57fa640 1 -- 172.21.15.87:0/2623037851 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4fac0041d0 con 0x7f4fb8102370 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 Processor -- start 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 -- start start 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 0x5653977aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5653977044f0 con 0x5653977ab350 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5653976a2870 con 0x5653977a0a90 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fc1233740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5653977004b0 con 0x5653977a0170 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 0x5653977aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:42428/0 (socket says 172.21.15.87:42428) 2024-06-06T07:52:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 0x5653977aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41354/0 (socket says 172.21.15.87:41354) 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1723980137 learned_addr learned my addr 172.21.15.87:0/1723980137 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.490+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1723980137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 msgr2=0x5653977aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fb57fa640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 0x5653977aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1723980137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 msgr2=0x5653977ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1723980137 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653976be9b0 con 0x5653977a0170 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fb57fa640 1 -- 172.21.15.87:0/2623037851 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4fac0cf130 con 0x7f4fb8102370 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009f60 tx=0x7f4fb0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a285313922b4a35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4f9e7fc640 1 -- 172.21.15.87:0/1723980137 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0022070 con 0x5653977a0170 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4f9e7fc640 1 -- 172.21.15.87:0/1723980137 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb001cdc0 con 0x5653977a0170 2024-06-06T07:52:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.491+0000 7f4f9e7fc640 1 -- 172.21.15.87:0/1723980137 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0016070 con 0x5653977a0170 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 -- 172.21.15.87:0/1723980137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 msgr2=0x5653977a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009f60 tx=0x7f4fb0009f90 comp rx=0 tx=0).stop 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 0x7f4fa00a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fb001f000 tx=0x7f4fb0022000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 -- 172.21.15.87:0/1723980137 shutdown_connections 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0a90 0x5653977aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1723980137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0170 0x5653977a0550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 -- 172.21.15.87:0/1723980137 >> 172.21.15.87:0/1723980137 conn(0x56539779c680 msgr2=0x56539779ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 -- 172.21.15.87:0/1723980137 shutdown_connections 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.492+0000 7f4fc1233740 1 -- 172.21.15.87:0/1723980137 wait complete. 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 -- start start 2024-06-06T07:52:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 Processor -- start 2024-06-06T07:52:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 0x5653977a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977ab350 0x5653977a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5653977adf80 con 0x5653977a0a90 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56539768dae0 con 0x5653977a0170 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5653977ae100 con 0x5653977ab350 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fc1233740 10 client.0 _ll_register_callbacks cb 0x5653976c69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 0x5653977a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41368/0 (socket says 172.21.15.87:41368) 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fb7fff640 1 -- 172.21.15.87:0/2882117611 learned_addr learned my addr 172.21.15.87:0/2882117611 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.493+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977ab350 0x5653977a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2882117611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977ab350 msgr2=0x5653977a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977ab350 0x5653977a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 msgr2=0x5653977a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 -- 172.21.15.87:0/2882117611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565397418210 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 0x5653977a8840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cc480 tx=0x7f4fac0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3898ae547ed21c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac09adc0 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4fac0cc730 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb001cdc0 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fac09abe0 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fc1233740 10 client.16350 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5653974e4620 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb001cdc0 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4fac0bb070 con 0x5653977a0a90 2024-06-06T07:52:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fc1233740 10 client.16350 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.494+0000 7f4fc1233740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fc1233740 20 client.16350 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5653977bb5a0 0x5653977bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fc1233740 4 client.16350 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565397717f10 con 0x5653977bb5a0 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5653977bb5a0 0x5653977bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4f977fe640 10 client.16350 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.495+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5653977bb5a0 0x5653977bd990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ef0f984016ec61 server_cookie=c868c3028a160448 in_seq=0 out_seq=0 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4f977fe640 1 -- 172.21.15.87:0/2882117611 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565397717f10 con 0x5653977bb5a0 2024-06-06T07:52:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4f977fe640 10 client.16350 handle_command_reply: tid=0 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 client.16350 shutdown 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 msgr2=0x5653977a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 0x5653977a8840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cc480 tx=0x7f4fac0051f0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 shutdown_connections 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5653977bb5a0 0x5653977bd990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977ab350 0x5653977a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977a0a90 0x5653977a8840 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4f977fe640 0 client.16350 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 --2- 172.21.15.87:0/2882117611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 >> 172.21.15.87:0/2882117611 conn(0x56539779c680 msgr2=0x5653977a2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 shutdown_connections 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.496+0000 7f4fc1233740 1 -- 172.21.15.87:0/2882117611 wait complete. 2024-06-06T07:52:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.497+0000 7f4fc1233740 20 client.16350 trim_cache size 0 max 16384 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 Processor -- start 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 -- start start 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x5653977c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81b1790 con 0x5653977ab350 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81af060 con 0x5653977a0170 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fc1233740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81aceb0 con 0x5653977a0a90 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39624/0 (socket says 172.21.15.87:39624) 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1112229920 learned_addr learned my addr 172.21.15.87:0/1112229920 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x5653977c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1112229920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 msgr2=0x5653977c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x5653977c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1112229920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 msgr2=0x5653977c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.512+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1112229920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653977ba140 con 0x5653977ab350 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0011fd0 tx=0x7f4fb0025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d55ab799184c813 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4f9dffb640 1 -- 172.21.15.87:0/1112229920 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0016040 con 0x5653977ab350 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4f9dffb640 1 -- 172.21.15.87:0/1112229920 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0035070 con 0x5653977ab350 2024-06-06T07:52:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4f9dffb640 1 -- 172.21.15.87:0/1112229920 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003d70 con 0x5653977ab350 2024-06-06T07:52:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4fc1233740 1 -- 172.21.15.87:0/1112229920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 msgr2=0x5653977a82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.513+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0011fd0 tx=0x7f4fb0025000 comp rx=0 tx=0).stop 2024-06-06T07:52:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- 172.21.15.87:0/1112229920 shutdown_connections 2024-06-06T07:52:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977a82c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x5653977c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1112229920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x5653977c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- 172.21.15.87:0/1112229920 >> 172.21.15.87:0/1112229920 conn(0x56539779c680 msgr2=0x5653977a3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- 172.21.15.87:0/1112229920 shutdown_connections 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- 172.21.15.87:0/1112229920 wait complete. 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- start start 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 Processor -- start 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x565397821330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x565397821870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ad520 con 0x5653977ab350 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ae990 con 0x5653977a0170 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81adbf0 con 0x5653977a0a90 2024-06-06T07:52:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fc1233740 10 client.0 _ll_register_callbacks cb 0x7f4fac0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x565397821330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fbcabf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39632/0 (socket says 172.21.15.87:39632) 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.514+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1695953875 learned_addr learned my addr 172.21.15.87:0/1695953875 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fb77fe640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x565397821870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1695953875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 msgr2=0x565397821870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x565397821870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 msgr2=0x565397821330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x565397821330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 -- 172.21.15.87:0/1695953875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653977b71e0 con 0x5653977ab350 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x565397821330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fbcabf640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009fe0 tx=0x7f4fb00126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5272042a2da288 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb003f020 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4fb00115f0 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb0035070 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fb004a050 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fc1233740 10 client.16356 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5653977a53e0 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.515+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0035070 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4fb000f040 con 0x5653977ab350 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 10 client.16356 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 20 client.16356 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56539782a570 0x56539782c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 4 client.16356 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:52:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565397711c20 con 0x56539782a570 2024-06-06T07:52:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56539782a570 0x56539782c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4f977fe640 10 client.16356 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.516+0000 7f4fb7fff640 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56539782a570 0x56539782c960 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72b5099ccc41dc4a server_cookie=15a1020924958970 in_seq=0 out_seq=0 2024-06-06T07:52:13.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 162556 12.08 MB/sec execute 17 sec latency 60.851 ms 2024-06-06T07:52:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4f977fe640 1 -- 172.21.15.87:0/1695953875 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565397711c20 con 0x56539782a570 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4f977fe640 10 client.16356 handle_command_reply: tid=0 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 client.16356 shutdown 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 msgr2=0x5653977b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009fe0 tx=0x7f4fb00126f0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 shutdown_connections 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56539782a570 0x56539782c960 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5653977ab350 0x5653977b3350 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5653977a0a90 0x565397821870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 --2- 172.21.15.87:0/1695953875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5653977a0170 0x565397821330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4f977fe640 0 client.16356 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 >> 172.21.15.87:0/1695953875 conn(0x56539779c680 msgr2=0x5653977a82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 shutdown_connections 2024-06-06T07:52:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 1 -- 172.21.15.87:0/1695953875 wait complete. 2024-06-06T07:52:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.804+0000 7f4fc1233740 20 client.16356 trim_cache size 0 max 16384 2024-06-06T07:52:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:52:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:52:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 msgr2=0x7f4fa00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 0x7f4fa00a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fb001f000 tx=0x7f4fb0022000 comp rx=0 tx=0).stop 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 msgr2=0x7f4fb8198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00d950 tx=0x7f4fac00de10 comp rx=0 tx=0).stop 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 shutdown_connections 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb81079c0 0x7f4fb819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fb8102370 0x7f4fb8198730 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 --2- 172.21.15.87:0/2623037851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb81019b0 0x7f4fb81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 >> 172.21.15.87:0/2623037851 conn(0x7f4fb8078e30 msgr2=0x7f4fb80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 shutdown_connections 2024-06-06T07:52:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:13.805+0000 7f4fbe522640 1 -- 172.21.15.87:0/2623037851 wait complete. 2024-06-06T07:52:14.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:13 smithi195 ceph-mon[37040]: pgmap v129: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 531 op/s 2024-06-06T07:52:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:13 smithi181 ceph-mon[36790]: pgmap v129: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 531 op/s 2024-06-06T07:52:14.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:13 smithi087 ceph-mon[32276]: pgmap v129: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 531 op/s 2024-06-06T07:52:14.658 INFO:tasks.workunit.client.0.smithi087.stdout: 1 164086 11.82 MB/sec execute 18 sec latency 156.382 ms 2024-06-06T07:52:14.965 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T07:52:14.965 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T07:52:14.965 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:52:15.224 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:15.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 166951 12.15 MB/sec execute 19 sec latency 25.979 ms 2024-06-06T07:52:16.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:15 smithi195 ceph-mon[37040]: pgmap v130: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-06-06T07:52:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:15 smithi181 ceph-mon[36790]: pgmap v130: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-06-06T07:52:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:15 smithi087 ceph-mon[32276]: pgmap v130: 129 pgs: 129 active+clean; 60 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-06-06T07:52:16.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 169211 11.94 MB/sec execute 20 sec latency 18.895 ms 2024-06-06T07:52:17.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 -- 172.21.15.87:0/3259233119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb4101390 msgr2=0x7fdfb41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:17.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 --2- 172.21.15.87:0/3259233119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb4101390 0x7fdfb41017f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009990 tx=0x7fdfa4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 -- 172.21.15.87:0/3259233119 shutdown_connections 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 --2- 172.21.15.87:0/3259233119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101d30 0x7fdfb410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 --2- 172.21.15.87:0/3259233119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb41073d0 0x7fdfb41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 --2- 172.21.15.87:0/3259233119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb4101390 0x7fdfb41017f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 -- 172.21.15.87:0/3259233119 >> 172.21.15.87:0/3259233119 conn(0x7fdfb40782c0 msgr2=0x7fdfb40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 -- 172.21.15.87:0/3259233119 shutdown_connections 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.062+0000 7fdfbb942700 1 -- 172.21.15.87:0/3259233119 wait complete. 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 Processor -- start 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 -- start start 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 0x7fdfb419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 0x7fdfb41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111630 con 0x7fdfb41073d0 2024-06-06T07:52:17.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41114b0 con 0x7fdfb4101390 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfbb942700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41117b0 con 0x7fdfb4101d30 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb96de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb8edd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 0x7fdfb419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb96de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54184/0 (socket says 172.21.15.87:54184) 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb96de700 1 -- 172.21.15.87:0/981585017 learned_addr learned my addr 172.21.15.87:0/981585017 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb9edf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 0x7fdfb41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.063+0000 7fdfb8edd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 0x7fdfb419dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:36276/0 (socket says 172.21.15.87:36276) 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 -- 172.21.15.87:0/981585017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 msgr2=0x7fdfb419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 0x7fdfb419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 -- 172.21.15.87:0/981585017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 msgr2=0x7fdfb419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 -- 172.21.15.87:0/981585017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb4198010 con 0x7fdfb41073d0 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb96de700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:17.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfb9edf700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 0x7fdfb41978b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00efc0 tx=0x7fdfac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:17.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0092e0 con 0x7fdfb41073d0 2024-06-06T07:52:17.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb41982a0 con 0x7fdfb41073d0 2024-06-06T07:52:17.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac004bb0 con 0x7fdfb41073d0 2024-06-06T07:52:17.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac004280 con 0x7fdfb41073d0 2024-06-06T07:52:17.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.064+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb406a170 con 0x7fdfb41073d0 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.065+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb4102a80 con 0x7fdfb41073d0 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.067+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdfac01b030 con 0x7fdfb41073d0 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.068+0000 7fdfaa7fc700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 0x7fdfa00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.068+0000 7fdfb96de700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 0x7fdfa00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.068+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fdfac0cf7d0 con 0x7fdfb41073d0 2024-06-06T07:52:17.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.068+0000 7fdfb96de700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 0x7fdfa00a5250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfb41989f0 tx=0x7fdfb000e040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:17.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.071+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfac010040 con 0x7fdfb41073d0 2024-06-06T07:52:17.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.329+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfb4198e10 con 0x7fdfb41073d0 2024-06-06T07:52:17.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.330+0000 7fdfaa7fc700 1 -- 172.21.15.87:0/981585017 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7fdfb4198e10 con 0x7fdfb41073d0 2024-06-06T07:52:17.330 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:52:17.330 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:52:17.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 msgr2=0x7fdfa00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:17.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 0x7fdfa00a5250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfb41989f0 tx=0x7fdfb000e040 comp rx=0 tx=0).stop 2024-06-06T07:52:17.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 msgr2=0x7fdfb41978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:17.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 0x7fdfb41978b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00efc0 tx=0x7fdfac00c3a0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 shutdown_connections 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfb4101390 0x7fdfb419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdfa00a2d90 0x7fdfa00a5250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfb41073d0 0x7fdfb41978b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 --2- 172.21.15.87:0/981585017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfb4101d30 0x7fdfb419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 >> 172.21.15.87:0/981585017 conn(0x7fdfb40782c0 msgr2=0x7fdfb40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 shutdown_connections 2024-06-06T07:52:17.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:17.332+0000 7fdfbb942700 1 -- 172.21.15.87:0/981585017 wait complete. 2024-06-06T07:52:17.334 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:52:17.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 171073 11.94 MB/sec execute 21 sec latency 52.986 ms 2024-06-06T07:52:18.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:17 smithi195 ceph-mon[37040]: pgmap v131: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-06-06T07:52:18.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:17 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/981585017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:18.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:17 smithi181 ceph-mon[36790]: pgmap v131: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-06-06T07:52:18.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:17 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/981585017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:17 smithi087 ceph-mon[32276]: pgmap v131: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-06-06T07:52:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:17 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/981585017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:18.148 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T07:52:18.149 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T07:52:18.420 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:18.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 174064 12.23 MB/sec execute 22 sec latency 10.366 ms 2024-06-06T07:52:18.821 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:52:18.821 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:52:18.821 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:52:18.821 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:52:19.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:52:19.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:52:19.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:52:19.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:52:19.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:52:19.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:52:19.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:52:19.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:52:19.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 176168 12.01 MB/sec execute 23 sec latency 53.443 ms 2024-06-06T07:52:20.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:19 smithi195 ceph-mon[37040]: pgmap v132: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:20.056 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:19 smithi087 ceph-mon[32276]: pgmap v132: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:20.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:19 smithi181 ceph-mon[36790]: pgmap v132: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:20.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.315+0000 7f54f1ffb700 1 -- 172.21.15.87:0/2368787007 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003c20 con 0x7f54f4102e20 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 -- 172.21.15.87:0/2368787007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f4102e20 msgr2=0x7f54f41032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 --2- 172.21.15.87:0/2368787007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f4102e20 0x7f54f41032a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f54e80099c0 tx=0x7f54e8009cd0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 -- 172.21.15.87:0/2368787007 shutdown_connections 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 --2- 172.21.15.87:0/2368787007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f41037e0 0x7f54f410e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 --2- 172.21.15.87:0/2368787007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4101530 0x7f54f4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 --2- 172.21.15.87:0/2368787007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f4102e20 0x7f54f41032a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 -- 172.21.15.87:0/2368787007 >> 172.21.15.87:0/2368787007 conn(0x7f54f40fcce0 msgr2=0x7f54f40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 -- 172.21.15.87:0/2368787007 shutdown_connections 2024-06-06T07:52:20.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.316+0000 7f54f9804700 1 -- 172.21.15.87:0/2368787007 wait complete. 2024-06-06T07:52:20.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 Processor -- start 2024-06-06T07:52:20.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 -- start start 2024-06-06T07:52:20.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 0x7f54f419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4111600 con 0x7f54f41037e0 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4111480 con 0x7f54f4101530 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f9804700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4111780 con 0x7f54f4102e20 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44668/0 (socket says 172.21.15.87:44668) 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54200/0 (socket says 172.21.15.87:54200) 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f37fe700 1 -- 172.21.15.87:0/2507681296 learned_addr learned my addr 172.21.15.87:0/2507681296 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.317+0000 7f54f27fc700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 0x7f54f419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 -- 172.21.15.87:0/2507681296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 msgr2=0x7f54f419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 0x7f54f419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 -- 172.21.15.87:0/2507681296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 msgr2=0x7f54f419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 -- 172.21.15.87:0/2507681296 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f4198fd0 con 0x7f54f41037e0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f27fc700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 0x7f54f419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f37fe700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ef30 tx=0x7f54ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f8802700 1 -- 172.21.15.87:0/2507681296 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0092e0 con 0x7f54f41037e0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f8802700 1 -- 172.21.15.87:0/2507681296 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec004bb0 con 0x7f54f41037e0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f4199260 con 0x7f54f41037e0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.318+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f41a5580 con 0x7f54f41037e0 2024-06-06T07:52:20.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.319+0000 7f54f8802700 1 -- 172.21.15.87:0/2507681296 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec004280 con 0x7f54f41037e0 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.321+0000 7f54f8802700 1 -- 172.21.15.87:0/2507681296 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f54ec01b030 con 0x7f54f41037e0 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.321+0000 7f54f8802700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 0x7f54dc0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.322+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 0x7f54dc0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.322+0000 7f54f8802700 1 -- 172.21.15.87:0/2507681296 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54ec0cf110 con 0x7f54f41037e0 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.322+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 0x7f54dc0a5370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54f41999b0 tx=0x7f54e4009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 Processor -- start 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 -- start start 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fafc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557895ea3100 con 0x557895fb01a0 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557895f019f0 con 0x557895fbb050 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f5504087b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557895e69600 con 0x557895faf880 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44674/0 (socket says 172.21.15.87:44674) 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f27fc700 1 -- 172.21.15.87:0/1937948777 learned_addr learned my addr 172.21.15.87:0/1937948777 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fafc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 -- 172.21.15.87:0/1937948777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 msgr2=0x557895fbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 -- 172.21.15.87:0/1937948777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 msgr2=0x557895fbab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f2ffd700 1 -- 172.21.15.87:0/1937948777 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557895e50a70 con 0x557895faf880 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f37fe700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.323+0000 7f54f27fc700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.324+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fafc60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011bb0 tx=0x7f54e4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc199665f75e799e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.324+0000 7f54d9ffb700 1 -- 172.21.15.87:0/1937948777 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4003d10 con 0x557895faf880 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.324+0000 7f54d9ffb700 1 -- 172.21.15.87:0/1937948777 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e40268e0 con 0x557895faf880 2024-06-06T07:52:20.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.324+0000 7f54d9ffb700 1 -- 172.21.15.87:0/1937948777 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e401e7e0 con 0x557895faf880 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- 172.21.15.87:0/1937948777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 msgr2=0x557895fafc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fafc60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011bb0 tx=0x7f54e4014000 comp rx=0 tx=0).stop 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- 172.21.15.87:0/1937948777 shutdown_connections 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fafc60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 --2- 172.21.15.87:0/1937948777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- 172.21.15.87:0/1937948777 >> 172.21.15.87:0/1937948777 conn(0x557895edd220 msgr2=0x557895fabd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- 172.21.15.87:0/1937948777 shutdown_connections 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- 172.21.15.87:0/1937948777 wait complete. 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.325+0000 7f5504087b80 1 -- start start 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 Processor -- start 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fb7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fb8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557895ea2160 con 0x557895fb01a0 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557895efd9b0 con 0x557895fbb050 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557895fbdb70 con 0x557895faf880 2024-06-06T07:52:20.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f5504087b80 10 client.0 _ll_register_callbacks cb 0x557895e97750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f54f27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fb7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f54f27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fb7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44686/0 (socket says 172.21.15.87:44686) 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f54f27fc700 1 -- 172.21.15.87:0/3240388897 learned_addr learned my addr 172.21.15.87:0/3240388897 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.326+0000 7f54f37fe700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fb8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 -- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 msgr2=0x557895fb8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fb8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 -- 172.21.15.87:0/3240388897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 msgr2=0x557895fb7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fb7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 -- 172.21.15.87:0/3240388897 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557895ec78f0 con 0x557895faf880 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fba7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e4026800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85bf7267a0e45d27 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4005240 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e40268e0 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54e40053e0 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54e403b050 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f5504087b80 10 client.25723 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557895fc8120 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.327+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e40268e0 con 0x557895faf880 2024-06-06T07:52:20.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f54e4030510 con 0x557895faf880 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 10 client.25723 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 20 client.25723 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557895fca5f0 0x557895fcc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 4 client.25723 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557895f0eef0 con 0x557895fca5f0 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f54f27fc700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557895fca5f0 0x557895fcc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f54d2ffd700 10 client.25723 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.328+0000 7f54f27fc700 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557895fca5f0 0x557895fcc9e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df99fb9c6e6754b server_cookie=97fb66f6145e0c8c in_seq=0 out_seq=0 2024-06-06T07:52:20.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.329+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3240388897 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557895f0eef0 con 0x557895fca5f0 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.329+0000 7f54d2ffd700 10 client.25723 handle_command_reply: tid=0 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.329+0000 7f5504087b80 1 client.25723 shutdown 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 msgr2=0x557895fba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fba7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e4026800 comp rx=0 tx=0).stop 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 shutdown_connections 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557895fca5f0 0x557895fcc9e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fb8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fba7c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f54d2ffd700 0 client.25723 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 --2- 172.21.15.87:0/3240388897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fb7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 >> 172.21.15.87:0/3240388897 conn(0x557895edd220 msgr2=0x557895fab920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 shutdown_connections 2024-06-06T07:52:20.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 1 -- 172.21.15.87:0/3240388897 wait complete. 2024-06-06T07:52:20.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.330+0000 7f5504087b80 20 client.25723 trim_cache size 0 max 16384 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 Processor -- start 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 -- start start 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895fb01a0 0x557895fb9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 0x557895fb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00116e0 con 0x557895fbb050 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000ef70 con 0x557895faf880 2024-06-06T07:52:20.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f5504087b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000cdc0 con 0x557895fb01a0 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 0x557895fb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 0x557895fb8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44692/0 (socket says 172.21.15.87:44692) 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54244/0 (socket says 172.21.15.87:54244) 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f37fe700 1 -- 172.21.15.87:0/2241291528 learned_addr learned my addr 172.21.15.87:0/2241291528 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f27fc700 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895fb01a0 0x557895fb9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 -- 172.21.15.87:0/2241291528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895fb01a0 msgr2=0x557895fb9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895fb01a0 0x557895fb9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 -- 172.21.15.87:0/2241291528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 msgr2=0x557895fb8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 0x557895fb8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.353+0000 7f54f2ffd700 1 -- 172.21.15.87:0/2241291528 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557895fa3490 con 0x557895faf880 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e40308b0 tx=0x7f54e40058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84cb8c2f8b9527fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f54d97fa700 1 -- 172.21.15.87:0/2241291528 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4043020 con 0x557895faf880 2024-06-06T07:52:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f54d97fa700 1 -- 172.21.15.87:0/2241291528 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e4030b20 con 0x557895faf880 2024-06-06T07:52:20.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f54d97fa700 1 -- 172.21.15.87:0/2241291528 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e403fb50 con 0x557895faf880 2024-06-06T07:52:20.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f5504087b80 1 -- 172.21.15.87:0/2241291528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 msgr2=0x557895fb6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.354+0000 7f5504087b80 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e40308b0 tx=0x7f54e40058e0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- 172.21.15.87:0/2241291528 shutdown_connections 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895faf880 0x557895fb6aa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fbb050 0x557895fb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- 172.21.15.87:0/2241291528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895fb01a0 0x557895fb9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- 172.21.15.87:0/2241291528 >> 172.21.15.87:0/2241291528 conn(0x557895edd220 msgr2=0x557895fb2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- 172.21.15.87:0/2241291528 shutdown_connections 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- 172.21.15.87:0/2241291528 wait complete. 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- start start 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 Processor -- start 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fc3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fc03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000d430 con 0x557895fb01a0 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000e8a0 con 0x557895fbb050 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54e000db00 con 0x557895faf880 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.355+0000 7f5504087b80 10 client.0 _ll_register_callbacks cb 0x557895e97750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fc3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fc3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:36334/0 (socket says 172.21.15.87:36334) 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fc03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f2ffd700 1 -- 172.21.15.87:0/3942307097 learned_addr learned my addr 172.21.15.87:0/3942307097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 -- 172.21.15.87:0/3942307097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 msgr2=0x557895fc3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fc3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 -- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 msgr2=0x557895fc03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:20.360 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fc03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 -- 172.21.15.87:0/3942307097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557895fc0990 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54f27fc700 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbfe60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x557895facae0 tx=0x7f54e800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb80f51430b4298b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003c70 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54e80050c0 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e4030b20 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54e8003e10 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f5504087b80 10 client.16380 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.356+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557896035dd0 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e4030b20 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f54e800fa70 con 0x557895fb01a0 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:20.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 10 client.16380 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 20 client.16380 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5578960365d0 0x5578960389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 4 client.16380 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x557895f09470 con 0x5578960365d0 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.357+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5578960365d0 0x5578960389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.358+0000 7f54d2ffd700 10 client.16380 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:20.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.358+0000 7f54f2ffd700 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5578960365d0 0x5578960389f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eae990596c76365 server_cookie=1c7b0f613471b58d in_seq=0 out_seq=0 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.474+0000 7f54d2ffd700 1 -- 172.21.15.87:0/3942307097 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557895f09470 con 0x5578960365d0 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.474+0000 7f54d2ffd700 10 client.16380 handle_command_reply: tid=0 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.474+0000 7f5504087b80 1 client.16380 shutdown 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.474+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 msgr2=0x557895fbfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.474+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbfe60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x557895facae0 tx=0x7f54e800be50 comp rx=0 tx=0).stop 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 shutdown_connections 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5578960365d0 0x5578960389f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557895fbb050 0x557895fc03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557895faf880 0x557895fc3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 --2- 172.21.15.87:0/3942307097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557895fb01a0 0x557895fbfe60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 >> 172.21.15.87:0/3942307097 conn(0x557895edd220 msgr2=0x557895fc5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54d2ffd700 0 client.16380 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 shutdown_connections 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 1 -- 172.21.15.87:0/3942307097 wait complete. 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f5504087b80 20 client.16380 trim_cache size 0 max 16384 2024-06-06T07:52:20.476 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 10, 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "fd470e96": { 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 10, 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:20.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:20.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 msgr2=0x7f54dc0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 0x7f54dc0a5370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54f41999b0 tx=0x7f54e4009260 comp rx=0 tx=0).stop 2024-06-06T07:52:20.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 msgr2=0x7f54f4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ef30 tx=0x7f54ec00c3a0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 shutdown_connections 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54f4101530 0x7f54f419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54dc0a2eb0 0x7f54dc0a5370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54f41037e0 0x7f54f4198870 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 --2- 172.21.15.87:0/2507681296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f4102e20 0x7f54f419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.475+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 >> 172.21.15.87:0/2507681296 conn(0x7f54f40fcce0 msgr2=0x7f54f406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.476+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 shutdown_connections 2024-06-06T07:52:20.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:20.476+0000 7f54f9804700 1 -- 172.21.15.87:0/2507681296 wait complete. 2024-06-06T07:52:20.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 12.01 MB/sec execute 24 sec latency 186.038 ms 2024-06-06T07:52:21.388 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fd470e96', quiesce took 6.4 seconds. Will release after: 18.85 2024-06-06T07:52:21.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 11.53 MB/sec execute 25 sec latency 1186.138 ms 2024-06-06T07:52:22.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:21 smithi195 ceph-mon[37040]: pgmap v133: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:21 smithi181 ceph-mon[36790]: pgmap v133: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:21 smithi087 ceph-mon[32276]: pgmap v133: 129 pgs: 129 active+clean; 50 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 403 op/s 2024-06-06T07:52:22.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 11.08 MB/sec execute 26 sec latency 2186.214 ms 2024-06-06T07:52:23.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 10.67 MB/sec execute 27 sec latency 3186.333 ms 2024-06-06T07:52:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:23 smithi181 ceph-mon[36790]: pgmap v134: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-06T07:52:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:23 smithi087 ceph-mon[32276]: pgmap v134: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-06T07:52:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:23 smithi195 ceph-mon[37040]: pgmap v134: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-06T07:52:24.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 10.29 MB/sec execute 28 sec latency 4186.469 ms 2024-06-06T07:52:25.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 9.94 MB/sec execute 29 sec latency 5186.555 ms 2024-06-06T07:52:26.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:25 smithi181 ceph-mon[36790]: pgmap v135: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 383 op/s 2024-06-06T07:52:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:25 smithi087 ceph-mon[32276]: pgmap v135: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 383 op/s 2024-06-06T07:52:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:25 smithi195 ceph-mon[37040]: pgmap v135: 129 pgs: 129 active+clean; 54 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 383 op/s 2024-06-06T07:52:26.659 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 9.60 MB/sec execute 30 sec latency 6186.612 ms 2024-06-06T07:52:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 -- 172.21.15.87:0/2622435017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 msgr2=0x7ffad410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 --2- 172.21.15.87:0/2622435017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad410d750 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009930 tx=0x7ffacc02a420 comp rx=0 tx=0).stop 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 -- 172.21.15.87:0/2622435017 shutdown_connections 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 --2- 172.21.15.87:0/2622435017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad410d750 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 --2- 172.21.15.87:0/2622435017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 --2- 172.21.15.87:0/2622435017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 0x7ffad4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 -- 172.21.15.87:0/2622435017 >> 172.21.15.87:0/2622435017 conn(0x7ffad4077a20 msgr2=0x7ffad4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 -- 172.21.15.87:0/2622435017 shutdown_connections 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.901+0000 7ffad9734640 1 -- 172.21.15.87:0/2622435017 wait complete. 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 Processor -- start 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 -- start start 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 0x7ffad4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4110ca0 con 0x7ffad4101290 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4110b20 con 0x7ffad4106890 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad9734640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4110e20 con 0x7ffad41008d0 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad27fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad27fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44710/0 (socket says 172.21.15.87:44710) 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad27fc640 1 -- 172.21.15.87:0/3420688239 learned_addr learned my addr 172.21.15.87:0/3420688239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.902+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 0x7ffad4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 -- 172.21.15.87:0/3420688239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 msgr2=0x7ffad4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 -- 172.21.15.87:0/3420688239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 msgr2=0x7ffad4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 0x7ffad4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 -- 172.21.15.87:0/3420688239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad41a6600 con 0x7ffad4101290 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad27fc640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d950 tx=0x7ffac800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffabbfff640 1 -- 172.21.15.87:0/3420688239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8017070 con 0x7ffad4101290 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a68f0 con 0x7ffad4101290 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffabbfff640 1 -- 172.21.15.87:0/3420688239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8009c30 con 0x7ffad4101290 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffabbfff640 1 -- 172.21.15.87:0/3420688239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800f6b0 con 0x7ffad4101290 2024-06-06T07:52:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.903+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a6e40 con 0x7ffad4101290 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.905+0000 7ffabbfff640 1 -- 172.21.15.87:0/3420688239 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ffac80109d0 con 0x7ffad4101290 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.905+0000 7ffadc442740 1 Processor -- start 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 -- start start 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562fac80550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac8ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5562fabe44f0 con 0x5562fac8b350 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5562fab82870 con 0x5562fac80a90 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffadc442740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5562fabe04b0 con 0x5562fac80170 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffabbfff640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 0x7ffabc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44724/0 (socket says 172.21.15.87:44724) 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 -- 172.21.15.87:0/1368504167 learned_addr learned my addr 172.21.15.87:0/1368504167 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad37fe640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562fac80550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac8ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 0x7ffabc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffabbfff640 1 -- 172.21.15.87:0/3420688239 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffac80cf4c0 con 0x7ffad4101290 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 -- 172.21.15.87:0/1368504167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 msgr2=0x5562fac80550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562fac80550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 -- 172.21.15.87:0/1368504167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 msgr2=0x5562fac8ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac8ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad27fc640 1 -- 172.21.15.87:0/1368504167 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fab9e9b0 con 0x5562fac8b350 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.906+0000 7ffad37fe640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562fac80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffad27fc640 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffac80cc880 tx=0x7ffac8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a97e63d8abab544 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 0x7ffabc0a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffab97fa640 1 -- 172.21.15.87:0/1368504167 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8009de0 con 0x5562fac8b350 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffab97fa640 1 -- 172.21.15.87:0/1368504167 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac809b6b0 con 0x5562fac8b350 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffab97fa640 1 -- 172.21.15.87:0/1368504167 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac809ab00 con 0x5562fac8b350 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.907+0000 7ffadc442740 1 -- 172.21.15.87:0/1368504167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 msgr2=0x5562fac8d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffac80cc880 tx=0x7ffac8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- 172.21.15.87:0/1368504167 shutdown_connections 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8d740 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac8ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- 172.21.15.87:0/1368504167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562fac80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- 172.21.15.87:0/1368504167 >> 172.21.15.87:0/1368504167 conn(0x5562fac7c680 msgr2=0x5562fac7ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- 172.21.15.87:0/1368504167 shutdown_connections 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- 172.21.15.87:0/1368504167 wait complete. 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- start start 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 Processor -- start 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac88870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac8b350 0x5562fac88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5562fac8df20 con 0x5562fac80170 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5562fab6dae0 con 0x5562fac80a90 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5562fac8e0a0 con 0x5562fac8b350 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffadc442740 10 client.0 _ll_register_callbacks cb 0x5562faba69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffad2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffad2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44734/0 (socket says 172.21.15.87:44734) 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.908+0000 7ffad2ffd640 1 -- 172.21.15.87:0/2669383285 learned_addr learned my addr 172.21.15.87:0/2669383285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad27fc640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac88870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad37fe640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac8b350 0x5562fac88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 -- 172.21.15.87:0/2669383285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac8b350 msgr2=0x5562fac88db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac8b350 0x5562fac88db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 -- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 msgr2=0x5562fac88870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac88870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 -- 172.21.15.87:0/2669383285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fa8f8210 con 0x5562fac80170 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad27fc640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffac4013960 tx=0x7ffac4013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff985caed98f6b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4003970 con 0x5562fac80170 2024-06-06T07:52:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffac4003b10 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac809b6b0 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffac4003d10 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffadc442740 10 client.16392 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562fa9c4620 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac809b6b0 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.909+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ffac401a070 con 0x5562fac80170 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 10 client.16392 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 20 client.16392 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fac9b5c0 0x5562fac9da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 4 client.16392 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562fabf7f10 con 0x5562fac9b5c0 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffad27fc640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fac9b5c0 0x5562fac9da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffab6ff5640 10 client.16392 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.910+0000 7ffad27fc640 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fac9b5c0 0x5562fac9da50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d050b452839dbd84 server_cookie=e7390bfa07a22394 in_seq=0 out_seq=0 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffab6ff5640 1 -- 172.21.15.87:0/2669383285 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5562fabf7f10 con 0x5562fac9b5c0 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffab6ff5640 10 client.16392 handle_command_reply: tid=0 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 client.16392 shutdown 2024-06-06T07:52:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 msgr2=0x5562fac88330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffac4013960 tx=0x7ffac4013e20 comp rx=0 tx=0).stop 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 shutdown_connections 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fac9b5c0 0x5562fac9da50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac8b350 0x5562fac88db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562fac88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffab6ff5640 0 client.16392 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 --2- 172.21.15.87:0/2669383285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac80170 0x5562fac88330 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 >> 172.21.15.87:0/2669383285 conn(0x5562fac7c680 msgr2=0x5562fac823b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 shutdown_connections 2024-06-06T07:52:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.911+0000 7ffadc442740 1 -- 172.21.15.87:0/2669383285 wait complete. 2024-06-06T07:52:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.912+0000 7ffadc442740 20 client.16392 trim_cache size 0 max 16384 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 Processor -- start 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 -- start start 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562faca3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562faca38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b39b0 con 0x5562fac8b350 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b1280 con 0x5562fac80a90 2024-06-06T07:52:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffadc442740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41af0d0 con 0x5562fac80170 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44748/0 (socket says 172.21.15.87:44748) 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 -- 172.21.15.87:0/3844814725 learned_addr learned my addr 172.21.15.87:0/3844814725 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562faca3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad27fc640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562faca38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 -- 172.21.15.87:0/3844814725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 msgr2=0x5562faca3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562faca3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 -- 172.21.15.87:0/3844814725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 msgr2=0x5562faca38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562faca38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 -- 172.21.15.87:0/3844814725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fac80880 con 0x5562fac8b350 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad27fc640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562faca38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562faca3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.927+0000 7ffad37fe640 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffacc033000 tx=0x7ffacc040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b149c133fdad50 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffab8ff9640 1 -- 172.21.15.87:0/3844814725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc051070 con 0x5562fac8b350 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffab8ff9640 1 -- 172.21.15.87:0/3844814725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc003ad0 con 0x5562fac8b350 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffab8ff9640 1 -- 172.21.15.87:0/3844814725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc035040 con 0x5562fac8b350 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- 172.21.15.87:0/3844814725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 msgr2=0x5562fac85590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffacc033000 tx=0x7ffacc040070 comp rx=0 tx=0).stop 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- 172.21.15.87:0/3844814725 shutdown_connections 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac85590 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80a90 0x5562faca38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 --2- 172.21.15.87:0/3844814725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80170 0x5562faca3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- 172.21.15.87:0/3844814725 >> 172.21.15.87:0/3844814725 conn(0x5562fac7c680 msgr2=0x5562fac7ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- 172.21.15.87:0/3844814725 shutdown_connections 2024-06-06T07:52:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- 172.21.15.87:0/3844814725 wait complete. 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.928+0000 7ffadc442740 1 -- start start 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 Processor -- start 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80170 0x5562fac93bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80a90 0x5562fac8ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41af740 con 0x5562fac8b350 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b0bb0 con 0x5562fac80170 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41afe10 con 0x5562fac80a90 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffadc442740 10 client.0 _ll_register_callbacks cb 0x7ffac4009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44756/0 (socket says 172.21.15.87:44756) 2024-06-06T07:52:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 -- 172.21.15.87:0/1711581206 learned_addr learned my addr 172.21.15.87:0/1711581206 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad27fc640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80a90 0x5562fac8ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80170 0x5562fac93bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 -- 172.21.15.87:0/1711581206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80a90 msgr2=0x5562fac8ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80a90 0x5562fac8ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 -- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80170 msgr2=0x5562fac93bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80170 0x5562fac93bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 -- 172.21.15.87:0/1711581206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fac8fd40 con 0x5562fac8b350 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffad37fe640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffacc005e90 tx=0x7ffacc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da64b7592af33ce1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.929+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc035070 con 0x5562fac8b350 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7ffacc04c920 con 0x5562fac8b350 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffacc003ad0 con 0x5562fac8b350 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffacc05d050 con 0x5562fac8b350 2024-06-06T07:52:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 10 client.16398 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562fad06440 con 0x5562fac8b350 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc003ad0 con 0x5562fac8b350 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ffacc03a040 con 0x5562fac8b350 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 10 client.16398 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 20 client.16398 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fad06a60 0x5562fad08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 4 client.16398 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562fabf1c20 con 0x5562fad06a60 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.930+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fad06a60 0x5562fad08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.931+0000 7ffab6ff5640 10 client.16398 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:26.931+0000 7ffad2ffd640 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fad06a60 0x5562fad08e80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e7803741876b29e server_cookie=97a359efec403676 in_seq=0 out_seq=0 2024-06-06T07:52:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.057+0000 7ffab6ff5640 1 -- 172.21.15.87:0/1711581206 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562fabf1c20 con 0x5562fad06a60 2024-06-06T07:52:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.057+0000 7ffab6ff5640 10 client.16398 handle_command_reply: tid=0 2024-06-06T07:52:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.057+0000 7ffadc442740 1 client.16398 shutdown 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.057+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 msgr2=0x5562fac8f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.057+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffacc005e90 tx=0x7ffacc003820 comp rx=0 tx=0).stop 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 shutdown_connections 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5562fad06a60 0x5562fad08e80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5562fac8b350 0x5562fac8f540 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffab6ff5640 0 client.16398 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562fac80a90 0x5562fac8ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 --2- 172.21.15.87:0/1711581206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562fac80170 0x5562fac93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 >> 172.21.15.87:0/1711581206 conn(0x5562fac7c680 msgr2=0x5562fac81b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 shutdown_connections 2024-06-06T07:52:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 1 -- 172.21.15.87:0/1711581206 wait complete. 2024-06-06T07:52:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffadc442740 20 client.16398 trim_cache size 0 max 16384 2024-06-06T07:52:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:52:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:52:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:52:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 msgr2=0x7ffabc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 0x7ffabc0a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc009260 comp rx=0 tx=0).stop 2024-06-06T07:52:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 msgr2=0x7ffad41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d950 tx=0x7ffac800de10 comp rx=0 tx=0).stop 2024-06-06T07:52:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 shutdown_connections 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffabc0a2da0 0x7ffabc0a5260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad4106890 0x7ffad4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffad4101290 0x7ffad41972d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 --2- 172.21.15.87:0/3420688239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad41008d0 0x7ffad4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.058+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 >> 172.21.15.87:0/3420688239 conn(0x7ffad4077a20 msgr2=0x7ffad41032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.059+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 shutdown_connections 2024-06-06T07:52:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:27.059+0000 7ffad9734640 1 -- 172.21.15.87:0/3420688239 wait complete. 2024-06-06T07:52:27.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 9.30 MB/sec execute 31 sec latency 7186.700 ms 2024-06-06T07:52:28.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:27 smithi181 ceph-mon[36790]: pgmap v136: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-06T07:52:28.101 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:27 smithi087 ceph-mon[32276]: pgmap v136: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-06T07:52:28.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:27 smithi195 ceph-mon[37040]: pgmap v136: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-06T07:52:28.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 9.00 MB/sec execute 32 sec latency 8186.819 ms 2024-06-06T07:52:29.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 8.73 MB/sec execute 33 sec latency 9186.873 ms 2024-06-06T07:52:30.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:29 smithi181 ceph-mon[36790]: pgmap v137: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:30.062 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:29 smithi195 ceph-mon[37040]: pgmap v137: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:29 smithi087 ceph-mon[32276]: pgmap v137: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:30.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 8.47 MB/sec execute 34 sec latency 10186.908 ms 2024-06-06T07:52:31.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 8.23 MB/sec execute 35 sec latency 11186.987 ms 2024-06-06T07:52:32.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:31 smithi181 ceph-mon[36790]: pgmap v138: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:32.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:31 smithi087 ceph-mon[32276]: pgmap v138: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:32.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:31 smithi195 ceph-mon[37040]: pgmap v138: 129 pgs: 129 active+clean; 52 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-06-06T07:52:32.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 8.00 MB/sec execute 36 sec latency 12187.042 ms 2024-06-06T07:52:33.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 7.79 MB/sec execute 37 sec latency 13187.151 ms 2024-06-06T07:52:34.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:33 smithi181 ceph-mon[36790]: pgmap v139: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 284 op/s 2024-06-06T07:52:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:33 smithi087 ceph-mon[32276]: pgmap v139: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 284 op/s 2024-06-06T07:52:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:33 smithi195 ceph-mon[37040]: pgmap v139: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 284 op/s 2024-06-06T07:52:34.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 7.58 MB/sec execute 38 sec latency 14187.204 ms 2024-06-06T07:52:35.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 7.39 MB/sec execute 39 sec latency 15187.286 ms 2024-06-06T07:52:36.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:35 smithi181 ceph-mon[36790]: pgmap v140: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-06-06T07:52:36.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:35 smithi087 ceph-mon[32276]: pgmap v140: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-06-06T07:52:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:35 smithi195 ceph-mon[37040]: pgmap v140: 129 pgs: 129 active+clean; 44 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-06-06T07:52:36.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 7.20 MB/sec execute 40 sec latency 16187.384 ms 2024-06-06T07:52:37.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 7.03 MB/sec execute 41 sec latency 17187.489 ms 2024-06-06T07:52:38.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:37 smithi181 ceph-mon[36790]: pgmap v141: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-06-06T07:52:38.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:37 smithi087 ceph-mon[32276]: pgmap v141: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-06-06T07:52:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:37 smithi195 ceph-mon[37040]: pgmap v141: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-06-06T07:52:38.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.86 MB/sec execute 42 sec latency 18187.590 ms 2024-06-06T07:52:39.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.70 MB/sec execute 43 sec latency 19187.677 ms 2024-06-06T07:52:40.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:39 smithi181 ceph-mon[36790]: pgmap v142: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:40.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:39 smithi087 ceph-mon[32276]: pgmap v142: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- 172.21.15.87:0/3586905280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 msgr2=0x7fc1d4105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3586905280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d4105c50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009900 tx=0x7fc1c402a410 comp rx=0 tx=0).stop 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- 172.21.15.87:0/3586905280 shutdown_connections 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3586905280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4101540 0x7fc1d410d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3586905280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4100b80 0x7fc1d4101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3586905280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d4105c50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- 172.21.15.87:0/3586905280 >> 172.21.15.87:0/3586905280 conn(0x7fc1d4077680 msgr2=0x7fc1d4077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- 172.21.15.87:0/3586905280 shutdown_connections 2024-06-06T07:52:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- 172.21.15.87:0/3586905280 wait complete. 2024-06-06T07:52:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 Processor -- start 2024-06-06T07:52:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.151+0000 7fc1da61c640 1 -- start start 2024-06-06T07:52:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4100b80 0x7fc1d419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4101540 0x7fc1d419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4111d80 con 0x7fc1d4105870 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4111c00 con 0x7fc1d4100b80 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4111f00 con 0x7fc1d4101540 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52862/0 (socket says 172.21.15.87:52862) 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 -- 172.21.15.87:0/3720780207 learned_addr learned my addr 172.21.15.87:0/3720780207 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4101540 0x7fc1d419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4100b80 0x7fc1d419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 -- 172.21.15.87:0/3720780207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4101540 msgr2=0x7fc1d419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4101540 0x7fc1d419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 -- 172.21.15.87:0/3720780207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4100b80 msgr2=0x7fc1d419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4100b80 0x7fc1d419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 -- 172.21.15.87:0/3720780207 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d419f290 con 0x7fc1d4105870 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d8bb9640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00cce0 tx=0x7fc1cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d17fa640 1 -- 172.21.15.87:0/3720780207 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00f450 con 0x7fc1d4105870 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d17fa640 1 -- 172.21.15.87:0/3720780207 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc00fad0 con 0x7fc1d4105870 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1d17fa640 1 -- 172.21.15.87:0/3720780207 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc018c10 con 0x7fc1d4105870 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d419f580 con 0x7fc1d4105870 2024-06-06T07:52:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.152+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d41a6e10 con 0x7fc1d4105870 2024-06-06T07:52:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1d17fa640 1 -- 172.21.15.87:0/3720780207 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc1cc00f5f0 con 0x7fc1d4105870 2024-06-06T07:52:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 Processor -- start 2024-06-06T07:52:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 -- start start 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb30740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b7fba874f0 con 0x55b7fbb23a90 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b7fba25870 con 0x55b7fbb23170 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1dd408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b7fba834b0 con 0x55b7fbb2e350 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1d17fa640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 0x7fc1bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1d3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.155+0000 7fc1d3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52876/0 (socket says 172.21.15.87:52876) 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 -- 172.21.15.87:0/1907930823 learned_addr learned my addr 172.21.15.87:0/1907930823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb30740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d8bb9640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 0x7fc1bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d17fa640 1 -- 172.21.15.87:0/3720780207 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc1cc0cfff0 con 0x7fc1d4105870 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 -- 172.21.15.87:0/1907930823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 msgr2=0x55b7fbb30740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb30740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 -- 172.21.15.87:0/1907930823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 msgr2=0x55b7fbb23550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb23550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 -- 172.21.15.87:0/1907930823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7fba419b0 con 0x55b7fbb23a90 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006010 tx=0x7fc1c4031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f83a04517115d93 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1d8bb9640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 0x7fc1bc0a5260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc09bde0 tx=0x7fc1cc06bf40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1ba7fc640 1 -- 172.21.15.87:0/1907930823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c403f070 con 0x55b7fbb23a90 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1ba7fc640 1 -- 172.21.15.87:0/1907930823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c40317b0 con 0x55b7fbb23a90 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.156+0000 7fc1ba7fc640 1 -- 172.21.15.87:0/1907930823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4035070 con 0x55b7fbb23a90 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- 172.21.15.87:0/1907930823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 msgr2=0x55b7fbb2de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006010 tx=0x7fc1c4031510 comp rx=0 tx=0).stop 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- 172.21.15.87:0/1907930823 shutdown_connections 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb30740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2de10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1907930823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb23550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- 172.21.15.87:0/1907930823 >> 172.21.15.87:0/1907930823 conn(0x55b7fbb1f680 msgr2=0x55b7fbb1fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- 172.21.15.87:0/1907930823 shutdown_connections 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- 172.21.15.87:0/1907930823 wait complete. 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 -- start start 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.157+0000 7fc1dd408740 1 Processor -- start 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb2bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b7fbb30f20 con 0x55b7fbb23170 2024-06-06T07:52:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b7fba10ae0 con 0x55b7fbb23a90 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b7fbb310a0 con 0x55b7fbb2e350 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1dd408740 10 client.0 _ll_register_callbacks cb 0x55b7fba499b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52892/0 (socket says 172.21.15.87:52892) 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 -- 172.21.15.87:0/1417930633 learned_addr learned my addr 172.21.15.87:0/1417930633 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb2bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 -- 172.21.15.87:0/1417930633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 msgr2=0x55b7fbb2bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb2bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 -- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 msgr2=0x55b7fbb2b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 -- 172.21.15.87:0/1417930633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7fb79b210 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40fe0c0 tx=0x7fc1c800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10c61fc83bb4b49 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.158+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8005020 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c40317b0 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc1c80051c0 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1c8028050 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 10 client.16410 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7fb867620 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c40317b0 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc1c800f040 con 0x55b7fbb23170 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 20 client.16410 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbb3e480 0x55b7fbb40870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 4 client.16410 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7fba9af10 con 0x55b7fbb3e480 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.159+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbb3e480 0x55b7fbb40870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.160+0000 7fc1b37fe640 10 client.16410 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.160+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbb3e480 0x55b7fbb40870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a169deacd58f561f server_cookie=198b752d3f468382 in_seq=0 out_seq=0 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.160+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1417930633 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b7fba9af10 con 0x55b7fbb3e480 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.160+0000 7fc1b37fe640 10 client.16410 handle_command_reply: tid=0 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.160+0000 7fc1dd408740 1 client.16410 shutdown 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 msgr2=0x55b7fbb2b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40fe0c0 tx=0x7fc1c800ea70 comp rx=0 tx=0).stop 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 shutdown_connections 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbb3e480 0x55b7fbb40870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb2e350 0x55b7fbb2bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1b37fe640 0 client.16410 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb2b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 --2- 172.21.15.87:0/1417930633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb23170 0x55b7fbb2b2a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 >> 172.21.15.87:0/1417930633 conn(0x55b7fbb1f680 msgr2=0x55b7fbb25400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 shutdown_connections 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 1 -- 172.21.15.87:0/1417930633 wait complete. 2024-06-06T07:52:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.161+0000 7fc1dd408740 20 client.16410 trim_cache size 0 max 16384 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 Processor -- start 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 -- start start 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb25b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23a90 0x55b7fbb46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41b3850 con 0x55b7fbb2e350 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41b1120 con 0x55b7fbb23170 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1dd408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41aef70 con 0x55b7fbb23a90 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52908/0 (socket says 172.21.15.87:52908) 2024-06-06T07:52:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 -- 172.21.15.87:0/3240413587 learned_addr learned my addr 172.21.15.87:0/3240413587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23a90 0x55b7fbb46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 -- 172.21.15.87:0/3240413587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23a90 msgr2=0x55b7fbb46130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23a90 0x55b7fbb46130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 -- 172.21.15.87:0/3240413587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 msgr2=0x55b7fbb25b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb25b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 -- 172.21.15.87:0/3240413587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7fbb244f0 con 0x55b7fbb2e350 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.176+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8005fd0 tx=0x7fc1c8010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de82e05f4aac010 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1b9ffb640 1 -- 172.21.15.87:0/3240413587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8031070 con 0x55b7fbb2e350 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1b9ffb640 1 -- 172.21.15.87:0/3240413587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c80127c0 con 0x55b7fbb2e350 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1b9ffb640 1 -- 172.21.15.87:0/3240413587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80043b0 con 0x55b7fbb2e350 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- 172.21.15.87:0/3240413587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 msgr2=0x55b7fbb46670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8005fd0 tx=0x7fc1c8010940 comp rx=0 tx=0).stop 2024-06-06T07:52:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- 172.21.15.87:0/3240413587 shutdown_connections 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb46670 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23a90 0x55b7fbb46130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 --2- 172.21.15.87:0/3240413587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23170 0x55b7fbb25b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- 172.21.15.87:0/3240413587 >> 172.21.15.87:0/3240413587 conn(0x55b7fbb1f680 msgr2=0x55b7fbb265d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- 172.21.15.87:0/3240413587 shutdown_connections 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- 172.21.15.87:0/3240413587 wait complete. 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.177+0000 7fc1dd408740 1 -- start start 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 Processor -- start 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23170 0x55b7fbb362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41af5e0 con 0x55b7fbb2e350 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41b0a50 con 0x55b7fbb23a90 2024-06-06T07:52:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41afcb0 con 0x55b7fbb23170 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1dd408740 10 client.0 _ll_register_callbacks cb 0x7fc1c80103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52920/0 (socket says 172.21.15.87:52920) 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 -- 172.21.15.87:0/2445210335 learned_addr learned my addr 172.21.15.87:0/2445210335 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 -- 172.21.15.87:0/2445210335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23170 msgr2=0x55b7fbb362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23170 0x55b7fbb362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 -- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 msgr2=0x55b7fbb36800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb36800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.178+0000 7fc1d37fe640 1 -- 172.21.15.87:0/2445210335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7fbb27e90 con 0x55b7fbb2e350 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1d37fe640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000c00 tx=0x7fc1c8005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2548e22492cab598 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8031040 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c80127c0 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fc1c800e9f0 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1c8040050 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 10 client.16416 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7fbba8930 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c80127c0 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc1c8034020 con 0x55b7fbb2e350 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 10 client.16416 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 20 client.16416 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbba8fc0 0x55b7fbbab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 4 client.16416 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.179+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7fba94c20 con 0x55b7fbba8fc0 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.180+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbba8fc0 0x55b7fbbab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.180+0000 7fc1d3fff640 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbba8fc0 0x55b7fbbab3e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbab15860bb637b server_cookie=1cff659d92d39912 in_seq=0 out_seq=0 2024-06-06T07:52:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.180+0000 7fc1b37fe640 10 client.16416 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:40.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.240+0000 7fc1b37fe640 1 -- 172.21.15.87:0/2445210335 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b7fba94c20 con 0x55b7fbba8fc0 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.240+0000 7fc1b37fe640 10 client.16416 handle_command_reply: tid=0 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.240+0000 7fc1dd408740 1 client.16416 shutdown 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.240+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 msgr2=0x55b7fbb27920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.240+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000c00 tx=0x7fc1c8005c20 comp rx=0 tx=0).stop 2024-06-06T07:52:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 shutdown_connections 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b7fbba8fc0 0x55b7fbbab3e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b7fbb2e350 0x55b7fbb27920 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1b37fe640 0 client.16416 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7fbb23a90 0x55b7fbb36800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 --2- 172.21.15.87:0/2445210335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7fbb23170 0x55b7fbb362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 >> 172.21.15.87:0/2445210335 conn(0x55b7fbb1f680 msgr2=0x55b7fbb2cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 shutdown_connections 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 1 -- 172.21.15.87:0/2445210335 wait complete. 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1dd408740 20 client.16416 trim_cache size 0 max 16384 2024-06-06T07:52:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:52:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:52:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:52:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 msgr2=0x7fc1bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 0x7fc1bc0a5260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc09bde0 tx=0x7fc1cc06bf40 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 msgr2=0x7fc1d419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00cce0 tx=0x7fc1cc00a620 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 shutdown_connections 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1bc0a2da0 0x7fc1bc0a5260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1d4105870 0x7fc1d419eb60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d4101540 0x7fc1d419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 --2- 172.21.15.87:0/3720780207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1d4100b80 0x7fc1d419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 >> 172.21.15.87:0/3720780207 conn(0x7fc1d4077680 msgr2=0x7fc1d410bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 shutdown_connections 2024-06-06T07:52:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:40.241+0000 7fc1da61c640 1 -- 172.21.15.87:0/3720780207 wait complete. 2024-06-06T07:52:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:39 smithi195 ceph-mon[37040]: pgmap v142: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:40.482 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:40.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.55 MB/sec execute 44 sec latency 20187.720 ms 2024-06-06T07:52:41.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.40 MB/sec execute 45 sec latency 21187.774 ms 2024-06-06T07:52:41.972 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:41 smithi087 ceph-mon[32276]: pgmap v143: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:42.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:41 smithi181 ceph-mon[36790]: pgmap v143: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:41 smithi195 ceph-mon[37040]: pgmap v143: 129 pgs: 129 active+clean; 43 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T07:52:42.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 -- 172.21.15.87:0/2347186987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 msgr2=0x7fa29c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 --2- 172.21.15.87:0/2347186987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c105fd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa2880099c0 tx=0x7fa288009cd0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 -- 172.21.15.87:0/2347186987 shutdown_connections 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 --2- 172.21.15.87:0/2347186987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 --2- 172.21.15.87:0/2347186987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 0x7fa29c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 --2- 172.21.15.87:0/2347186987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c105fd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 -- 172.21.15.87:0/2347186987 >> 172.21.15.87:0/2347186987 conn(0x7fa29c078360 msgr2=0x7fa29c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 -- 172.21.15.87:0/2347186987 shutdown_connections 2024-06-06T07:52:42.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.398+0000 7fa2a3122700 1 -- 172.21.15.87:0/2347186987 wait complete. 2024-06-06T07:52:42.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 Processor -- start 2024-06-06T07:52:42.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 -- start start 2024-06-06T07:52:42.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 0x7fa29c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c111690 con 0x7fa29c101fd0 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c111510 con 0x7fa29c1066b0 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.399+0000 7fa2a3122700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c111810 con 0x7fa29c101620 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52942/0 (socket says 172.21.15.87:52942) 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 -- 172.21.15.87:0/494016665 learned_addr learned my addr 172.21.15.87:0/494016665 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa2a0ebe700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 0x7fa29c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa2a16bf700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 -- 172.21.15.87:0/494016665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 msgr2=0x7fa29c19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 0x7fa29c19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 -- 172.21.15.87:0/494016665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 msgr2=0x7fa29c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:42.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 -- 172.21.15.87:0/494016665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c1980d0 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa2a16bf700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa29bfff700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c010 tx=0x7fa2880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288029070 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c198360 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288003730 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288024470 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.400+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c06abe0 con 0x7fa29c101fd0 2024-06-06T07:52:42.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.401+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa29c102ab0 con 0x7fa29c101fd0 2024-06-06T07:52:42.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.403+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa28802c070 con 0x7fa29c101fd0 2024-06-06T07:52:42.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.403+0000 7fa299ffb700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 0x7fa28c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:42.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.403+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fa2880d2bd0 con 0x7fa29c101fd0 2024-06-06T07:52:42.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.403+0000 7fa2a0ebe700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 0x7fa28c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:42.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.404+0000 7fa2a0ebe700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 0x7fa28c0a5250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa29c198ab0 tx=0x7fa29000b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:42.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.407+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa288033020 con 0x7fa29c101fd0 2024-06-06T07:52:42.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.649+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa29c069180 con 0x7fa29c101fd0 2024-06-06T07:52:42.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.650+0000 7fa299ffb700 1 -- 172.21.15.87:0/494016665 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7fa29c069180 con 0x7fa29c101fd0 2024-06-06T07:52:42.650 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:52:42.650 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 msgr2=0x7fa28c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 0x7fa28c0a5250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa29c198ab0 tx=0x7fa29000b3c0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 msgr2=0x7fa29c19dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c010 tx=0x7fa2880047b0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 shutdown_connections 2024-06-06T07:52:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c1066b0 0x7fa29c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa28c0a2d90 0x7fa28c0a5250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c101620 0x7fa29c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 --2- 172.21.15.87:0/494016665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101fd0 0x7fa29c19dcf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 >> 172.21.15.87:0/494016665 conn(0x7fa29c078360 msgr2=0x7fa29c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 shutdown_connections 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:42.652+0000 7fa2a3122700 1 -- 172.21.15.87:0/494016665 wait complete. 2024-06-06T07:52:42.653 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:52:42.660 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.26 MB/sec execute 46 sec latency 22187.810 ms 2024-06-06T07:52:43.023 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:42 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/494016665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:43.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:42 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/494016665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:43.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:42 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/494016665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:43.494 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id fd470e96 --release --await' 2024-06-06T07:52:43.494 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id fd470e96 --release --await 2024-06-06T07:52:43.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.13 MB/sec execute 47 sec latency 23187.843 ms 2024-06-06T07:52:43.734 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:44.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:43 smithi181 ceph-mon[36790]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-06-06T07:52:44.102 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:43 smithi087 ceph-mon[32276]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-06-06T07:52:44.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:43 smithi195 ceph-mon[37040]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-06-06T07:52:44.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177631 6.00 MB/sec execute 48 sec latency 24187.887 ms 2024-06-06T07:52:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 -- 172.21.15.87:0/3254829050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134107420 msgr2=0x7fd134107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 --2- 172.21.15.87:0/3254829050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134107420 0x7fd134107800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd124009990 tx=0x7fd124009ca0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 -- 172.21.15.87:0/3254829050 shutdown_connections 2024-06-06T07:52:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 --2- 172.21.15.87:0/3254829050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134101d30 0x7fd13410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 --2- 172.21.15.87:0/3254829050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134107420 0x7fd134107800 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 --2- 172.21.15.87:0/3254829050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd1341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 -- 172.21.15.87:0/3254829050 >> 172.21.15.87:0/3254829050 conn(0x7fd134078340 msgr2=0x7fd134078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 -- 172.21.15.87:0/3254829050 shutdown_connections 2024-06-06T07:52:45.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.462+0000 7fd138b4c700 1 -- 172.21.15.87:0/3254829050 wait complete. 2024-06-06T07:52:45.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 Processor -- start 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 -- start start 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd13419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 0x7fd1341978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd134111660 con 0x7fd134101d30 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341114e0 con 0x7fd134107420 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd138b4c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341117e0 con 0x7fd134101370 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52958/0 (socket says 172.21.15.87:52958) 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 -- 172.21.15.87:0/1558572032 learned_addr learned my addr 172.21.15.87:0/1558572032 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd13259c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd13419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 -- 172.21.15.87:0/1558572032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 msgr2=0x7fd13419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd13419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 -- 172.21.15.87:0/1558572032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 msgr2=0x7fd1341978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd132d9d700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 0x7fd1341978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 0x7fd1341978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd131d9b700 1 -- 172.21.15.87:0/1558572032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd134198020 con 0x7fd134101d30 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd13259c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd13419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.463+0000 7fd132d9d700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 0x7fd1341978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c970 tx=0x7fd12800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd1237fe700 1 -- 172.21.15.87:0/1558572032 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128007aa0 con 0x7fd134101d30 2024-06-06T07:52:45.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd134198310 con 0x7fd134101d30 2024-06-06T07:52:45.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd1237fe700 1 -- 172.21.15.87:0/1558572032 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd128004d10 con 0x7fd134101d30 2024-06-06T07:52:45.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd1237fe700 1 -- 172.21.15.87:0/1558572032 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128005690 con 0x7fd134101d30 2024-06-06T07:52:45.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.464+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1341a45d0 con 0x7fd134101d30 2024-06-06T07:52:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.466+0000 7fd1237fe700 1 -- 172.21.15.87:0/1558572032 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd128004750 con 0x7fd134101d30 2024-06-06T07:52:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.466+0000 7fd1237fe700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 0x7fd11c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.467+0000 7fd1237fe700 1 -- 172.21.15.87:0/1558572032 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd1280cf600 con 0x7fd134101d30 2024-06-06T07:52:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.467+0000 7fd13259c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 0x7fd11c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.467+0000 7fd13259c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 0x7fd11c0a5250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.467+0000 7fd1433cfb80 1 Processor -- start 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 -- start start 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b0772e910 con 0x564b077e5770 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b076d6820 con 0x564b077e4e50 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564b0772a8d0 con 0x564b077df5e0 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52972/0 (socket says 172.21.15.87:52972) 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 -- 172.21.15.87:0/1961942638 learned_addr learned my addr 172.21.15.87:0/1961942638 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd13259c700 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd132d9d700 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 -- 172.21.15.87:0/1961942638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 msgr2=0x564b077dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077dfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 -- 172.21.15.87:0/1961942638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 msgr2=0x564b077e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.468+0000 7fd131d9b700 1 -- 172.21.15.87:0/1961942638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b076cd050 con 0x564b077e5770 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd131d9b700 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd1280981b0 tx=0x7fd12809b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3632b6971e18c78a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd120ff9700 1 -- 172.21.15.87:0/1961942638 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280a20c0 con 0x564b077e5770 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd120ff9700 1 -- 172.21.15.87:0/1961942638 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1280a43c0 con 0x564b077e5770 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd120ff9700 1 -- 172.21.15.87:0/1961942638 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128004b20 con 0x564b077e5770 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd1433cfb80 1 -- 172.21.15.87:0/1961942638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 msgr2=0x564b077df0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.469+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd1280981b0 tx=0x7fd12809b240 comp rx=0 tx=0).stop 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- 172.21.15.87:0/1961942638 shutdown_connections 2024-06-06T07:52:45.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/1961942638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e5770 0x564b077df0a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- 172.21.15.87:0/1961942638 >> 172.21.15.87:0/1961942638 conn(0x564b07707dc0 msgr2=0x564b077d8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- 172.21.15.87:0/1961942638 shutdown_connections 2024-06-06T07:52:45.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- 172.21.15.87:0/1961942638 wait complete. 2024-06-06T07:52:45.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- start start 2024-06-06T07:52:45.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 Processor -- start 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b077e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b077dd170 con 0x564b077df5e0 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b076cecf0 con 0x564b077e4e50 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564b077dd2f0 con 0x564b077e5770 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd13259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd1433cfb80 10 client.0 _ll_register_callbacks cb 0x564b076c8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd13259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52984/0 (socket says 172.21.15.87:52984) 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.470+0000 7fd13259c700 1 -- 172.21.15.87:0/2172824285 learned_addr learned my addr 172.21.15.87:0/2172824285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd132d9d700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b077e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 -- 172.21.15.87:0/2172824285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 msgr2=0x564b077e7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b077e7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 -- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 msgr2=0x564b077e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 -- 172.21.15.87:0/2172824285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0767d9d0 con 0x564b077df5e0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd13259c700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd1240053a0 tx=0x7fd124009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acd7ee287564ac5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240056b0 con 0x564b077df5e0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1280a43c0 con 0x564b077df5e0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd12400f460 con 0x564b077df5e0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd124003e10 con 0x564b077df5e0 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd1433cfb80 10 client.16434 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:45.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.471+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b077f8d10 con 0x564b077df5e0 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1280a43c0 con 0x564b077df5e0 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd12402d070 con 0x564b077df5e0 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 10 client.16434 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 10 client.16434 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 20 client.16434 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b077fb350 0x564b077fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 4 client.16434 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b0773be10 con 0x564b077fb350 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b077fb350 0x564b077fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd119ffb700 10 client.16434 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.472+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b077fb350 0x564b077fd790 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c23e444a02fb9e0 server_cookie=16fa49dc38258876 in_seq=0 out_seq=0 2024-06-06T07:52:45.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd119ffb700 1 -- 172.21.15.87:0/2172824285 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b0773be10 con 0x564b077fb350 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd119ffb700 10 client.16434 handle_command_reply: tid=0 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 client.16434 shutdown 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 msgr2=0x564b077e6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd1240053a0 tx=0x7fd124009ca0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 shutdown_connections 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b077fb350 0x564b077fd790 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b077e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd119ffb700 0 client.16434 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077df5e0 0x564b077e6d80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2172824285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e4e50 0x564b077e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 >> 172.21.15.87:0/2172824285 conn(0x564b07707dc0 msgr2=0x564b077d9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 shutdown_connections 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.473+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2172824285 wait complete. 2024-06-06T07:52:45.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.474+0000 7fd1433cfb80 20 client.16434 trim_cache size 0 max 16384 2024-06-06T07:52:45.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.493+0000 7fd1433cfb80 1 Processor -- start 2024-06-06T07:52:45.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.493+0000 7fd1433cfb80 1 -- start start 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 0x564b078035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341aec20 con 0x564b077e4e50 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341ac4b0 con 0x564b077e5770 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341aa300 con 0x564b077df5e0 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52992/0 (socket says 172.21.15.87:52992) 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 -- 172.21.15.87:0/2973329592 learned_addr learned my addr 172.21.15.87:0/2973329592 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd13259c700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 -- 172.21.15.87:0/2973329592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 msgr2=0x564b077e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 -- 172.21.15.87:0/2973329592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 msgr2=0x564b078035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd132d9d700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 0x564b078035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 0x564b078035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 -- 172.21.15.87:0/2973329592 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b077e55a0 con 0x564b077e4e50 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd13259c700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd132d9d700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 0x564b078035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd131d9b700 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd12809b2b0 tx=0x7fd1280a2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b8bc59d7c99096 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd11bfff700 1 -- 172.21.15.87:0/2973329592 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280ba070 con 0x564b077e4e50 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd11bfff700 1 -- 172.21.15.87:0/2973329592 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1280b3040 con 0x564b077e4e50 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.494+0000 7fd11bfff700 1 -- 172.21.15.87:0/2973329592 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128097e50 con 0x564b077e4e50 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2973329592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 msgr2=0x564b07803060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd12809b2b0 tx=0x7fd1280a2040 comp rx=0 tx=0).stop 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2973329592 shutdown_connections 2024-06-06T07:52:45.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077e5770 0x564b078035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077df5e0 0x564b077e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/2973329592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b07803060 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2973329592 >> 172.21.15.87:0/2973329592 conn(0x564b07707dc0 msgr2=0x564b077d80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2973329592 shutdown_connections 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- 172.21.15.87:0/2973329592 wait complete. 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 -- start start 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.495+0000 7fd1433cfb80 1 Processor -- start 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077df5e0 0x564b077f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b07865030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341aa970 con 0x564b077e4e50 2024-06-06T07:52:45.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341abde0 con 0x564b077df5e0 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341ab040 con 0x564b077e5770 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd1433cfb80 10 client.0 _ll_register_callbacks cb 0x564b076c8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52998/0 (socket says 172.21.15.87:52998) 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 -- 172.21.15.87:0/618309557 learned_addr learned my addr 172.21.15.87:0/618309557 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd13259c700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077df5e0 0x564b077f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd132d9d700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b07865030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 -- 172.21.15.87:0/618309557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 msgr2=0x564b07865030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b07865030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 -- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077df5e0 msgr2=0x564b077f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077df5e0 0x564b077f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 -- 172.21.15.87:0/618309557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b07865570 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd132d9d700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b07865030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd131d9b700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c940 tx=0x7fd12809b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cfa3189c7b017a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.496+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280ba040 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd1280860b0 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1280b3040 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd128097e50 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 10 client.16440 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b07866730 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1280b3040 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd128060ce0 con 0x564b077e4e50 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 10 client.16440 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:45.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 10 client.16440 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 20 client.16440 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b07866950 0x564b07868d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 4 client.16440 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fd470e96", "release": true, "await": true}] 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fd470e96", "release": true, "await": true}) -- 0x564b07736390 con 0x564b07866950 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.497+0000 7fd13259c700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b07866950 0x564b07868d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.498+0000 7fd13259c700 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b07866950 0x564b07868d40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd8cd4c65ba4267 server_cookie=e220c890a41fa55b in_seq=0 out_seq=0 2024-06-06T07:52:45.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.498+0000 7fd119ffb700 10 client.16440 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:45.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd119ffb700 1 -- 172.21.15.87:0/618309557 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564b07736390 con 0x564b07866950 2024-06-06T07:52:45.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd119ffb700 10 client.16440 handle_command_reply: tid=0 2024-06-06T07:52:45.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 client.16440 shutdown 2024-06-06T07:52:45.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 msgr2=0x564b077f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c940 tx=0x7fd12809b240 comp rx=0 tx=0).stop 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 shutdown_connections 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b07866950 0x564b07868d40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b077df5e0 0x564b077f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b077e5770 0x564b07865030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 --2- 172.21.15.87:0/618309557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b077e4e50 0x564b077f2c00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 >> 172.21.15.87:0/618309557 conn(0x564b07707dc0 msgr2=0x564b077dbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd119ffb700 0 client.16440 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 shutdown_connections 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 1 -- 172.21.15.87:0/618309557 wait complete. 2024-06-06T07:52:45.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.511+0000 7fd1433cfb80 20 client.16440 trim_cache size 0 max 16384 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 12, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "fd470e96": { 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 12, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:52:45.513 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:45.514 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 msgr2=0x7fd11c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 0x7fd11c0a5250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124005fb0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 msgr2=0x7fd13419dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c970 tx=0x7fd12800cd30 comp rx=0 tx=0).stop 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 shutdown_connections 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd134107420 0x7fd1341978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd11c0a2d90 0x7fd11c0a5250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd134101370 0x7fd13419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 --2- 172.21.15.87:0/1558572032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134101d30 0x7fd13419dc40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:45.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 >> 172.21.15.87:0/1558572032 conn(0x7fd134078340 msgr2=0x7fd1341000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:45.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 shutdown_connections 2024-06-06T07:52:45.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:45.512+0000 7fd138b4c700 1 -- 172.21.15.87:0/1558572032 wait complete. 2024-06-06T07:52:45.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 177790 5.89 MB/sec execute 49 sec latency 25041.478 ms 2024-06-06T07:52:46.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:45 smithi181 ceph-mon[36790]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T07:52:46.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:45 smithi087 ceph-mon[32276]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T07:52:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:45 smithi195 ceph-mon[37040]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T07:52:46.418 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fd470e96', total seconds elapsed: 31.5 2024-06-06T07:52:46.418 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T07:52:46.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:52:46.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 180714 6.06 MB/sec execute 50 sec latency 22.222 ms 2024-06-06T07:52:46.690 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:47.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 183003 6.14 MB/sec execute 51 sec latency 16.151 ms 2024-06-06T07:52:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:47 smithi087 ceph-mon[32276]: pgmap v146: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T07:52:48.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:47 smithi181 ceph-mon[36790]: pgmap v146: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T07:52:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:47 smithi195 ceph-mon[37040]: pgmap v146: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- 172.21.15.87:0/3824718652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb150104a30 msgr2=0x7fb15010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 --2- 172.21.15.87:0/3824718652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb150104a30 0x7fb15010f470 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb14800b0d0 tx=0x7fb14800b3e0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- 172.21.15.87:0/3824718652 shutdown_connections 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 --2- 172.21.15.87:0/3824718652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb150104a30 0x7fb15010f470 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 --2- 172.21.15.87:0/3824718652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15006af80 0x7fb15006b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 --2- 172.21.15.87:0/3824718652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104070 0x7fb1501044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- 172.21.15.87:0/3824718652 >> 172.21.15.87:0/3824718652 conn(0x7fb150077400 msgr2=0x7fb150077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- 172.21.15.87:0/3824718652 shutdown_connections 2024-06-06T07:52:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- 172.21.15.87:0/3824718652 wait complete. 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 Processor -- start 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.445+0000 7fb15796f700 1 -- start start 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb150104070 0x7fb1501019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104a30 0x7fb150101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb150112930 con 0x7fb15006af80 2024-06-06T07:52:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501127b0 con 0x7fb150104a30 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15796f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb150112ab0 con 0x7fb150104070 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51036/0 (socket says 172.21.15.87:51036) 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 -- 172.21.15.87:0/2754253662 learned_addr learned my addr 172.21.15.87:0/2754253662 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb155f0c700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104a30 0x7fb150101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb154f0a700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb150104070 0x7fb1501019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 -- 172.21.15.87:0/2754253662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb150104070 msgr2=0x7fb1501019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb150104070 0x7fb1501019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 -- 172.21.15.87:0/2754253662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104a30 msgr2=0x7fb150101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104a30 0x7fb150101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 -- 172.21.15.87:0/2754253662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1501a7180 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb15570b700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d950 tx=0x7fb14c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.446+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c016070 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.447+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1501a7470 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.447+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c00fd70 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.447+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00e610 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.447+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1501a7a00 con 0x7fb15006af80 2024-06-06T07:52:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.448+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb150069180 con 0x7fb15006af80 2024-06-06T07:52:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.449+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb14c019070 con 0x7fb15006af80 2024-06-06T07:52:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.449+0000 7fb1467fc700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 0x7fb13c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.450+0000 7fb154f0a700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 0x7fb13c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.450+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fb14c0cf2b0 con 0x7fb15006af80 2024-06-06T07:52:48.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.450+0000 7fb154f0a700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 0x7fb13c0a5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb150077920 tx=0x7fb140009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:48.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.453+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb14c09bb00 con 0x7fb15006af80 2024-06-06T07:52:48.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 185193 6.29 MB/sec execute 52 sec latency 21.305 ms 2024-06-06T07:52:48.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.723+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb150102970 con 0x7fb15006af80 2024-06-06T07:52:48.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.724+0000 7fb1467fc700 1 -- 172.21.15.87:0/2754253662 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7fb14c09b400 con 0x7fb15006af80 2024-06-06T07:52:48.725 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:52:48.725 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.727+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 msgr2=0x7fb13c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.727+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 0x7fb13c0a5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb150077920 tx=0x7fb140009260 comp rx=0 tx=0).stop 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 msgr2=0x7fb150101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d950 tx=0x7fb14c00dc60 comp rx=0 tx=0).stop 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 shutdown_connections 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb150104a30 0x7fb150101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb13c0a2d90 0x7fb13c0a5250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb15006af80 0x7fb150101490 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 --2- 172.21.15.87:0/2754253662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb150104070 0x7fb1501019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 >> 172.21.15.87:0/2754253662 conn(0x7fb150077400 msgr2=0x7fb1500fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 shutdown_connections 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:48.728+0000 7fb15796f700 1 -- 172.21.15.87:0/2754253662 wait complete. 2024-06-06T07:52:48.729 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:52:49.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:48 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2754253662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:49.106 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:48 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2754253662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:49.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:48 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2754253662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:52:49.575 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T07:52:49.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T07:52:49.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 188074 6.50 MB/sec execute 53 sec latency 77.457 ms 2024-06-06T07:52:49.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:49 smithi087 ceph-mon[32276]: pgmap v147: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:49.868 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:52:50.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:49 smithi181 ceph-mon[36790]: pgmap v147: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:50.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:49 smithi195 ceph-mon[37040]: pgmap v147: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:50.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 189565 6.50 MB/sec execute 54 sec latency 126.725 ms 2024-06-06T07:52:51.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192104 6.64 MB/sec execute 55 sec latency 42.090 ms 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 -- 172.21.15.87:0/4019631410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 msgr2=0x7fd34810e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/4019631410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34810e290 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd34000b0d0 tx=0x7fd34000b3e0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 -- 172.21.15.87:0/4019631410 shutdown_connections 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/4019631410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34810e290 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/4019631410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 0x7fd3481019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/4019631410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd348105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 -- 172.21.15.87:0/4019631410 >> 172.21.15.87:0/4019631410 conn(0x7fd348078330 msgr2=0x7fd348078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 -- 172.21.15.87:0/4019631410 shutdown_connections 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.768+0000 7fd34ecc4700 1 -- 172.21.15.87:0/4019631410 wait complete. 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 Processor -- start 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 -- start start 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 0x7fd3481a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd3481a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd348110610 con 0x7fd3481066c0 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd348110490 con 0x7fd348101fa0 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34ecc4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd348110790 con 0x7fd3481015f0 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34d261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34d261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51056/0 (socket says 172.21.15.87:51056) 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.769+0000 7fd34d261700 1 -- 172.21.15.87:0/2843796359 learned_addr learned my addr 172.21.15.87:0/2843796359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd347fff700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd3481a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 0x7fd3481a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 -- 172.21.15.87:0/2843796359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 msgr2=0x7fd3481a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 0x7fd3481a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 -- 172.21.15.87:0/2843796359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 msgr2=0x7fd3481a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd3481a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 -- 172.21.15.87:0/2843796359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34819b440 con 0x7fd3481066c0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd347fff700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd3481a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34d261700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd340006070 tx=0x7fd340007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd345ffb700 1 -- 172.21.15.87:0/2843796359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340003c00 con 0x7fd3481066c0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd345ffb700 1 -- 172.21.15.87:0/2843796359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd340022070 con 0x7fd3481066c0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34819b6d0 con 0x7fd3481066c0 2024-06-06T07:52:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd345ffb700 1 -- 172.21.15.87:0/2843796359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34001b6c0 con 0x7fd3481066c0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.770+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3481a79c0 con 0x7fd3481066c0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.773+0000 7fd345ffb700 1 -- 172.21.15.87:0/2843796359 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd34001d470 con 0x7fd3481066c0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.774+0000 7fd345ffb700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 0x7fd3380a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.774+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 0x7fd3380a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.774+0000 7fd345ffb700 1 -- 172.21.15.87:0/2843796359 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd34000f020 con 0x7fd3481066c0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 0x7fd3380a5370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b3c0 tx=0x7fd33c00d040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 Processor -- start 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 -- start start 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55da4f91f9b0 con 0x55da4f9ce470 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55da4f8c78c0 con 0x55da4f9d4600 2024-06-06T07:52:51.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd359547b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da4f91b970 con 0x55da4f9d3ce0 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd34d261700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd347fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52922/0 (socket says 172.21.15.87:52922) 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd347fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41282/0 (socket says 172.21.15.87:41282) 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.775+0000 7fd34ca60700 1 -- 172.21.15.87:0/2946178626 learned_addr learned my addr 172.21.15.87:0/2946178626 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 -- 172.21.15.87:0/2946178626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 msgr2=0x55da4f9d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 -- 172.21.15.87:0/2946178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 msgr2=0x55da4f9cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 -- 172.21.15.87:0/2946178626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4f8be0f0 con 0x55da4f9ce470 2024-06-06T07:52:51.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd347fff700 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:51.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd34d261700 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9ce8b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd3400cd850 tx=0x7fd3400989d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a9c3119ffde48 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd332ffd700 1 -- 172.21.15.87:0/2946178626 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34009c010 con 0x55da4f9ce470 2024-06-06T07:52:51.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd332ffd700 1 -- 172.21.15.87:0/2946178626 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34009cdb0 con 0x55da4f9ce470 2024-06-06T07:52:51.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.776+0000 7fd332ffd700 1 -- 172.21.15.87:0/2946178626 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340098d20 con 0x55da4f9ce470 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 -- 172.21.15.87:0/2946178626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 msgr2=0x55da4f9ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9ce8b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd3400cd850 tx=0x7fd3400989d0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 -- 172.21.15.87:0/2946178626 shutdown_connections 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9ce8b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 --2- 172.21.15.87:0/2946178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 -- 172.21.15.87:0/2946178626 >> 172.21.15.87:0/2946178626 conn(0x55da4f8f8e60 msgr2=0x55da4f9c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 -- 172.21.15.87:0/2946178626 shutdown_connections 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.778+0000 7fd359547b80 1 -- 172.21.15.87:0/2946178626 wait complete. 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 -- start start 2024-06-06T07:52:51.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 Processor -- start 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55da4f9dad90 con 0x55da4f9ce470 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55da4f8bfd90 con 0x55da4f9d4600 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da4f9daf10 con 0x55da4f9d3ce0 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.779+0000 7fd359547b80 10 client.0 _ll_register_callbacks cb 0x55da4f8b9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51070/0 (socket says 172.21.15.87:51070) 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 -- 172.21.15.87:0/3795526704 learned_addr learned my addr 172.21.15.87:0/3795526704 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd347fff700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 -- 172.21.15.87:0/3795526704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 msgr2=0x55da4f9d7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 -- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 msgr2=0x55da4f9d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 -- 172.21.15.87:0/3795526704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4f86ea70 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd34ca60700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005920 tx=0x7fd33c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b90062bcae252c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.780+0000 7fd327fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c016640 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd33c009050 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34009cdb0 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd33c032050 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd359547b80 10 client.16458 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55da4f9e5510 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.781+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34009cdb0 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd33c0248e0 con 0x55da4f9ce470 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:51.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 10 client.16458 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 20 client.16458 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4f9e7ba0 0x55da4f9e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 4 client.16458 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da4f92ceb0 con 0x55da4f9e7ba0 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd347fff700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4f9e7ba0 0x55da4f9e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.782+0000 7fd327fff700 10 client.16458 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.783+0000 7fd347fff700 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4f9e7ba0 0x55da4f9e9fb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331c9c676a4ee425 server_cookie=793855765e061c57 in_seq=0 out_seq=0 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd327fff700 1 -- 172.21.15.87:0/3795526704 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55da4f92ceb0 con 0x55da4f9e7ba0 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd327fff700 10 client.16458 handle_command_reply: tid=0 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 client.16458 shutdown 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 msgr2=0x55da4f9da440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005920 tx=0x7fd33c004750 comp rx=0 tx=0).stop 2024-06-06T07:52:51.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 shutdown_connections 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4f9e7ba0 0x55da4f9e9fb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d4600 0x55da4f9d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9da440 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 --2- 172.21.15.87:0/3795526704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d3ce0 0x55da4f9d7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 >> 172.21.15.87:0/3795526704 conn(0x55da4f8f8e60 msgr2=0x55da4f9ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd327fff700 0 client.16458 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 shutdown_connections 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.784+0000 7fd359547b80 1 -- 172.21.15.87:0/3795526704 wait complete. 2024-06-06T07:52:51.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.785+0000 7fd359547b80 20 client.16458 trim_cache size 0 max 16384 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 Processor -- start 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 -- start start 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d4600 0x55da4f9efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b3050 con 0x55da4f9ce470 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b08e0 con 0x55da4f9d3ce0 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd359547b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481ae730 con 0x55da4f9d4600 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd34ca60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51086/0 (socket says 172.21.15.87:51086) 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd34ca60700 1 -- 172.21.15.87:0/2213905186 learned_addr learned my addr 172.21.15.87:0/2213905186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:51.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd347fff700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.806+0000 7fd34d261700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d4600 0x55da4f9efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 -- 172.21.15.87:0/2213905186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d4600 msgr2=0x55da4f9efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d4600 0x55da4f9efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 -- 172.21.15.87:0/2213905186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 msgr2=0x55da4f9ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 -- 172.21.15.87:0/2213905186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4f9d5cb0 con 0x55da4f9ce470 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd347fff700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:52:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.807+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd33c024d80 tx=0x7fd33c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe69d0aa306167e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.808+0000 7fd3327fc700 1 -- 172.21.15.87:0/2213905186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c03b070 con 0x55da4f9ce470 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.808+0000 7fd3327fc700 1 -- 172.21.15.87:0/2213905186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c029070 con 0x55da4f9ce470 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.808+0000 7fd3327fc700 1 -- 172.21.15.87:0/2213905186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005040 con 0x55da4f9ce470 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- 172.21.15.87:0/2213905186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 msgr2=0x55da4f9d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd33c024d80 tx=0x7fd33c018000 comp rx=0 tx=0).stop 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- 172.21.15.87:0/2213905186 shutdown_connections 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9d4600 0x55da4f9efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9ce470 0x55da4f9d1e70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 --2- 172.21.15.87:0/2213905186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- 172.21.15.87:0/2213905186 >> 172.21.15.87:0/2213905186 conn(0x55da4f8f8e60 msgr2=0x55da4f9d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- 172.21.15.87:0/2213905186 shutdown_connections 2024-06-06T07:52:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- 172.21.15.87:0/2213905186 wait complete. 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 -- start start 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.809+0000 7fd359547b80 1 Processor -- start 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 0x55da4f9e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9d4600 0x55da4f9dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481aeda0 con 0x55da4f9d4600 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b0210 con 0x55da4f9d3ce0 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481af470 con 0x55da4f9ce470 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd359547b80 10 client.0 _ll_register_callbacks cb 0x55da4f8b9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41304/0 (socket says 172.21.15.87:41304) 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 -- 172.21.15.87:0/2239192235 learned_addr learned my addr 172.21.15.87:0/2239192235 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:51.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 0x55da4f9e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 -- 172.21.15.87:0/2239192235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 msgr2=0x55da4f9e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 0x55da4f9e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 -- 172.21.15.87:0/2239192235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9d4600 msgr2=0x55da4f9dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9d4600 0x55da4f9dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 -- 172.21.15.87:0/2239192235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4f9dd720 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 0x55da4f9e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd347fff700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd34819c260 tx=0x7fd334003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aefe3c50ae6033a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.810+0000 7fd327fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334022070 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd334004a30 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c029070 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd334004c30 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 10 client.25718 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55da4fa53190 con 0x55da4f9d3ce0 2024-06-06T07:52:51.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c029070 con 0x55da4f9d3ce0 2024-06-06T07:52:51.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd33401fa80 con 0x55da4f9d3ce0 2024-06-06T07:52:51.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:51.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 10 client.25718 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:52:51.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 20 client.25718 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4fa53940 0x55da4fa55d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 4 client.25718 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.811+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x55da4f927430 con 0x55da4fa53940 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.812+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4fa53940 0x55da4fa55d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.812+0000 7fd327fff700 10 client.25718 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:51.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.812+0000 7fd34ca60700 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4fa53940 0x55da4fa55d60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df49487f0b64058 server_cookie=5472e932368d9d52 in_seq=0 out_seq=0 2024-06-06T07:52:51.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd327fff700 1 -- 172.21.15.87:0/2239192235 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55da4f927430 con 0x55da4fa53940 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd327fff700 10 client.25718 handle_command_reply: tid=0 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 client.25718 shutdown 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 msgr2=0x55da4f9e06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd34819c260 tx=0x7fd334003a10 comp rx=0 tx=0).stop 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 shutdown_connections 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55da4fa53940 0x55da4fa55d60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da4f9d4600 0x55da4f9dd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da4f9ce470 0x55da4f9e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 --2- 172.21.15.87:0/2239192235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55da4f9d3ce0 0x55da4f9e06a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd327fff700 0 client.25718 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.897+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 >> 172.21.15.87:0/2239192235 conn(0x55da4f8f8e60 msgr2=0x55da4f9d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 shutdown_connections 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd359547b80 1 -- 172.21.15.87:0/2239192235 wait complete. 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd359547b80 20 client.25718 trim_cache size 0 max 16384 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 14, 2024-06-06T07:52:51.899 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "30414161": { 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 14, 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:52:51.900 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:52:51.901 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 msgr2=0x7fd3380a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 0x7fd3380a5370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b3c0 tx=0x7fd33c00d040 comp rx=0 tx=0).stop 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 msgr2=0x7fd34819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd340006070 tx=0x7fd340007a20 comp rx=0 tx=0).stop 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 shutdown_connections 2024-06-06T07:52:51.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3380a2eb0 0x7fd3380a5370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3481066c0 0x7fd34819ab60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3481015f0 0x7fd3481a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 --2- 172.21.15.87:0/2843796359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd348101fa0 0x7fd3481a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 >> 172.21.15.87:0/2843796359 conn(0x7fd348078330 msgr2=0x7fd3480ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 shutdown_connections 2024-06-06T07:52:51.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:52:51.898+0000 7fd34ecc4700 1 -- 172.21.15.87:0/2843796359 wait complete. 2024-06-06T07:52:52.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:51 smithi181 ceph-mon[36790]: pgmap v148: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:52.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:51 smithi087 ceph-mon[32276]: pgmap v148: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:51 smithi195 ceph-mon[37040]: pgmap v148: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.9 MiB/s wr, 128 op/s 2024-06-06T07:52:52.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 6.52 MB/sec execute 56 sec latency 829.698 ms 2024-06-06T07:52:52.710 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '30414161', quiesce took 6.3 seconds. Will release after: 18.95 2024-06-06T07:52:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/3963706706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c381079a0 msgr2=0x7f4c38107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/3963706706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c381079a0 0x7f4c38107d80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009930 tx=0x7f4c2c02a420 comp rx=0 tx=0).stop 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/3963706706 shutdown_connections 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/3963706706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c38102350 0x7f4c3810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/3963706706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c38101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/3963706706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c381079a0 0x7f4c38107d80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/3963706706 >> 172.21.15.87:0/3963706706 conn(0x7f4c380fd460 msgr2=0x7f4c380ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/3963706706 shutdown_connections 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/3963706706 wait complete. 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 Processor -- start 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.332+0000 7f4c3ef5c640 1 -- start start 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c381983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 0x7f4c3819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38111e30 con 0x7f4c38102350 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38111cb0 con 0x7f4c38101990 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38111fb0 con 0x7f4c381079a0 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51110/0 (socket says 172.21.15.87:51110) 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 -- 172.21.15.87:0/2876906555 learned_addr learned my addr 172.21.15.87:0/2876906555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c381983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 0x7f4c3819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 -- 172.21.15.87:0/2876906555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 msgr2=0x7f4c3819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 0x7f4c3819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 -- 172.21.15.87:0/2876906555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 msgr2=0x7f4c381983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c381983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 -- 172.21.15.87:0/2876906555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3819d350 con 0x7f4c38102350 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 0x7f4c3819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c381983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c37fff640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b4a0 tx=0x7f4c2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c35ffb640 1 -- 172.21.15.87:0/2876906555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24014070 con 0x7f4c38102350 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c35ffb640 1 -- 172.21.15.87:0/2876906555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c24010af0 con 0x7f4c38102350 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3819d640 con 0x7f4c38102350 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.333+0000 7f4c35ffb640 1 -- 172.21.15.87:0/2876906555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24017d10 con 0x7f4c38102350 2024-06-06T07:52:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.334+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a4ed0 con 0x7f4c38102350 2024-06-06T07:52:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c35ffb640 1 -- 172.21.15.87:0/2876906555 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4c24017070 con 0x7f4c38102350 2024-06-06T07:52:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 Processor -- start 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 -- start start 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c35ffb640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 0x7f4c280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c290b0 0x5610e0c29490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c3ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5610e0b8d4f0 con 0x5610e0c299d0 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5610e0b2b870 con 0x5610e0c38760 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.336+0000 7f4c41c60740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5610e0b894b0 con 0x5610e0c290b0 2024-06-06T07:52:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 0x7f4c280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c35ffb640 1 -- 172.21.15.87:0/2876906555 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4c240ceb90 con 0x7f4c38102350 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51120/0 (socket says 172.21.15.87:51120) 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 -- 172.21.15.87:0/1402798725 learned_addr learned my addr 172.21.15.87:0/1402798725 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c290b0 0x5610e0c29490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c3ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 -- 172.21.15.87:0/1402798725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c290b0 msgr2=0x5610e0c29490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c290b0 0x5610e0c29490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 -- 172.21.15.87:0/1402798725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 msgr2=0x5610e0c3ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c3ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 -- 172.21.15.87:0/1402798725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e0b479b0 con 0x5610e0c299d0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 0x7f4c280a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c30013700 tx=0x7f4c30013b00 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c37fff640 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4c240cc020 tx=0x7f4c2409a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add56576729db741 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c22ffd640 1 -- 172.21.15.87:0/1402798725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2401e030 con 0x5610e0c299d0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c22ffd640 1 -- 172.21.15.87:0/1402798725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2409b490 con 0x5610e0c299d0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.337+0000 7f4c22ffd640 1 -- 172.21.15.87:0/1402798725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c240a3a00 con 0x5610e0c299d0 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- 172.21.15.87:0/1402798725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 msgr2=0x5610e0c38220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4c240cc020 tx=0x7f4c2409a7b0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- 172.21.15.87:0/1402798725 shutdown_connections 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c3ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c38220 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 --2- 172.21.15.87:0/1402798725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c290b0 0x5610e0c29490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- 172.21.15.87:0/1402798725 >> 172.21.15.87:0/1402798725 conn(0x5610e0c25680 msgr2=0x5610e0c25a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- 172.21.15.87:0/1402798725 shutdown_connections 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- 172.21.15.87:0/1402798725 wait complete. 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 -- start start 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.338+0000 7f4c41c60740 1 Processor -- start 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c35720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0c361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5610e0c3b330 con 0x5610e0c299d0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5610e0b16ae0 con 0x5610e0c290b0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5610e0c3b4b0 con 0x5610e0c38760 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 10 client.0 _ll_register_callbacks cb 0x5610e0b4f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51126/0 (socket says 172.21.15.87:51126) 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 -- 172.21.15.87:0/4263121920 learned_addr learned my addr 172.21.15.87:0/4263121920 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0c361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c35720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 -- 172.21.15.87:0/4263121920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 msgr2=0x5610e0c361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0c361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 -- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 msgr2=0x5610e0c35720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c35720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 -- 172.21.15.87:0/4263121920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e08a1210 con 0x5610e0c299d0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c3d4f9640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0c361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c37fff640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4c24017fd0 tx=0x7f4c24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41096018990e6d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2401e050 con 0x5610e0c299d0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4c24010ca0 con 0x5610e0c299d0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.339+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2409b490 con 0x5610e0c299d0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c240a3a00 con 0x5610e0c299d0 2024-06-06T07:52:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 10 client.16473 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610e096d620 con 0x5610e0c299d0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2409b490 con 0x5610e0c299d0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4c240cc130 con 0x5610e0c299d0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 10 client.16473 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 20 client.16473 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0c487f0 0x5610e0c4abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 4 client.16473 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e0ba0f10 con 0x5610e0c487f0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.340+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0c487f0 0x5610e0c4abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.341+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0c487f0 0x5610e0c4abe0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb4fb0c86c8fd68 server_cookie=6e6dd0b81d8e36b4 in_seq=0 out_seq=0 2024-06-06T07:52:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.341+0000 7f4c17fff640 10 client.16473 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c17fff640 1 -- 172.21.15.87:0/4263121920 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5610e0ba0f10 con 0x5610e0c487f0 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c17fff640 10 client.16473 handle_command_reply: tid=0 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 client.16473 shutdown 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 msgr2=0x5610e0c35c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4c24017fd0 tx=0x7f4c24000bc0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 shutdown_connections 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0c487f0 0x5610e0c4abe0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0c361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c35c60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 --2- 172.21.15.87:0/4263121920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c35720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c17fff640 0 client.16473 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 >> 172.21.15.87:0/4263121920 conn(0x5610e0c25680 msgr2=0x5610e0c2b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 shutdown_connections 2024-06-06T07:52:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 1 -- 172.21.15.87:0/4263121920 wait complete. 2024-06-06T07:52:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.342+0000 7f4c41c60740 20 client.16473 trim_cache size 0 max 16384 2024-06-06T07:52:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.356+0000 7f4c41c60740 1 Processor -- start 2024-06-06T07:52:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 -- start start 2024-06-06T07:52:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c299d0 0x5610e0c34530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381af790 con 0x5610e0c290b0 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ad060 con 0x5610e0c38760 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c41c60740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381aaeb0 con 0x5610e0c299d0 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51140/0 (socket says 172.21.15.87:51140) 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 -- 172.21.15.87:0/3450254879 learned_addr learned my addr 172.21.15.87:0/3450254879 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c37fff640 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c299d0 0x5610e0c34530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 -- 172.21.15.87:0/3450254879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c299d0 msgr2=0x5610e0c34530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c299d0 0x5610e0c34530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 -- 172.21.15.87:0/3450254879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 msgr2=0x5610e0c50180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 -- 172.21.15.87:0/3450254879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e0c4af50 con 0x5610e0c290b0 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0096c0 tx=0x7f4c2c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b750f35d6a1729d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c227fc640 1 -- 172.21.15.87:0/3450254879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c048070 con 0x5610e0c290b0 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c227fc640 1 -- 172.21.15.87:0/3450254879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c005500 con 0x5610e0c290b0 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.357+0000 7f4c227fc640 1 -- 172.21.15.87:0/3450254879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c035040 con 0x5610e0c290b0 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- 172.21.15.87:0/3450254879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 msgr2=0x5610e0c33ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0096c0 tx=0x7f4c2c0058b0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- 172.21.15.87:0/3450254879 shutdown_connections 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c38760 0x5610e0c50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c299d0 0x5610e0c34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3450254879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c290b0 0x5610e0c33ff0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- 172.21.15.87:0/3450254879 >> 172.21.15.87:0/3450254879 conn(0x5610e0c25680 msgr2=0x5610e0c2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- 172.21.15.87:0/3450254879 shutdown_connections 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- 172.21.15.87:0/3450254879 wait complete. 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 -- start start 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.358+0000 7f4c41c60740 1 Processor -- start 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0cb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ab520 con 0x5610e0c299d0 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ac990 con 0x5610e0c290b0 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381abbf0 con 0x5610e0c38760 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 10 client.0 _ll_register_callbacks cb 0x7f4c2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51152/0 (socket says 172.21.15.87:51152) 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 -- 172.21.15.87:0/3032451422 learned_addr learned my addr 172.21.15.87:0/3032451422 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 -- 172.21.15.87:0/3032451422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 msgr2=0x5610e0cb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0cb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 -- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 msgr2=0x5610e0c3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c3bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 -- 172.21.15.87:0/3032451422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e0cb16b0 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c37fff640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b470 tx=0x7f4c2409b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524428a7a882d1e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2401e070 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f4c24010ca0 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c005500 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c2409ad90 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 10 client.16479 fetch_fsmap learned FSMap version 32 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.359+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610e0cb16b0 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c005500 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4c24072bb0 con 0x5610e0c299d0 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 10 client.16479 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:52:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 20 client.16479 populate_metadata read hostname 'smithi087' 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0cb2c00 0x5610e0cb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 4 client.16479 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610e0b9ac20 con 0x5610e0cb2c00 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.360+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0cb2c00 0x5610e0cb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.361+0000 7f4c17fff640 10 client.16479 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.361+0000 7f4c3ccf8640 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0cb2c00 0x5610e0cb5010 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1405e8714fa79b80 server_cookie=1c158ec12648a6e5 in_seq=0 out_seq=0 2024-06-06T07:52:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c17fff640 1 -- 172.21.15.87:0/3032451422 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5610e0b9ac20 con 0x5610e0cb2c00 2024-06-06T07:52:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c17fff640 10 client.16479 handle_command_reply: tid=0 2024-06-06T07:52:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 client.16479 shutdown 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 msgr2=0x5610e0c3c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b470 tx=0x7f4c2409b490 comp rx=0 tx=0).stop 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 shutdown_connections 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5610e0cb2c00 0x5610e0cb5010 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610e0c38760 0x5610e0cb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c17fff640 0 client.16479 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5610e0c299d0 0x5610e0c3c270 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 --2- 172.21.15.87:0/3032451422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5610e0c290b0 0x5610e0c3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 >> 172.21.15.87:0/3032451422 conn(0x5610e0c25680 msgr2=0x5610e0c33fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 shutdown_connections 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.483+0000 7f4c41c60740 1 -- 172.21.15.87:0/3032451422 wait complete. 2024-06-06T07:52:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c41c60740 20 client.16479 trim_cache size 0 max 16384 2024-06-06T07:52:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:52:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:52:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 msgr2=0x7f4c280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 0x7f4c280a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c30013700 tx=0x7f4c30013b00 comp rx=0 tx=0).stop 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 msgr2=0x7f4c381988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b4a0 tx=0x7f4c2400b960 comp rx=0 tx=0).stop 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 shutdown_connections 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4c280a2da0 0x7f4c280a5260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c381079a0 0x7f4c3819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c38102350 0x7f4c381988f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 --2- 172.21.15.87:0/2876906555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c38101990 0x7f4c381983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.484+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 >> 172.21.15.87:0/2876906555 conn(0x7f4c380fd460 msgr2=0x7f4c380fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.485+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 shutdown_connections 2024-06-06T07:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:52:53.485+0000 7f4c3ef5c640 1 -- 172.21.15.87:0/2876906555 wait complete. 2024-06-06T07:52:53.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 6.41 MB/sec execute 57 sec latency 1829.789 ms 2024-06-06T07:52:54.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:53 smithi087 ceph-mon[32276]: pgmap v149: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-06-06T07:52:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:53 smithi195 ceph-mon[37040]: pgmap v149: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-06-06T07:52:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:53 smithi181 ceph-mon[36790]: pgmap v149: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-06-06T07:52:54.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 6.30 MB/sec execute 58 sec latency 2829.886 ms 2024-06-06T07:52:55.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 6.19 MB/sec execute 59 sec latency 3829.981 ms 2024-06-06T07:52:56.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:55 smithi087 ceph-mon[32276]: pgmap v150: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-06-06T07:52:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:55 smithi195 ceph-mon[37040]: pgmap v150: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-06-06T07:52:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:55 smithi181 ceph-mon[36790]: pgmap v150: 129 pgs: 129 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-06-06T07:52:56.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 6.09 MB/sec execute 60 sec latency 4830.080 ms 2024-06-06T07:52:57.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.99 MB/sec execute 61 sec latency 5830.134 ms 2024-06-06T07:52:58.108 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:57 smithi181 ceph-mon[36790]: pgmap v151: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-06-06T07:52:58.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:57 smithi087 ceph-mon[32276]: pgmap v151: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-06-06T07:52:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:57 smithi195 ceph-mon[37040]: pgmap v151: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-06-06T07:52:58.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.89 MB/sec execute 62 sec latency 6830.195 ms 2024-06-06T07:52:59.661 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.80 MB/sec execute 63 sec latency 7830.231 ms 2024-06-06T07:53:00.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:52:59 smithi087 ceph-mon[32276]: pgmap v152: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:52:59 smithi195 ceph-mon[37040]: pgmap v152: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:52:59 smithi181 ceph-mon[36790]: pgmap v152: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:00.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.71 MB/sec execute 64 sec latency 8830.261 ms 2024-06-06T07:53:01.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.62 MB/sec execute 65 sec latency 9830.296 ms 2024-06-06T07:53:02.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:01 smithi087 ceph-mon[32276]: pgmap v153: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:02.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:01 smithi195 ceph-mon[37040]: pgmap v153: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:01 smithi181 ceph-mon[36790]: pgmap v153: 129 pgs: 129 active+clean; 40 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:53:02.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.53 MB/sec execute 66 sec latency 10830.378 ms 2024-06-06T07:53:03.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.45 MB/sec execute 67 sec latency 11830.477 ms 2024-06-06T07:53:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:03 smithi087 ceph-mon[32276]: pgmap v154: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-06-06T07:53:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:03 smithi195 ceph-mon[37040]: pgmap v154: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-06-06T07:53:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:03 smithi181 ceph-mon[36790]: pgmap v154: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-06-06T07:53:04.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.37 MB/sec execute 68 sec latency 12830.582 ms 2024-06-06T07:53:05.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.29 MB/sec execute 69 sec latency 13830.681 ms 2024-06-06T07:53:06.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:05 smithi087 ceph-mon[32276]: pgmap v155: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 337 KiB/s wr, 109 op/s 2024-06-06T07:53:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:05 smithi195 ceph-mon[37040]: pgmap v155: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 337 KiB/s wr, 109 op/s 2024-06-06T07:53:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:05 smithi181 ceph-mon[36790]: pgmap v155: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 337 KiB/s wr, 109 op/s 2024-06-06T07:53:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 -- 172.21.15.87:0/154117205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 msgr2=0x7fd36c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 --2- 172.21.15.87:0/154117205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c10de20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd360009900 tx=0x7fd36002a410 comp rx=0 tx=0).stop 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 -- 172.21.15.87:0/154117205 shutdown_connections 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 --2- 172.21.15.87:0/154117205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 --2- 172.21.15.87:0/154117205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c10de20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 --2- 172.21.15.87:0/154117205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 0x7fd36c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 -- 172.21.15.87:0/154117205 >> 172.21.15.87:0/154117205 conn(0x7fd36c077a20 msgr2=0x7fd36c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 -- 172.21.15.87:0/154117205 shutdown_connections 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 -- 172.21.15.87:0/154117205 wait complete. 2024-06-06T07:53:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.581+0000 7fd371ac4640 1 Processor -- start 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 -- start start 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 0x7fd36c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113f50 con 0x7fd36c1010d0 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113dd0 con 0x7fd36c10e360 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd371ac4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1140d0 con 0x7fd36c1007b0 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38388/0 (socket says 172.21.15.87:38388) 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 -- 172.21.15.87:0/1254597888 learned_addr learned my addr 172.21.15.87:0/1254597888 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 0x7fd36c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36bfff640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 -- 172.21.15.87:0/1254597888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 msgr2=0x7fd36c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 0x7fd36c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 -- 172.21.15.87:0/1254597888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 msgr2=0x7fd36c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36affd640 1 -- 172.21.15.87:0/1254597888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c19f580 con 0x7fd36c1010d0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.582+0000 7fd36bfff640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd36affd640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd360006010 tx=0x7fd3600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd368ff9640 1 -- 172.21.15.87:0/1254597888 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36003f030 con 0x7fd36c1010d0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd368ff9640 1 -- 172.21.15.87:0/1254597888 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd360003ce0 con 0x7fd36c1010d0 2024-06-06T07:53:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c19f810 con 0x7fd36c1010d0 2024-06-06T07:53:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd368ff9640 1 -- 172.21.15.87:0/1254597888 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360035070 con 0x7fd36c1010d0 2024-06-06T07:53:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.583+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a7100 con 0x7fd36c1010d0 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.585+0000 7fd368ff9640 1 -- 172.21.15.87:0/1254597888 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd360042030 con 0x7fd36c1010d0 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 Processor -- start 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 -- start start 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e395170 0x55a39e395550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e3a0350 0x55a39e3a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a39e2f94f0 con 0x55a39e395a90 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a39e297870 con 0x55a39e3a0350 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd3747d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a39e2f54b0 con 0x55a39e395170 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38400/0 (socket says 172.21.15.87:38400) 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 -- 172.21.15.87:0/532608685 learned_addr learned my addr 172.21.15.87:0/532608685 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd368ff9640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 0x7fd3540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 -- 172.21.15.87:0/532608685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e395170 msgr2=0x55a39e395550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e395170 0x55a39e395550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 -- 172.21.15.87:0/532608685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e3a0350 msgr2=0x55a39e3a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e3a0350 0x55a39e3a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36b7fe640 1 -- 172.21.15.87:0/532608685 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39e2b39b0 con 0x55a39e395a90 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.586+0000 7fd36bfff640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 0x7fd3540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd368ff9640 1 -- 172.21.15.87:0/1254597888 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd3600efce0 con 0x7fd36c1010d0 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0039f0 tx=0x7fd35c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51c1360b80b0401 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd351ffb640 1 -- 172.21.15.87:0/532608685 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004b20 con 0x55a39e395a90 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd36bfff640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 0x7fd3540a52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd364005d80 tx=0x7fd364005e50 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd351ffb640 1 -- 172.21.15.87:0/532608685 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c004cc0 con 0x55a39e395a90 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.587+0000 7fd351ffb640 1 -- 172.21.15.87:0/532608685 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c012780 con 0x55a39e395a90 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- 172.21.15.87:0/532608685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 msgr2=0x55a39e39fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0039f0 tx=0x7fd35c003eb0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- 172.21.15.87:0/532608685 shutdown_connections 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e3a0350 0x55a39e3a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e39fe10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 --2- 172.21.15.87:0/532608685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e395170 0x55a39e395550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- 172.21.15.87:0/532608685 >> 172.21.15.87:0/532608685 conn(0x55a39e391680 msgr2=0x55a39e391a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- 172.21.15.87:0/532608685 shutdown_connections 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- 172.21.15.87:0/532608685 wait complete. 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 -- start start 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.588+0000 7fd3747d8740 1 Processor -- start 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 0x55a39e39d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e39de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a39e3a2f80 con 0x55a39e395170 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a39e282ae0 con 0x55a39e395a90 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a39e3a3100 con 0x55a39e3a0350 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 10 client.0 _ll_register_callbacks cb 0x55a39e2bb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38406/0 (socket says 172.21.15.87:38406) 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 -- 172.21.15.87:0/2743824058 learned_addr learned my addr 172.21.15.87:0/2743824058 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36bfff640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e39de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36affd640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 0x55a39e39d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 -- 172.21.15.87:0/2743824058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 msgr2=0x55a39e39de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e39de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 -- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 msgr2=0x55a39e39d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 0x55a39e39d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 -- 172.21.15.87:0/2743824058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39e00d210 con 0x55a39e395170 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36bfff640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e39de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36affd640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 0x55a39e39d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009e30 tx=0x7fd35c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d5ba957c4638e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd34affd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c005340 con 0x55a39e395170 2024-06-06T07:53:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd35c0054e0 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c004cc0 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd35c0056e0 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 10 client.16491 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.589+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a39e0d9620 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c004cc0 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd35c00f040 con 0x55a39e395170 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 10 client.16491 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 20 client.16491 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e3b0500 0x55a39e3b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 4 client.16491 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a39e30cf10 con 0x55a39e3b0500 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.590+0000 7fd36affd640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e3b0500 0x55a39e3b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.591+0000 7fd34affd640 10 client.16491 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.591+0000 7fd36affd640 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e3b0500 0x55a39e3b28f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9d926533796084 server_cookie=2c16d240ec30ba50 in_seq=0 out_seq=0 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.591+0000 7fd34affd640 1 -- 172.21.15.87:0/2743824058 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a39e30cf10 con 0x55a39e3b0500 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.591+0000 7fd34affd640 10 client.16491 handle_command_reply: tid=0 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 client.16491 shutdown 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 msgr2=0x55a39e39d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009e30 tx=0x7fd35c005bd0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 shutdown_connections 2024-06-06T07:53:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e3b0500 0x55a39e3b28f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e39de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395a90 0x55a39e39d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd34affd640 0 client.16491 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2743824058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395170 0x55a39e39d3c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 >> 172.21.15.87:0/2743824058 conn(0x55a39e391680 msgr2=0x55a39e3973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 shutdown_connections 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 1 -- 172.21.15.87:0/2743824058 wait complete. 2024-06-06T07:53:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.592+0000 7fd3747d8740 20 client.16491 trim_cache size 0 max 16384 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 Processor -- start 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 -- start start 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3b8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e38ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b3a40 con 0x55a39e395a90 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b1310 con 0x55a39e395170 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd3747d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1af160 con 0x55a39e3a0350 2024-06-06T07:53:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38408/0 (socket says 172.21.15.87:38408) 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.607+0000 7fd36affd640 1 -- 172.21.15.87:0/2319099797 learned_addr learned my addr 172.21.15.87:0/2319099797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36bfff640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e38ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 -- 172.21.15.87:0/2319099797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 msgr2=0x55a39e38ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e38ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 -- 172.21.15.87:0/2319099797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 msgr2=0x55a39e3b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 -- 172.21.15.87:0/2319099797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39e3ac1e0 con 0x55a39e395a90 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36bfff640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e38ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd36affd640 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ed0c0 tx=0x7fd36002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca1f88649012700 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3517fa640 1 -- 172.21.15.87:0/2319099797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360049040 con 0x55a39e395a90 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3517fa640 1 -- 172.21.15.87:0/2319099797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3600bc5f0 con 0x55a39e395a90 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3517fa640 1 -- 172.21.15.87:0/2319099797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36003bcd0 con 0x55a39e395a90 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 -- 172.21.15.87:0/2319099797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 msgr2=0x55a39e3b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ed0c0 tx=0x7fd36002a410 comp rx=0 tx=0).stop 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 -- 172.21.15.87:0/2319099797 shutdown_connections 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e38ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3b85a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 --2- 172.21.15.87:0/2319099797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.608+0000 7fd3747d8740 1 -- 172.21.15.87:0/2319099797 >> 172.21.15.87:0/2319099797 conn(0x55a39e391680 msgr2=0x55a39e398670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- 172.21.15.87:0/2319099797 shutdown_connections 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- 172.21.15.87:0/2319099797 wait complete. 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- start start 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 Processor -- start 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e41b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1af7d0 con 0x55a39e395a90 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b0c40 con 0x55a39e395170 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1afea0 con 0x55a39e3a0350 2024-06-06T07:53:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd3747d8740 10 client.0 _ll_register_callbacks cb 0x7fd35c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd36affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38416/0 (socket says 172.21.15.87:38416) 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd36affd640 1 -- 172.21.15.87:0/1724730175 learned_addr learned my addr 172.21.15.87:0/1724730175 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.609+0000 7fd36bfff640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e41b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 -- 172.21.15.87:0/1724730175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 msgr2=0x55a39e41b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e41b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 -- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 msgr2=0x55a39e3a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 -- 172.21.15.87:0/1724730175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39e41be70 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd36affd640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ed660 tx=0x7fd3600c0320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d385cd6c9cd3ac6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360049040 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fd3600ed2e0 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3600bc5f0 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3600b5050 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd3747d8740 10 client.16497 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a39e4231d0 con 0x55a39e395a90 2024-06-06T07:53:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3600bc5f0 con 0x55a39e395a90 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.610+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd3600bc1e0 con 0x55a39e395a90 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 10 client.16497 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 20 client.16497 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e4238f0 0x55a39e425d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 4 client.16497 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a39e306c20 con 0x55a39e4238f0 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e4238f0 0x55a39e425d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd36b7fe640 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e4238f0 0x55a39e425d10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9523419226fedc0 server_cookie=631af3caf08cc5ef in_seq=0 out_seq=0 2024-06-06T07:53:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.611+0000 7fd34affd640 10 client.16497 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd34affd640 1 -- 172.21.15.87:0/1724730175 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a39e306c20 con 0x55a39e4238f0 2024-06-06T07:53:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd34affd640 10 client.16497 handle_command_reply: tid=0 2024-06-06T07:53:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 client.16497 shutdown 2024-06-06T07:53:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 msgr2=0x55a39e3a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ed660 tx=0x7fd3600c0320 comp rx=0 tx=0).stop 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 shutdown_connections 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a39e4238f0 0x55a39e425d10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a39e3a0350 0x55a39e41b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a39e395a90 0x55a39e3a4060 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 --2- 172.21.15.87:0/1724730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a39e395170 0x55a39e3a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd34affd640 0 client.16497 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 >> 172.21.15.87:0/1724730175 conn(0x55a39e391680 msgr2=0x55a39e3ab1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 shutdown_connections 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 1 -- 172.21.15.87:0/1724730175 wait complete. 2024-06-06T07:53:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.636+0000 7fd3747d8740 20 client.16497 trim_cache size 0 max 16384 2024-06-06T07:53:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:53:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:53:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 msgr2=0x7fd3540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 0x7fd3540a52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd364005d80 tx=0x7fd364005e50 comp rx=0 tx=0).stop 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 msgr2=0x7fd36c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd360006010 tx=0x7fd3600042f0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 shutdown_connections 2024-06-06T07:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3540a2e20 0x7fd3540a52e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd36c10e360 0x7fd36c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd36c1010d0 0x7fd36c19aad0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 --2- 172.21.15.87:0/1254597888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd36c1007b0 0x7fd36c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 >> 172.21.15.87:0/1254597888 conn(0x7fd36c077a20 msgr2=0x7fd36c10f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 shutdown_connections 2024-06-06T07:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:06.637+0000 7fd371ac4640 1 -- 172.21.15.87:0/1254597888 wait complete. 2024-06-06T07:53:06.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.22 MB/sec execute 70 sec latency 14830.787 ms 2024-06-06T07:53:07.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.14 MB/sec execute 71 sec latency 15830.824 ms 2024-06-06T07:53:08.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:07 smithi087 ceph-mon[32276]: pgmap v156: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 340 KiB/s wr, 112 op/s 2024-06-06T07:53:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:07 smithi195 ceph-mon[37040]: pgmap v156: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 340 KiB/s wr, 112 op/s 2024-06-06T07:53:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:07 smithi181 ceph-mon[36790]: pgmap v156: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 340 KiB/s wr, 112 op/s 2024-06-06T07:53:08.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.07 MB/sec execute 72 sec latency 16830.876 ms 2024-06-06T07:53:09.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 5.00 MB/sec execute 73 sec latency 17830.931 ms 2024-06-06T07:53:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:09 smithi087 ceph-mon[32276]: pgmap v157: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:09 smithi195 ceph-mon[37040]: pgmap v157: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:09 smithi181 ceph-mon[36790]: pgmap v157: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:10.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.94 MB/sec execute 74 sec latency 18831.017 ms 2024-06-06T07:53:11.662 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:53:11.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.87 MB/sec execute 75 sec latency 19831.144 ms 2024-06-06T07:53:11.913 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:53:12.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:11 smithi087 ceph-mon[32276]: pgmap v158: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:11 smithi195 ceph-mon[37040]: pgmap v158: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:11 smithi181 ceph-mon[36790]: pgmap v158: 129 pgs: 129 active+clean; 40 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 22 op/s 2024-06-06T07:53:12.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.81 MB/sec execute 76 sec latency 20831.188 ms 2024-06-06T07:53:13.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.74 MB/sec execute 77 sec latency 21831.213 ms 2024-06-06T07:53:13.926 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:13 smithi087 ceph-mon[32276]: pgmap v159: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-06-06T07:53:13.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 -- 172.21.15.87:0/87271433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c100e50 msgr2=0x7eff6c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:13.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 --2- 172.21.15.87:0/87271433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c100e50 0x7eff6c1012d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7eff54009990 tx=0x7eff54009ca0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 -- 172.21.15.87:0/87271433 shutdown_connections 2024-06-06T07:53:13.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 --2- 172.21.15.87:0/87271433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c107570 0x7eff6c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 --2- 172.21.15.87:0/87271433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c101810 0x7eff6c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 --2- 172.21.15.87:0/87271433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c100e50 0x7eff6c1012d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 -- 172.21.15.87:0/87271433 >> 172.21.15.87:0/87271433 conn(0x7eff6c0766b0 msgr2=0x7eff6c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 -- 172.21.15.87:0/87271433 shutdown_connections 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.943+0000 7eff70d61700 1 -- 172.21.15.87:0/87271433 wait complete. 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 Processor -- start 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 -- start start 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 0x7eff6c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c107570 0x7eff6c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c112870 con 0x7eff6c101810 2024-06-06T07:53:13.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1126f0 con 0x7eff6c100e50 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff70d61700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1129f0 con 0x7eff6c107570 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff61d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff61d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36468/0 (socket says 172.21.15.87:36468) 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff61d9b700 1 -- 172.21.15.87:0/3734853346 learned_addr learned my addr 172.21.15.87:0/3734853346 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff6ad9d700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c107570 0x7eff6c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.944+0000 7eff6a59c700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 0x7eff6c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 -- 172.21.15.87:0/3734853346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c107570 msgr2=0x7eff6c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c107570 0x7eff6c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 -- 172.21.15.87:0/3734853346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 msgr2=0x7eff6c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 0x7eff6c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 -- 172.21.15.87:0/3734853346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c199210 con 0x7eff6c101810 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff6a59c700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 0x7eff6c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff61d9b700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7eff54000c00 tx=0x7eff540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:13.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54027070 con 0x7eff6c101810 2024-06-06T07:53:13.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff54003780 con 0x7eff6c101810 2024-06-06T07:53:13.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54022dd0 con 0x7eff6c101810 2024-06-06T07:53:13.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1994a0 con 0x7eff6c101810 2024-06-06T07:53:13.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.945+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a57c0 con 0x7eff6c101810 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.946+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c069180 con 0x7eff6c101810 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.947+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7eff5402a070 con 0x7eff6c101810 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.948+0000 7eff63fff700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 0x7eff580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.948+0000 7eff6a59c700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 0x7eff580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.948+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7eff540d08a0 con 0x7eff6c101810 2024-06-06T07:53:13.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.948+0000 7eff6a59c700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 0x7eff580a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff6c199bf0 tx=0x7eff5c008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:13.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:13.951+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff54013070 con 0x7eff6c101810 2024-06-06T07:53:14.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.191+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff6c1a5990 con 0x7eff6c101810 2024-06-06T07:53:14.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.191+0000 7eff63fff700 1 -- 172.21.15.87:0/3734853346 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7eff5409d210 con 0x7eff6c101810 2024-06-06T07:53:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:53:14.192 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 msgr2=0x7eff580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 0x7eff580a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff6c199bf0 tx=0x7eff5c008040 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 msgr2=0x7eff6c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7eff54000c00 tx=0x7eff540047b0 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 shutdown_connections 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff6c100e50 0x7eff6c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7eff580a2d90 0x7eff580a5250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff6c107570 0x7eff6c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 --2- 172.21.15.87:0/3734853346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff6c101810 0x7eff6c19ee30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 >> 172.21.15.87:0/3734853346 conn(0x7eff6c0766b0 msgr2=0x7eff6c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:14.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 shutdown_connections 2024-06-06T07:53:14.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:14.194+0000 7eff70d61700 1 -- 172.21.15.87:0/3734853346 wait complete. 2024-06-06T07:53:14.195 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:53:14.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:13 smithi195 ceph-mon[37040]: pgmap v159: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-06-06T07:53:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:13 smithi181 ceph-mon[36790]: pgmap v159: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-06-06T07:53:14.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.68 MB/sec execute 78 sec latency 22831.239 ms 2024-06-06T07:53:14.998 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 30414161 --release --await' 2024-06-06T07:53:14.998 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 30414161 --release --await 2024-06-06T07:53:15.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:14 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3734853346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:53:15.245 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:53:15.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:14 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3734853346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:53:15.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:14 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3734853346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:53:15.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.62 MB/sec execute 79 sec latency 23831.265 ms 2024-06-06T07:53:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:15 smithi087 ceph-mon[32276]: pgmap v160: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-06-06T07:53:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:15 smithi195 ceph-mon[37040]: pgmap v160: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-06-06T07:53:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:15 smithi181 ceph-mon[36790]: pgmap v160: 129 pgs: 129 active+clean; 38 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-06-06T07:53:16.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 192289 4.57 MB/sec execute 80 sec latency 24831.294 ms 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- 172.21.15.87:0/1058436180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53281073f0 msgr2=0x7f53281077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- 172.21.15.87:0/1058436180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53281073f0 0x7f53281077d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f531c009990 tx=0x7f531c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- 172.21.15.87:0/1058436180 shutdown_connections 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- 172.21.15.87:0/1058436180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5328101d40 0x7f532810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- 172.21.15.87:0/1058436180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53281073f0 0x7f53281077d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- 172.21.15.87:0/1058436180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101380 0x7f5328101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- 172.21.15.87:0/1058436180 >> 172.21.15.87:0/1058436180 conn(0x7f5328078300 msgr2=0x7f5328078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- 172.21.15.87:0/1058436180 shutdown_connections 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- 172.21.15.87:0/1058436180 wait complete. 2024-06-06T07:53:17.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 Processor -- start 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 -- start start 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101d40 0x7f5328197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.106+0000 7f532ee76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53281073f0 0x7f5328198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532ee76700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53281116e0 con 0x7f5328101380 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532ee76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5328111560 con 0x7f53281073f0 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532ee76700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5328111860 con 0x7f5328101d40 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45208/0 (socket says 172.21.15.87:45208) 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 -- 172.21.15.87:0/3221486693 learned_addr learned my addr 172.21.15.87:0/3221486693 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f5327fff700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101d40 0x7f5328197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532d413700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53281073f0 0x7f5328198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 -- 172.21.15.87:0/3221486693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101d40 msgr2=0x7f5328197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101d40 0x7f5328197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 -- 172.21.15.87:0/3221486693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53281073f0 msgr2=0x7f5328198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53281073f0 0x7f5328198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 -- 172.21.15.87:0/3221486693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53281a7100 con 0x7f5328101380 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532cc12700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f531c0038b0 tx=0x7f531c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f5325ffb700 1 -- 172.21.15.87:0/3221486693 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c027070 con 0x7f5328101380 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53281a7390 con 0x7f5328101380 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f5325ffb700 1 -- 172.21.15.87:0/3221486693 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c0040d0 con 0x7f5328101380 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.107+0000 7f5325ffb700 1 -- 172.21.15.87:0/3221486693 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c022dc0 con 0x7f5328101380 2024-06-06T07:53:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.108+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53281a78c0 con 0x7f5328101380 2024-06-06T07:53:17.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.110+0000 7f5325ffb700 1 -- 172.21.15.87:0/3221486693 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f531c02a070 con 0x7f5328101380 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.110+0000 7f5325ffb700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 0x7f53180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f5325ffb700 1 -- 172.21.15.87:0/3221486693 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f531c0d04a0 con 0x7f5328101380 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f5327fff700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 0x7f53180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f5327fff700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 0x7f53180a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5314005fd0 tx=0x7f5314005f60 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f53396f9b80 1 Processor -- start 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f53396f9b80 1 -- start start 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.111+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8ec40 0x5626f0d8f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f53396f9b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5626f0cda910 con 0x5626f0d893d0 2024-06-06T07:53:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f53396f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5626f0c82820 con 0x5626f0d8f560 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f53396f9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626f0cd68d0 con 0x5626f0d8ec40 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45224/0 (socket says 172.21.15.87:45224) 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 -- 172.21.15.87:0/1457037143 learned_addr learned my addr 172.21.15.87:0/1457037143 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f5327fff700 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532cc12700 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8ec40 0x5626f0d8f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 -- 172.21.15.87:0/1457037143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8ec40 msgr2=0x5626f0d8f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8ec40 0x5626f0d8f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 -- 172.21.15.87:0/1457037143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 msgr2=0x5626f0d88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d88e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 -- 172.21.15.87:0/1457037143 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626f0c79050 con 0x5626f0d893d0 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f532d413700 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5320009f20 tx=0x7f5320009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebd7b1cd8c5ce91 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f5312ffd700 1 -- 172.21.15.87:0/1457037143 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532001a070 con 0x5626f0d893d0 2024-06-06T07:53:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f5312ffd700 1 -- 172.21.15.87:0/1457037143 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5320015410 con 0x5626f0d893d0 2024-06-06T07:53:17.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.112+0000 7f5312ffd700 1 -- 172.21.15.87:0/1457037143 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000a660 con 0x5626f0d893d0 2024-06-06T07:53:17.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 -- 172.21.15.87:0/1457037143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 msgr2=0x5626f0d89810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5320009f20 tx=0x7f5320009f50 comp rx=0 tx=0).stop 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 -- 172.21.15.87:0/1457037143 shutdown_connections 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d89810 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8ec40 0x5626f0d8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 --2- 172.21.15.87:0/1457037143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 -- 172.21.15.87:0/1457037143 >> 172.21.15.87:0/1457037143 conn(0x5626f0cb3dc0 msgr2=0x5626f0d84120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 -- 172.21.15.87:0/1457037143 shutdown_connections 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.113+0000 7f53396f9b80 1 -- 172.21.15.87:0/1457037143 wait complete. 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 -- start start 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 Processor -- start 2024-06-06T07:53:17.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d893d0 0x5626f0d95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d92aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5626f0d95fd0 con 0x5626f0d8ec40 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5626f0c7acf0 con 0x5626f0d8f560 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626f0d96150 con 0x5626f0d893d0 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f53396f9b80 10 client.0 _ll_register_callbacks cb 0x5626f0c74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f5327fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f5327fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45230/0 (socket says 172.21.15.87:45230) 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.114+0000 7f5327fff700 1 -- 172.21.15.87:0/2481111084 learned_addr learned my addr 172.21.15.87:0/2481111084 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f532d413700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d92aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f532cc12700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d893d0 0x5626f0d95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 -- 172.21.15.87:0/2481111084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d893d0 msgr2=0x5626f0d95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d893d0 0x5626f0d95260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 -- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 msgr2=0x5626f0d92aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d92aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 -- 172.21.15.87:0/2481111084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626f0c299d0 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5327fff700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f531400ff30 tx=0x7f5314004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6781ede369bd27 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5307fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314028670 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5320015410 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f531401f430 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5314028810 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f53396f9b80 10 client.16515 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.115+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626f0da0830 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5320015410 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f531401fbd0 con 0x5626f0d8ec40 2024-06-06T07:53:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 10 client.16515 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 20 client.16515 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0da2f10 0x5626f0da5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 4 client.16515 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626f0ce7e10 con 0x5626f0da2f10 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f532cc12700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0da2f10 0x5626f0da5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f532cc12700 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0da2f10 0x5626f0da5300 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797d762bd1be7195 server_cookie=5013d1f94a2eb75c in_seq=0 out_seq=0 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.116+0000 7f5307fff700 10 client.16515 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f5307fff700 1 -- 172.21.15.87:0/2481111084 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5626f0ce7e10 con 0x5626f0da2f10 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f5307fff700 10 client.16515 handle_command_reply: tid=0 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 client.16515 shutdown 2024-06-06T07:53:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 msgr2=0x5626f0d92560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f531400ff30 tx=0x7f5314004750 comp rx=0 tx=0).stop 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 shutdown_connections 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0da2f10 0x5626f0da5300 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8f560 0x5626f0d92aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d893d0 0x5626f0d95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f5307fff700 0 client.16515 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 --2- 172.21.15.87:0/2481111084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d92560 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.117+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 >> 172.21.15.87:0/2481111084 conn(0x5626f0cb3dc0 msgr2=0x5626f0d83d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.118+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 shutdown_connections 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.118+0000 7f53396f9b80 1 -- 172.21.15.87:0/2481111084 wait complete. 2024-06-06T07:53:17.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.118+0000 7f53396f9b80 20 client.16515 trim_cache size 0 max 16384 2024-06-06T07:53:17.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.137+0000 7f53396f9b80 1 Processor -- start 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.137+0000 7f53396f9b80 1 -- start start 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8ec40 0x5626f0d916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0daab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53281b31e0 con 0x5626f0d893d0 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53281b0a70 con 0x5626f0d8ec40 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53396f9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53281ae8c0 con 0x5626f0d8f560 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45234/0 (socket says 172.21.15.87:45234) 2024-06-06T07:53:17.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 -- 172.21.15.87:0/3781014728 learned_addr learned my addr 172.21.15.87:0/3781014728 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f5327fff700 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8ec40 0x5626f0d916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532d413700 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0daab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 -- 172.21.15.87:0/3781014728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 msgr2=0x5626f0daab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0daab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 -- 172.21.15.87:0/3781014728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8ec40 msgr2=0x5626f0d916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8ec40 0x5626f0d916e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 -- 172.21.15.87:0/3781014728 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626f0d854b0 con 0x5626f0d893d0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f532cc12700 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f531c0098e0 tx=0x7f531c0a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b658e0b1b0aed1b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.138+0000 7f53127fc700 1 -- 172.21.15.87:0/3781014728 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c09ce20 con 0x5626f0d893d0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53127fc700 1 -- 172.21.15.87:0/3781014728 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c09c720 con 0x5626f0d893d0 2024-06-06T07:53:17.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53127fc700 1 -- 172.21.15.87:0/3781014728 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0052f0 con 0x5626f0d893d0 2024-06-06T07:53:17.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 -- 172.21.15.87:0/3781014728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 msgr2=0x5626f0d88e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f531c0098e0 tx=0x7f531c0a3040 comp rx=0 tx=0).stop 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 -- 172.21.15.87:0/3781014728 shutdown_connections 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0daab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d893d0 0x5626f0d88e90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3781014728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d8ec40 0x5626f0d916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 -- 172.21.15.87:0/3781014728 >> 172.21.15.87:0/3781014728 conn(0x5626f0cb3dc0 msgr2=0x5626f0d8ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 -- 172.21.15.87:0/3781014728 shutdown_connections 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.139+0000 7f53396f9b80 1 -- 172.21.15.87:0/3781014728 wait complete. 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 -- start start 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 Processor -- start 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d893d0 0x5626f0d983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0d98e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53281aef30 con 0x5626f0d8ec40 2024-06-06T07:53:17.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53281b03a0 con 0x5626f0d893d0 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.140+0000 7f53396f9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53281af600 con 0x5626f0d8f560 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f53396f9b80 10 client.0 _ll_register_callbacks cb 0x5626f0c74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45248/0 (socket says 172.21.15.87:45248) 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 -- 172.21.15.87:0/3115499140 learned_addr learned my addr 172.21.15.87:0/3115499140 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f532cc12700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d893d0 0x5626f0d983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f532d413700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0d98e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 -- 172.21.15.87:0/3115499140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 msgr2=0x5626f0d98e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0d98e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 -- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d893d0 msgr2=0x5626f0d983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d893d0 0x5626f0d983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 -- 172.21.15.87:0/3115499140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626f0e0db30 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f532d413700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0d98e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5327fff700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f531401e5a0 tx=0x7f531401e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e21d49145b8765 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5307fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314011ab0 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f53140289b0 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531c09c720 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5314028bb0 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f53396f9b80 10 client.16521 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.141+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626f0e0ee70 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c09c720 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5314004950 con 0x5626f0d8ec40 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:17.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 10 client.16521 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 20 client.16521 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0e0f5b0 0x5626f0e119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 4 client.16521 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "30414161", "release": true, "await": true}] 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "30414161", "release": true, "await": true}) -- 0x5626f0ce2390 con 0x5626f0e0f5b0 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.142+0000 7f532cc12700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0e0f5b0 0x5626f0e119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.143+0000 7f532cc12700 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0e0f5b0 0x5626f0e119d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fbbf294b9a496f server_cookie=d92c3e1e6a96ff92 in_seq=0 out_seq=0 2024-06-06T07:53:17.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.143+0000 7f5307fff700 10 client.16521 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f5307fff700 1 -- 172.21.15.87:0/3115499140 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5626f0ce2390 con 0x5626f0e0f5b0 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f5307fff700 10 client.16521 handle_command_reply: tid=0 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 client.16521 shutdown 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 msgr2=0x5626f0d98920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f531401e5a0 tx=0x7f531401e850 comp rx=0 tx=0).stop 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 shutdown_connections 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5626f0e0f5b0 0x5626f0e119d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5626f0d893d0 0x5626f0d983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626f0d8f560 0x5626f0d98e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 --2- 172.21.15.87:0/3115499140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5626f0d8ec40 0x5626f0d98920 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 >> 172.21.15.87:0/3115499140 conn(0x5626f0cb3dc0 msgr2=0x5626f0d93d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f5307fff700 0 client.16521 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 shutdown_connections 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 1 -- 172.21.15.87:0/3115499140 wait complete. 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.152+0000 7f53396f9b80 20 client.16521 trim_cache size 0 max 16384 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 16, 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "30414161": { 2024-06-06T07:53:17.155 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 16, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:53:17.156 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:53:17.157 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 msgr2=0x7f53180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 0x7f53180a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5314005fd0 tx=0x7f5314005f60 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 msgr2=0x7f53281978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f531c0038b0 tx=0x7f531c005590 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 shutdown_connections 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53281073f0 0x7f5328198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53180a2d90 0x7f53180a5250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328101380 0x7f53281978f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 --2- 172.21.15.87:0/3221486693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5328101d40 0x7f5328197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 >> 172.21.15.87:0/3221486693 conn(0x7f5328078300 msgr2=0x7f5328100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 shutdown_connections 2024-06-06T07:53:17.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:53:17.153+0000 7f532ee76700 1 -- 172.21.15.87:0/3221486693 wait complete. 2024-06-06T07:53:17.662 INFO:tasks.workunit.client.0.smithi087.stdout: 1 193943 4.59 MB/sec execute 81 sec latency 25320.683 ms 2024-06-06T07:53:17.960 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '30414161', total seconds elapsed: 31.5 2024-06-06T07:53:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:17 smithi087 ceph-mon[32276]: pgmap v161: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-06-06T07:53:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:17 smithi195 ceph-mon[37040]: pgmap v161: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-06-06T07:53:18.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:17 smithi181 ceph-mon[36790]: pgmap v161: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-06-06T07:53:18.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 195923 4.69 MB/sec execute 82 sec latency 15.782 ms 2024-06-06T07:53:19.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:18 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:53:19.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:18 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:53:19.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:18 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:53:19.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 198901 4.85 MB/sec execute 83 sec latency 15.275 ms 2024-06-06T07:53:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 -- 172.21.15.87:0/3082410121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 msgr2=0x7f9ca01004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 --2- 172.21.15.87:0/3082410121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca01004f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009900 tx=0x7f9c9002a410 comp rx=0 tx=0).stop 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 -- 172.21.15.87:0/3082410121 shutdown_connections 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 --2- 172.21.15.87:0/3082410121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca0106cd0 0x7f9ca010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 --2- 172.21.15.87:0/3082410121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0100ac0 0x7f9ca01065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 --2- 172.21.15.87:0/3082410121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca01004f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 -- 172.21.15.87:0/3082410121 >> 172.21.15.87:0/3082410121 conn(0x7f9ca0076e30 msgr2=0x7f9ca00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 -- 172.21.15.87:0/3082410121 shutdown_connections 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.729+0000 7f9ca5064640 1 -- 172.21.15.87:0/3082410121 wait complete. 2024-06-06T07:53:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 Processor -- start 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 -- start start 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca019ec40 0x7f9ca019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111d80 con 0x7f9ca0100110 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111c00 con 0x7f9ca0106cd0 2024-06-06T07:53:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.730+0000 7f9ca5064640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111f00 con 0x7f9ca019ec40 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45254/0 (socket says 172.21.15.87:45254) 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9f59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca019ec40 0x7f9ca019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2323157312 learned_addr learned my addr 172.21.15.87:0/2323157312 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2323157312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca019ec40 msgr2=0x7f9ca019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca019ec40 0x7f9ca019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2323157312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0106cd0 msgr2=0x7f9ca019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2323157312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca019f730 con 0x7f9ca0100110 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006010 tx=0x7f9c90004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9cd99640 1 -- 172.21.15.87:0/2323157312 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9003f030 con 0x7f9ca0100110 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca019f9c0 con 0x7f9ca0100110 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9cd99640 1 -- 172.21.15.87:0/2323157312 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c90004440 con 0x7f9ca0100110 2024-06-06T07:53:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9c9cd99640 1 -- 172.21.15.87:0/2323157312 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90035070 con 0x7f9ca0100110 2024-06-06T07:53:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.731+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a7180 con 0x7f9ca0100110 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.733+0000 7f9c9cd99640 1 -- 172.21.15.87:0/2323157312 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9c90042030 con 0x7f9ca0100110 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 Processor -- start 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 -- start start 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2a90 0x55de82bbce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bbd350 0x55de82bbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55de82b164f0 con 0x55de82bbd350 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55de82ab4870 con 0x55de82bb2170 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de82b124b0 con 0x55de82bb2a90 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9c9cd99640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 0x7f9c8c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9c9f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9c9659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bbd350 0x55de82bbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9c9f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:50960/0 (socket says 172.21.15.87:50960) 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.734+0000 7f9c9f59e640 1 -- 172.21.15.87:0/4277345563 learned_addr learned my addr 172.21.15.87:0/4277345563 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 0x7f9c8c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2a90 0x55de82bbce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9cd99640 1 -- 172.21.15.87:0/2323157312 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9c900efb70 con 0x7f9ca0100110 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 -- 172.21.15.87:0/4277345563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2a90 msgr2=0x55de82bbce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2a90 0x55de82bbce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 -- 172.21.15.87:0/4277345563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bbd350 msgr2=0x55de82bbf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bbd350 0x55de82bbf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 -- 172.21.15.87:0/4277345563 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de82ad09b0 con 0x55de82bb2170 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c98011da0 tx=0x7f9c9800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15efa3fa782eee10 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 0x7f9c8c0a5260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800fc90 tx=0x7f9c9801e000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9559a640 1 -- 172.21.15.87:0/4277345563 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98022070 con 0x55de82bb2170 2024-06-06T07:53:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9559a640 1 -- 172.21.15.87:0/4277345563 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c98009dc0 con 0x55de82bb2170 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.735+0000 7f9c9559a640 1 -- 172.21.15.87:0/4277345563 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004050 con 0x55de82bb2170 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- 172.21.15.87:0/4277345563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 msgr2=0x55de82bb2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c98011da0 tx=0x7f9c9800f790 comp rx=0 tx=0).stop 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- 172.21.15.87:0/4277345563 shutdown_connections 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bbd350 0x55de82bbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2a90 0x55de82bbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/4277345563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2170 0x55de82bb2550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- 172.21.15.87:0/4277345563 >> 172.21.15.87:0/4277345563 conn(0x55de82bae680 msgr2=0x55de82baea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- 172.21.15.87:0/4277345563 shutdown_connections 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- 172.21.15.87:0/4277345563 wait complete. 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 -- start start 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.736+0000 7f9ca7d74740 1 Processor -- start 2024-06-06T07:53:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bbae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55de82bbff20 con 0x55de82bb2170 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55de82a9fae0 con 0x55de82bb2a90 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de82bc00a0 con 0x55de82bbd350 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 10 client.0 _ll_register_callbacks cb 0x55de82ad89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45280/0 (socket says 172.21.15.87:45280) 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2058260549 learned_addr learned my addr 172.21.15.87:0/2058260549 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bbae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2058260549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 msgr2=0x55de82bbae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bbae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 msgr2=0x55de82bba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/2058260549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de8282a210 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9c900ecf50 tx=0x7f9c90086e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72731d17fd2b6e94 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c867fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90049050 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9c900b83e0 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.737+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c98009dc0 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9c900bbd70 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 10 client.16533 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55de828f6620 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c98009dc0 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9c90086a10 con 0x55de82bb2170 2024-06-06T07:53:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 10 client.16533 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 20 client.16533 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82bcd540 0x55de82bcf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 4 client.16533 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de82b29f10 con 0x55de82bcd540 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82bcd540 0x55de82bcf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82bcd540 0x55de82bcf9b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3b3f03586069cb server_cookie=6f6249d0fe3103e9 in_seq=0 out_seq=0 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.738+0000 7f9c867fc640 10 client.16533 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9c867fc640 1 -- 172.21.15.87:0/2058260549 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de82b29f10 con 0x55de82bcd540 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9c867fc640 10 client.16533 handle_command_reply: tid=0 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 client.16533 shutdown 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 msgr2=0x55de82bba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9c900ecf50 tx=0x7f9c90086e40 comp rx=0 tx=0).stop 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 shutdown_connections 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82bcd540 0x55de82bcf9b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bbae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9c867fc640 0 client.16533 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2058260549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bba3c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 >> 172.21.15.87:0/2058260549 conn(0x55de82bae680 msgr2=0x55de82bb4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 shutdown_connections 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2058260549 wait complete. 2024-06-06T07:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.739+0000 7f9ca7d74740 20 client.16533 trim_cache size 0 max 16384 2024-06-06T07:53:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.755+0000 7f9ca7d74740 1 Processor -- start 2024-06-06T07:53:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.755+0000 7f9ca7d74740 1 -- start start 2024-06-06T07:53:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.755+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.755+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bd52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bd5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b3960 con 0x55de82bb2170 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b1230 con 0x55de82bb2a90 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01af080 con 0x55de82bbd350 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45286/0 (socket says 172.21.15.87:45286) 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9f59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bd5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/228585547 learned_addr learned my addr 172.21.15.87:0/228585547 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9659c640 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bd52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/228585547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 msgr2=0x55de82bd5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bd5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/228585547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 msgr2=0x55de82bd52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bd52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 -- 172.21.15.87:0/228585547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de82bc9180 con 0x55de82bb2170 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9c900bba70 tx=0x7f9c900eff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e659d1548c1c98c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c94d99640 1 -- 172.21.15.87:0/228585547 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900d9070 con 0x55de82bb2170 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c94d99640 1 -- 172.21.15.87:0/228585547 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c900c0070 con 0x55de82bb2170 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.756+0000 7f9c94d99640 1 -- 172.21.15.87:0/228585547 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90049070 con 0x55de82bb2170 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- 172.21.15.87:0/228585547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 msgr2=0x55de82bbc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9c900bba70 tx=0x7f9c900eff00 comp rx=0 tx=0).stop 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- 172.21.15.87:0/228585547 shutdown_connections 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bbd350 0x55de82bd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bb2a90 0x55de82bd52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/228585547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2170 0x55de82bbc300 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- 172.21.15.87:0/228585547 >> 172.21.15.87:0/228585547 conn(0x55de82bae680 msgr2=0x55de82bbbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- 172.21.15.87:0/228585547 shutdown_connections 2024-06-06T07:53:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- 172.21.15.87:0/228585547 wait complete. 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 -- start start 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.757+0000 7f9ca7d74740 1 Processor -- start 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2170 0x55de82bc52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 0x55de82bc22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01af6f0 con 0x55de82bb2a90 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b0b60 con 0x55de82bbd350 2024-06-06T07:53:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01afdc0 con 0x55de82bb2170 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 10 client.0 _ll_register_callbacks cb 0x7f9c9003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45292/0 (socket says 172.21.15.87:45292) 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 -- 172.21.15.87:0/2276047473 learned_addr learned my addr 172.21.15.87:0/2276047473 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2170 0x55de82bc52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 0x55de82bc22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 -- 172.21.15.87:0/2276047473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2170 msgr2=0x55de82bc52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2170 0x55de82bc52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 -- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 msgr2=0x55de82bc22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 0x55de82bc22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 -- 172.21.15.87:0/2276047473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de82bc2800 con 0x55de82bb2a90 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9f59e640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 0x55de82bc22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c9659c640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9c88000c00 tx=0x7f9c88010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fcbcea4dbb590a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c867fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88022070 con 0x55de82bb2a90 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f9c8801f9f0 con 0x55de82bb2a90 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c900c0070 con 0x55de82bb2a90 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9c8801fbf0 con 0x55de82bb2a90 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.758+0000 7f9ca7d74740 10 client.16539 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55de82c38a20 con 0x55de82bb2a90 2024-06-06T07:53:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c900c0070 con 0x55de82bb2a90 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9c8801e5f0 con 0x55de82bb2a90 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 10 client.16539 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 20 client.16539 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82c39220 0x55de82c3b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 4 client.16539 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:53:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de82b23c20 con 0x55de82c39220 2024-06-06T07:53:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82c39220 0x55de82c3b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9c867fc640 10 client.16539 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.759+0000 7f9c9ed9d640 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82c39220 0x55de82c3b640 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24535043a2286692 server_cookie=9e44ba906b584d28 in_seq=0 out_seq=0 2024-06-06T07:53:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9c867fc640 1 -- 172.21.15.87:0/2276047473 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de82b23c20 con 0x55de82c39220 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9c867fc640 10 client.16539 handle_command_reply: tid=0 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 client.16539 shutdown 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 msgr2=0x55de82bc5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9c88000c00 tx=0x7f9c88010a40 comp rx=0 tx=0).stop 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 shutdown_connections 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de82c39220 0x55de82c3b640 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de82bbd350 0x55de82bc22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de82bb2a90 0x55de82bc5830 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9c867fc640 0 client.16539 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 --2- 172.21.15.87:0/2276047473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de82bb2170 0x55de82bc52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 >> 172.21.15.87:0/2276047473 conn(0x55de82bae680 msgr2=0x55de82bbac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 shutdown_connections 2024-06-06T07:53:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 1 -- 172.21.15.87:0/2276047473 wait complete. 2024-06-06T07:53:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.871+0000 7f9ca7d74740 20 client.16539 trim_cache size 0 max 16384 2024-06-06T07:53:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:53:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:53:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 msgr2=0x7f9c8c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 0x7f9c8c0a5260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800fc90 tx=0x7f9c9801e000 comp rx=0 tx=0).stop 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 msgr2=0x7f9ca019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006010 tx=0x7f9c90004040 comp rx=0 tx=0).stop 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 shutdown_connections 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c8c0a2da0 0x7f9c8c0a5260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ca019ec40 0x7f9ca019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 --2- 172.21.15.87:0/2323157312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca0100110 0x7f9ca019a380 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 >> 172.21.15.87:0/2323157312 conn(0x7f9ca0076e30 msgr2=0x7f9ca0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 shutdown_connections 2024-06-06T07:53:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:19.872+0000 7f9ca5064640 1 -- 172.21.15.87:0/2323157312 wait complete. 2024-06-06T07:53:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:19 smithi087 ceph-mon[32276]: pgmap v162: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:19 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:53:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:19 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:53:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:19 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:53:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:19 smithi195 ceph-mon[37040]: pgmap v162: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:19 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:53:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:19 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:53:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:19 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:53:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:19 smithi181 ceph-mon[36790]: pgmap v162: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:19 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:53:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:19 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:53:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:19 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:53:20.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 201198 4.89 MB/sec execute 84 sec latency 70.792 ms 2024-06-06T07:53:21.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 203245 4.99 MB/sec execute 85 sec latency 16.336 ms 2024-06-06T07:53:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:21 smithi195 ceph-mon[37040]: pgmap v163: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:21 smithi181 ceph-mon[36790]: pgmap v163: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:22.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:21 smithi087 ceph-mon[32276]: pgmap v163: 129 pgs: 129 active+clean; 38 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-06-06T07:53:22.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206120 5.13 MB/sec execute 86 sec latency 16.582 ms 2024-06-06T07:53:23.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 5.07 MB/sec execute 87 sec latency 829.012 ms 2024-06-06T07:53:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:23 smithi195 ceph-mon[37040]: pgmap v164: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-06-06T07:53:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:23 smithi181 ceph-mon[36790]: pgmap v164: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-06-06T07:53:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:23 smithi087 ceph-mon[32276]: pgmap v164: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-06-06T07:53:24.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 5.02 MB/sec execute 88 sec latency 1829.046 ms 2024-06-06T07:53:25.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.96 MB/sec execute 89 sec latency 2829.091 ms 2024-06-06T07:53:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:25 smithi195 ceph-mon[37040]: pgmap v165: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 KiB/s wr, 160 op/s 2024-06-06T07:53:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:25 smithi181 ceph-mon[36790]: pgmap v165: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 KiB/s wr, 160 op/s 2024-06-06T07:53:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:25 smithi087 ceph-mon[32276]: pgmap v165: 129 pgs: 129 active+clean; 42 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 KiB/s wr, 160 op/s 2024-06-06T07:53:26.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.90 MB/sec execute 90 sec latency 3829.211 ms 2024-06-06T07:53:27.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.85 MB/sec execute 91 sec latency 4829.301 ms 2024-06-06T07:53:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:27 smithi195 ceph-mon[37040]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-06-06T07:53:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:27 smithi181 ceph-mon[36790]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-06-06T07:53:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:27 smithi087 ceph-mon[32276]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-06-06T07:53:28.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.80 MB/sec execute 92 sec latency 5829.416 ms 2024-06-06T07:53:29.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.75 MB/sec execute 93 sec latency 6829.535 ms 2024-06-06T07:53:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:29 smithi195 ceph-mon[37040]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:29 smithi181 ceph-mon[36790]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:29 smithi087 ceph-mon[32276]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:30.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.70 MB/sec execute 94 sec latency 7829.636 ms 2024-06-06T07:53:31.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.65 MB/sec execute 95 sec latency 8829.694 ms 2024-06-06T07:53:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:31 smithi195 ceph-mon[37040]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:31 smithi181 ceph-mon[36790]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:31 smithi087 ceph-mon[32276]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1018 KiB/s wr, 191 op/s 2024-06-06T07:53:32.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 206323 4.60 MB/sec execute 96 sec latency 9829.794 ms 2024-06-06T07:53:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 -- 172.21.15.87:0/1408192275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 msgr2=0x7f0dec103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 --2- 172.21.15.87:0/1408192275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec103010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009900 tx=0x7f0ddc02a510 comp rx=0 tx=0).stop 2024-06-06T07:53:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 -- 172.21.15.87:0/1408192275 shutdown_connections 2024-06-06T07:53:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 --2- 172.21.15.87:0/1408192275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec10bf70 0x7f0dec10e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 --2- 172.21.15.87:0/1408192275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec103550 0x7f0dec10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 --2- 172.21.15.87:0/1408192275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec103010 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 -- 172.21.15.87:0/1408192275 >> 172.21.15.87:0/1408192275 conn(0x7f0dec077a20 msgr2=0x7f0dec077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 -- 172.21.15.87:0/1408192275 shutdown_connections 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 -- 172.21.15.87:0/1408192275 wait complete. 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.967+0000 7f0df2516640 1 Processor -- start 2024-06-06T07:53:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 -- start start 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec103550 0x7f0dec198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 0x7f0dec19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec111d70 con 0x7f0dec102c30 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec111bf0 con 0x7f0dec10bf70 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df2516640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec111ef0 con 0x7f0dec103550 2024-06-06T07:53:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35270/0 (socket says 172.21.15.87:35270) 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 -- 172.21.15.87:0/3000283485 learned_addr learned my addr 172.21.15.87:0/3000283485 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 0x7f0dec19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec103550 0x7f0dec198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 -- 172.21.15.87:0/3000283485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec103550 msgr2=0x7f0dec198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec103550 0x7f0dec198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 -- 172.21.15.87:0/3000283485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 msgr2=0x7f0dec19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 0x7f0dec19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0debfff640 1 -- 172.21.15.87:0/3000283485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec19d270 con 0x7f0dec102c30 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.968+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 0x7f0dec19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0debfff640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc005ab0 tx=0x7f0ddc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0de97fa640 1 -- 172.21.15.87:0/3000283485 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc03f030 con 0x7f0dec102c30 2024-06-06T07:53:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0de97fa640 1 -- 172.21.15.87:0/3000283485 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ddc003ce0 con 0x7f0dec102c30 2024-06-06T07:53:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec19d500 con 0x7f0dec102c30 2024-06-06T07:53:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0de97fa640 1 -- 172.21.15.87:0/3000283485 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc035070 con 0x7f0dec102c30 2024-06-06T07:53:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.969+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec1a4df0 con 0x7f0dec102c30 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.971+0000 7f0de97fa640 1 -- 172.21.15.87:0/3000283485 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0ddc042030 con 0x7f0dec102c30 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 Processor -- start 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 -- start start 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0de97fa640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 0x7f0dd40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74c170 0x55b9fe74c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe756e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe6b04f0 con 0x55b9fe757350 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe64e870 con 0x55b9fe74ca90 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df521c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe6ac4b0 con 0x55b9fe74c170 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 0x7f0dd40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0de97fa640 1 -- 172.21.15.87:0/3000283485 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0ddc0f01e0 con 0x7f0dec102c30 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35278/0 (socket says 172.21.15.87:35278) 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 -- 172.21.15.87:0/1216190493 learned_addr learned my addr 172.21.15.87:0/1216190493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0debfff640 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74c170 0x55b9fe74c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe756e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 -- 172.21.15.87:0/1216190493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74c170 msgr2=0x55b9fe74c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74c170 0x55b9fe74c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 -- 172.21.15.87:0/1216190493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 msgr2=0x55b9fe756e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe756e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.972+0000 7f0df0ab3640 1 -- 172.21.15.87:0/1216190493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9fe66a9b0 con 0x55b9fe757350 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 0x7f0dd40a5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0de400afd0 tx=0x7f0de400ad40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0de4010d90 tx=0x7f0de401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c676a8e818214e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0dd27fc640 1 -- 172.21.15.87:0/1216190493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4010e80 con 0x55b9fe757350 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0dd27fc640 1 -- 172.21.15.87:0/1216190493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4003e50 con 0x55b9fe757350 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0dd27fc640 1 -- 172.21.15.87:0/1216190493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4004310 con 0x55b9fe757350 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 -- 172.21.15.87:0/1216190493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 msgr2=0x55b9fe759740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0de4010d90 tx=0x7f0de401a000 comp rx=0 tx=0).stop 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 -- 172.21.15.87:0/1216190493 shutdown_connections 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe759740 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe756e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 --2- 172.21.15.87:0/1216190493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74c170 0x55b9fe74c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.973+0000 7f0df521c740 1 -- 172.21.15.87:0/1216190493 >> 172.21.15.87:0/1216190493 conn(0x55b9fe748680 msgr2=0x55b9fe748a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- 172.21.15.87:0/1216190493 shutdown_connections 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- 172.21.15.87:0/1216190493 wait complete. 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- start start 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 Processor -- start 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe754810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe754d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe759f20 con 0x55b9fe74c170 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe639ae0 con 0x55b9fe74ca90 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b9fe75a0a0 con 0x55b9fe757350 2024-06-06T07:53:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0df521c740 10 client.0 _ll_register_callbacks cb 0x55b9fe6729b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0debfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.974+0000 7f0debfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35288/0 (socket says 172.21.15.87:35288) 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 -- 172.21.15.87:0/3726585912 learned_addr learned my addr 172.21.15.87:0/3726585912 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe754d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe754810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 -- 172.21.15.87:0/3726585912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 msgr2=0x55b9fe754d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe754d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 -- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 msgr2=0x55b9fe754810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe754810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 -- 172.21.15.87:0/3726585912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9fe3c4210 con 0x55b9fe74c170 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe754810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0debfff640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0bc030 tx=0x7f0ddc0ba840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f9ff1fb5f6d2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0dcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc049070 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0ddc0b89a0 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de4003e50 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ddc0bca30 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0df521c740 10 client.16551 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9fe490620 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.975+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4003e50 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0ddc03b460 con 0x55b9fe74c170 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 10 client.16551 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 20 client.16551 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7674d0 0x55b9fe769940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 4 client.16551 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9fe6c3f10 con 0x55b9fe7674d0 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7674d0 0x55b9fe769940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0dcb7fe640 10 client.16551 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.976+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7674d0 0x55b9fe769940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7524337d630838b server_cookie=ad003f5a22801ca9 in_seq=0 out_seq=0 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/3726585912 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b9fe6c3f10 con 0x55b9fe7674d0 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0dcb7fe640 10 client.16551 handle_command_reply: tid=0 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 client.16551 shutdown 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 msgr2=0x55b9fe7542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0bc030 tx=0x7f0ddc0ba840 comp rx=0 tx=0).stop 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 shutdown_connections 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7674d0 0x55b9fe769940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe754d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74ca90 0x55b9fe754810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0dcb7fe640 0 client.16551 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 --2- 172.21.15.87:0/3726585912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74c170 0x55b9fe7542d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.977+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 >> 172.21.15.87:0/3726585912 conn(0x55b9fe748680 msgr2=0x55b9fe74e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.978+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 shutdown_connections 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.978+0000 7f0df521c740 1 -- 172.21.15.87:0/3726585912 wait complete. 2024-06-06T07:53:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.978+0000 7f0df521c740 20 client.16551 trim_cache size 0 max 16384 2024-06-06T07:53:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 Processor -- start 2024-06-06T07:53:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 -- start start 2024-06-06T07:53:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe74eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74ca90 0x55b9fe76f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1b16d0 con 0x55b9fe757350 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1aefa0 con 0x55b9fe74c170 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df521c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1acdf0 con 0x55b9fe74ca90 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35292/0 (socket says 172.21.15.87:35292) 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 -- 172.21.15.87:0/3654610128 learned_addr learned my addr 172.21.15.87:0/3654610128 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 -- 172.21.15.87:0/3654610128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74ca90 msgr2=0x55b9fe76f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74ca90 0x55b9fe76f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 -- 172.21.15.87:0/3654610128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 msgr2=0x55b9fe74eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe74eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 -- 172.21.15.87:0/3654610128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9fe752820 con 0x55b9fe757350 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0de4010d90 tx=0x7f0de400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea420de387b93e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.993+0000 7f0dd1ffb640 1 -- 172.21.15.87:0/3654610128 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4010e80 con 0x55b9fe757350 2024-06-06T07:53:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0dd1ffb640 1 -- 172.21.15.87:0/3654610128 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4013500 con 0x55b9fe757350 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0dd1ffb640 1 -- 172.21.15.87:0/3654610128 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40053d0 con 0x55b9fe757350 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- 172.21.15.87:0/3654610128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 msgr2=0x55b9fe76f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0de4010d90 tx=0x7f0de400a890 comp rx=0 tx=0).stop 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- 172.21.15.87:0/3654610128 shutdown_connections 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe757350 0x55b9fe76f790 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe74ca90 0x55b9fe76f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 --2- 172.21.15.87:0/3654610128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe74eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- 172.21.15.87:0/3654610128 >> 172.21.15.87:0/3654610128 conn(0x55b9fe748680 msgr2=0x55b9fe756390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- 172.21.15.87:0/3654610128 shutdown_connections 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- 172.21.15.87:0/3654610128 wait complete. 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.994+0000 7f0df521c740 1 -- start start 2024-06-06T07:53:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 Processor -- start 2024-06-06T07:53:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe75f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe750e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1ad460 con 0x55b9fe74ca90 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1ae8d0 con 0x55b9fe74c170 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec1adb30 con 0x55b9fe757350 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df521c740 10 client.0 _ll_register_callbacks cb 0x7f0ddc003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35304/0 (socket says 172.21.15.87:35304) 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 -- 172.21.15.87:0/423929944 learned_addr learned my addr 172.21.15.87:0/423929944 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0df0ab3640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe750e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0debfff640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe75f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 -- 172.21.15.87:0/423929944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 msgr2=0x55b9fe750e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe750e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 -- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 msgr2=0x55b9fe75f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe75f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0deb7fe640 1 -- 172.21.15.87:0/423929944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9fe751690 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.995+0000 7f0debfff640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe75f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0deb7fe640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0de001bfd0 tx=0x7f0de00138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db96aabb930a2ed server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0004220 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f0de00043c0 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de4013500 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0de0029050 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9fe7d1f70 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4013500 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0de00047a0 con 0x55b9fe74ca90 2024-06-06T07:53:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 20 client.16557 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7d2770 0x55b9fe7d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 4 client.16557 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.996+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9fe6bdc20 con 0x55b9fe7d2770 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.997+0000 7f0debfff640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7d2770 0x55b9fe7d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.997+0000 7f0dcb7fe640 10 client.16557 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:32.997+0000 7f0debfff640 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7d2770 0x55b9fe7d4b90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0abc8b2f95ee28 server_cookie=9eb34e28a0d6012f in_seq=0 out_seq=0 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0dcb7fe640 1 -- 172.21.15.87:0/423929944 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b9fe6bdc20 con 0x55b9fe7d2770 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0dcb7fe640 10 client.16557 handle_command_reply: tid=0 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 client.16557 shutdown 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 msgr2=0x55b9fe75fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0de001bfd0 tx=0x7f0de00138f0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 shutdown_connections 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b9fe7d2770 0x55b9fe7d4b90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9fe757350 0x55b9fe750e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0dcb7fe640 0 client.16557 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9fe74ca90 0x55b9fe75fc30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.083+0000 7f0df521c740 1 --2- 172.21.15.87:0/423929944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9fe74c170 0x55b9fe75f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 >> 172.21.15.87:0/423929944 conn(0x55b9fe748680 msgr2=0x55b9fe746f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 shutdown_connections 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df521c740 1 -- 172.21.15.87:0/423929944 wait complete. 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df521c740 20 client.16557 trim_cache size 0 max 16384 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:53:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:53:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 msgr2=0x7f0dd40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 0x7f0dd40a5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0de400afd0 tx=0x7f0de400ad40 comp rx=0 tx=0).stop 2024-06-06T07:53:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 msgr2=0x7f0dec1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc005ab0 tx=0x7f0ddc0042f0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 shutdown_connections 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0dd40a2da0 0x7f0dd40a5260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dec10bf70 0x7f0dec19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dec103550 0x7f0dec198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 --2- 172.21.15.87:0/3000283485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0dec102c30 0x7f0dec1982d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 >> 172.21.15.87:0/3000283485 conn(0x7f0dec077a20 msgr2=0x7f0dec0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 shutdown_connections 2024-06-06T07:53:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:33.084+0000 7f0df2516640 1 -- 172.21.15.87:0/3000283485 wait complete. 2024-06-06T07:53:33.663 INFO:tasks.workunit.client.0.smithi087.stdout: 1 207224 4.60 MB/sec execute 97 sec latency 10260.775 ms 2024-06-06T07:53:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:33 smithi195 ceph-mon[37040]: pgmap v169: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-06-06T07:53:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:33 smithi181 ceph-mon[36790]: pgmap v169: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-06-06T07:53:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:33 smithi087 ceph-mon[32276]: pgmap v169: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-06-06T07:53:34.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 209519 4.70 MB/sec execute 98 sec latency 14.730 ms 2024-06-06T07:53:35.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 211173 4.71 MB/sec execute 99 sec latency 18.632 ms 2024-06-06T07:53:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:35 smithi195 ceph-mon[37040]: pgmap v170: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 357 KiB/s wr, 45 op/s 2024-06-06T07:53:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:35 smithi181 ceph-mon[36790]: pgmap v170: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 357 KiB/s wr, 45 op/s 2024-06-06T07:53:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:35 smithi087 ceph-mon[32276]: pgmap v170: 129 pgs: 129 active+clean; 45 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 357 KiB/s wr, 45 op/s 2024-06-06T07:53:36.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 213412 4.80 MB/sec execute 100 sec latency 21.093 ms 2024-06-06T07:53:37.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 215865 4.87 MB/sec execute 101 sec latency 21.205 ms 2024-06-06T07:53:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:37 smithi195 ceph-mon[37040]: pgmap v171: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 136 op/s 2024-06-06T07:53:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:37 smithi181 ceph-mon[36790]: pgmap v171: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 136 op/s 2024-06-06T07:53:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:37 smithi087 ceph-mon[32276]: pgmap v171: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 136 op/s 2024-06-06T07:53:38.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 217706 4.91 MB/sec execute 102 sec latency 26.829 ms 2024-06-06T07:53:39.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 220129 5.02 MB/sec execute 103 sec latency 32.421 ms 2024-06-06T07:53:40.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:39 smithi195 ceph-mon[37040]: pgmap v172: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 612 KiB/s wr, 98 op/s 2024-06-06T07:53:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:39 smithi181 ceph-mon[36790]: pgmap v172: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 612 KiB/s wr, 98 op/s 2024-06-06T07:53:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:39 smithi087 ceph-mon[32276]: pgmap v172: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 612 KiB/s wr, 98 op/s 2024-06-06T07:53:40.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 221881 5.03 MB/sec execute 104 sec latency 18.323 ms 2024-06-06T07:53:41.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 224115 5.12 MB/sec execute 105 sec latency 15.927 ms 2024-06-06T07:53:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:41 smithi195 ceph-mon[37040]: pgmap v173: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 613 KiB/s wr, 98 op/s 2024-06-06T07:53:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:41 smithi181 ceph-mon[36790]: pgmap v173: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 613 KiB/s wr, 98 op/s 2024-06-06T07:53:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:41 smithi087 ceph-mon[32276]: pgmap v173: 129 pgs: 129 active+clean; 46 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 613 KiB/s wr, 98 op/s 2024-06-06T07:53:42.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 226484 5.19 MB/sec execute 106 sec latency 19.306 ms 2024-06-06T07:53:43.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 228441 5.25 MB/sec execute 107 sec latency 16.135 ms 2024-06-06T07:53:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:43 smithi195 ceph-mon[37040]: pgmap v174: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-06T07:53:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:43 smithi181 ceph-mon[36790]: pgmap v174: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-06T07:53:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:43 smithi087 ceph-mon[32276]: pgmap v174: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-06-06T07:53:44.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 230841 5.32 MB/sec execute 108 sec latency 40.236 ms 2024-06-06T07:53:45.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 232903 5.34 MB/sec execute 109 sec latency 23.348 ms 2024-06-06T07:53:46.161 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:45 smithi087 ceph-mon[32276]: pgmap v175: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 -- 172.21.15.87:0/2545101566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c101290 msgr2=0x7fed2c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 --2- 172.21.15.87:0/2545101566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c101290 0x7fed2c10d750 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fed24009cd0 tx=0x7fed2402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 -- 172.21.15.87:0/2545101566 shutdown_connections 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 --2- 172.21.15.87:0/2545101566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c101290 0x7fed2c10d750 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 --2- 172.21.15.87:0/2545101566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c1008d0 0x7fed2c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 --2- 172.21.15.87:0/2545101566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c106890 0x7fed2c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 -- 172.21.15.87:0/2545101566 >> 172.21.15.87:0/2545101566 conn(0x7fed2c077a20 msgr2=0x7fed2c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 -- 172.21.15.87:0/2545101566 shutdown_connections 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.178+0000 7fed32c83640 1 -- 172.21.15.87:0/2545101566 wait complete. 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 Processor -- start 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 -- start start 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c1008d0 0x7fed2c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c101290 0x7fed2c0fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110ca0 con 0x7fed2c106890 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110b20 con 0x7fed2c101290 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed32c83640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c110e20 con 0x7fed2c1008d0 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed30a1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c1008d0 0x7fed2c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41754/0 (socket says 172.21.15.87:41754) 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 -- 172.21.15.87:0/2663868431 learned_addr learned my addr 172.21.15.87:0/2663868431 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed23fff640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c101290 0x7fed2c0fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 -- 172.21.15.87:0/2663868431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c1008d0 msgr2=0x7fed2c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c1008d0 0x7fed2c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 -- 172.21.15.87:0/2663868431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c101290 msgr2=0x7fed2c0fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c101290 0x7fed2c0fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.179+0000 7fed31220640 1 -- 172.21.15.87:0/2663868431 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a6720 con 0x7fed2c106890 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed31220640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fed24005cc0 tx=0x7fed240093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed21ffb640 1 -- 172.21.15.87:0/2663868431 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24003e00 con 0x7fed2c106890 2024-06-06T07:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed21ffb640 1 -- 172.21.15.87:0/2663868431 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24004520 con 0x7fed2c106890 2024-06-06T07:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed21ffb640 1 -- 172.21.15.87:0/2663868431 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24030db0 con 0x7fed2c106890 2024-06-06T07:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a69b0 con 0x7fed2c106890 2024-06-06T07:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.180+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a6f00 con 0x7fed2c106890 2024-06-06T07:53:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.182+0000 7fed21ffb640 1 -- 172.21.15.87:0/2663868431 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fed240046d0 con 0x7fed2c106890 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 Processor -- start 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 -- start start 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9b170 0x55a3a3d9b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3da6350 0x55a3a3da8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3cff4f0 con 0x55a3a3da6350 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3c9d870 con 0x55a3a3d9ba90 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed3598a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3cfb4b0 con 0x55a3a3d9b170 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed21ffb640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 0x7fed140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed30a1f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed23fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3da6350 0x55a3a3da8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed31220640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 0x7fed140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed30a1f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:44370/0 (socket says 172.21.15.87:44370) 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed30a1f640 1 -- 172.21.15.87:0/883026730 learned_addr learned my addr 172.21.15.87:0/883026730 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed23fff640 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9b170 0x55a3a3d9b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.183+0000 7fed21ffb640 1 -- 172.21.15.87:0/2663868431 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fed2402e030 con 0x7fed2c106890 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 -- 172.21.15.87:0/883026730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9b170 msgr2=0x55a3a3d9b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9b170 0x55a3a3d9b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 -- 172.21.15.87:0/883026730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3da6350 msgr2=0x55a3a3da8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3da6350 0x55a3a3da8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 -- 172.21.15.87:0/883026730 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a3cb99b0 con 0x55a3a3d9ba90 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed31220640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 0x7fed140a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed240bcca0 tx=0x7fed24090010 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed30a1f640 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed28009ca0 tx=0x7fed28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dca10c7e3018ecc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed12ffd640 1 -- 172.21.15.87:0/883026730 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2801e070 con 0x55a3a3d9ba90 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed12ffd640 1 -- 172.21.15.87:0/883026730 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed28009e50 con 0x55a3a3d9ba90 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.184+0000 7fed12ffd640 1 -- 172.21.15.87:0/883026730 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280187e0 con 0x55a3a3d9ba90 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- 172.21.15.87:0/883026730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 msgr2=0x55a3a3da5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed28009ca0 tx=0x7fed28011810 comp rx=0 tx=0).stop 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- 172.21.15.87:0/883026730 shutdown_connections 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3da6350 0x55a3a3da8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da5e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 --2- 172.21.15.87:0/883026730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9b170 0x55a3a3d9b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- 172.21.15.87:0/883026730 >> 172.21.15.87:0/883026730 conn(0x55a3a3d97680 msgr2=0x55a3a3d97a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- 172.21.15.87:0/883026730 shutdown_connections 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- 172.21.15.87:0/883026730 wait complete. 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 -- start start 2024-06-06T07:53:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.185+0000 7fed3598a740 1 Processor -- start 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3da6350 0x55a3a3da3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3da8f80 con 0x55a3a3d9b170 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3c88ae0 con 0x55a3a3da6350 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a3a3da9100 con 0x55a3a3d9ba90 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed3598a740 10 client.0 _ll_register_callbacks cb 0x55a3a3cc19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41776/0 (socket says 172.21.15.87:41776) 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 -- 172.21.15.87:0/3353640703 learned_addr learned my addr 172.21.15.87:0/3353640703 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3da6350 0x55a3a3da3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 -- 172.21.15.87:0/3353640703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9ba90 msgr2=0x55a3a3da3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 -- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3da6350 msgr2=0x55a3a3da3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3da6350 0x55a3a3da3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 -- 172.21.15.87:0/3353640703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a3a13210 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed23fff640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0045b0 tx=0x7fed1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7278ae285802cedf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.186+0000 7fed0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c011050 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fed1c0111f0 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed28009e50 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed1c010430 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 10 client.16569 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3a3adf620 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed28009e50 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fed1c021420 con 0x55a3a3d9b170 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 10 client.16569 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 20 client.16569 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3db6570 0x55a3a3db8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 4 client.16569 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3a3d12f10 con 0x55a3a3db6570 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.187+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3db6570 0x55a3a3db8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.188+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3db6570 0x55a3a3db8960 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfb357a36fb8551 server_cookie=6100e7c780dfc1ea in_seq=0 out_seq=0 2024-06-06T07:53:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.188+0000 7fed0bfff640 10 client.16569 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed0bfff640 1 -- 172.21.15.87:0/3353640703 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a3a3d12f10 con 0x55a3a3db6570 2024-06-06T07:53:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed0bfff640 10 client.16569 handle_command_reply: tid=0 2024-06-06T07:53:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 client.16569 shutdown 2024-06-06T07:53:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 msgr2=0x55a3a3da32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0045b0 tx=0x7fed1c004730 comp rx=0 tx=0).stop 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 shutdown_connections 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3db6570 0x55a3a3db8960 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3da6350 0x55a3a3da3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 --2- 172.21.15.87:0/3353640703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3da32e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed0bfff640 0 client.16569 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 >> 172.21.15.87:0/3353640703 conn(0x55a3a3d97680 msgr2=0x55a3a3d9d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 shutdown_connections 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 1 -- 172.21.15.87:0/3353640703 wait complete. 2024-06-06T07:53:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.190+0000 7fed3598a740 20 client.16569 trim_cache size 0 max 16384 2024-06-06T07:53:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 Processor -- start 2024-06-06T07:53:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 -- start start 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3dbe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3da0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b38d0 con 0x55a3a3d9b170 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b11a0 con 0x55a3a3d9ba90 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed3598a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1aeff0 con 0x55a3a3da6350 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed23fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed23fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41782/0 (socket says 172.21.15.87:41782) 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed23fff640 1 -- 172.21.15.87:0/3543083392 learned_addr learned my addr 172.21.15.87:0/3543083392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3dbe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed23fff640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3da0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 -- 172.21.15.87:0/3543083392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 msgr2=0x55a3a3da0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3da0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 -- 172.21.15.87:0/3543083392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 msgr2=0x55a3a3dbe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed30a1f640 1 -- 172.21.15.87:0/3543083392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a3d98250 con 0x55a3a3d9ba90 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.207+0000 7fed23fff640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed30a1f640 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3dbe7b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed2801c040 tx=0x7fed28019f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133f14cf57e287e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed127fc640 1 -- 172.21.15.87:0/3543083392 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2801e040 con 0x55a3a3d9ba90 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed127fc640 1 -- 172.21.15.87:0/3543083392 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed28005320 con 0x55a3a3d9ba90 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed127fc640 1 -- 172.21.15.87:0/3543083392 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28019670 con 0x55a3a3d9ba90 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed3598a740 1 -- 172.21.15.87:0/3543083392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 msgr2=0x55a3a3dbe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.208+0000 7fed3598a740 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3dbe7b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed2801c040 tx=0x7fed28019f30 comp rx=0 tx=0).stop 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- 172.21.15.87:0/3543083392 shutdown_connections 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3da0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9ba90 0x55a3a3dbe7b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- 172.21.15.87:0/3543083392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9b170 0x55a3a3dbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- 172.21.15.87:0/3543083392 >> 172.21.15.87:0/3543083392 conn(0x55a3a3d97680 msgr2=0x55a3a3d9e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- 172.21.15.87:0/3543083392 shutdown_connections 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- 172.21.15.87:0/3543083392 wait complete. 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- start start 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 Processor -- start 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3daa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1af660 con 0x55a3a3d9ba90 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b0ad0 con 0x55a3a3d9b170 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1afd30 con 0x55a3a3da6350 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed3598a740 10 client.0 _ll_register_callbacks cb 0x7fed1c004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed30a1f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed23fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.209+0000 7fed30a1f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41786/0 (socket says 172.21.15.87:41786) 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed23fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:44408/0 (socket says 172.21.15.87:44408) 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 -- 172.21.15.87:0/1951563028 learned_addr learned my addr 172.21.15.87:0/1951563028 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed23fff640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3daa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 -- 172.21.15.87:0/1951563028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 msgr2=0x55a3a3daa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3daa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 -- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 msgr2=0x55a3a3daeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 -- 172.21.15.87:0/1951563028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a3daac60 con 0x55a3a3d9ba90 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed23fff640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed30a1f640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fed280098c0 tx=0x7fed28009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821836e37f189940 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28039060 con 0x55a3a3d9ba90 2024-06-06T07:53:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fed28005610 con 0x55a3a3d9ba90 2024-06-06T07:53:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.210+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed28005320 con 0x55a3a3d9ba90 2024-06-06T07:53:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed28040050 con 0x55a3a3d9ba90 2024-06-06T07:53:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 10 client.16575 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3a3e212c0 con 0x55a3a3d9ba90 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed28005320 con 0x55a3a3d9ba90 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fed280199f0 con 0x55a3a3d9ba90 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 10 client.16575 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 20 client.16575 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3e21780 0x55a3a3e23b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 4 client.16575 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3a3d0cc20 con 0x55a3a3e21780 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.211+0000 7fed23fff640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3e21780 0x55a3a3e23b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.212+0000 7fed23fff640 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3e21780 0x55a3a3e23b70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0230a9fd893b41d server_cookie=217f43214e23f032 in_seq=0 out_seq=0 2024-06-06T07:53:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.212+0000 7fed0bfff640 10 client.16575 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:45 smithi195 ceph-mon[37040]: pgmap v175: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-06-06T07:53:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:45 smithi181 ceph-mon[36790]: pgmap v175: 129 pgs: 129 active+clean; 54 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed0bfff640 1 -- 172.21.15.87:0/1951563028 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a3a3d0cc20 con 0x55a3a3e21780 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed0bfff640 10 client.16575 handle_command_reply: tid=0 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 client.16575 shutdown 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 msgr2=0x55a3a3da9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fed280098c0 tx=0x7fed28009e50 comp rx=0 tx=0).stop 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 shutdown_connections 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55a3a3e21780 0x55a3a3e23b70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed0bfff640 0 client.16575 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a3a3da6350 0x55a3a3daa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3a3d9ba90 0x55a3a3da9e80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 --2- 172.21.15.87:0/1951563028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3a3d9b170 0x55a3a3daeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 >> 172.21.15.87:0/1951563028 conn(0x55a3a3d97680 msgr2=0x55a3a3d9cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 shutdown_connections 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 1 -- 172.21.15.87:0/1951563028 wait complete. 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.430+0000 7fed3598a740 20 client.16575 trim_cache size 0 max 16384 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:53:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:53:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 msgr2=0x7fed140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 0x7fed140a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed240bcca0 tx=0x7fed24090010 comp rx=0 tx=0).stop 2024-06-06T07:53:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 msgr2=0x7fed2c0fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fed24005cc0 tx=0x7fed240093d0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 shutdown_connections 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fed140a2da0 0x7fed140a5260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed2c106890 0x7fed2c0fe2e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2c101290 0x7fed2c0fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 --2- 172.21.15.87:0/2663868431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c1008d0 0x7fed2c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 >> 172.21.15.87:0/2663868431 conn(0x7fed2c077a20 msgr2=0x7fed2c069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 shutdown_connections 2024-06-06T07:53:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:46.431+0000 7fed32c83640 1 -- 172.21.15.87:0/2663868431 wait complete. 2024-06-06T07:53:46.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 234686 5.40 MB/sec execute 110 sec latency 159.740 ms 2024-06-06T07:53:47.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 236907 5.45 MB/sec execute 111 sec latency 23.722 ms 2024-06-06T07:53:48.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:47 smithi195 ceph-mon[37040]: pgmap v176: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 369 op/s 2024-06-06T07:53:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:47 smithi181 ceph-mon[36790]: pgmap v176: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 369 op/s 2024-06-06T07:53:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:47 smithi087 ceph-mon[32276]: pgmap v176: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 369 op/s 2024-06-06T07:53:48.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 238804 5.50 MB/sec execute 112 sec latency 31.364 ms 2024-06-06T07:53:49.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 241461 5.60 MB/sec execute 113 sec latency 24.080 ms 2024-06-06T07:53:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:49 smithi195 ceph-mon[37040]: pgmap v177: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:49 smithi181 ceph-mon[36790]: pgmap v177: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:49 smithi087 ceph-mon[32276]: pgmap v177: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:50.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 243331 5.60 MB/sec execute 114 sec latency 20.321 ms 2024-06-06T07:53:51.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 245503 5.67 MB/sec execute 115 sec latency 15.908 ms 2024-06-06T07:53:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:51 smithi195 ceph-mon[37040]: pgmap v178: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:51 smithi181 ceph-mon[36790]: pgmap v178: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:51 smithi087 ceph-mon[32276]: pgmap v178: 129 pgs: 129 active+clean; 45 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 278 op/s 2024-06-06T07:53:52.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 248127 5.74 MB/sec execute 116 sec latency 15.216 ms 2024-06-06T07:53:53.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 249714 5.78 MB/sec execute 117 sec latency 60.222 ms 2024-06-06T07:53:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:53 smithi195 ceph-mon[37040]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-06-06T07:53:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:53 smithi181 ceph-mon[36790]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-06-06T07:53:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:53 smithi087 ceph-mon[32276]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-06-06T07:53:54.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 252086 5.85 MB/sec execute 118 sec latency 44.163 ms 2024-06-06T07:53:55.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 253659 5.84 MB/sec execute 119 sec latency 34.346 ms 2024-06-06T07:53:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:55 smithi195 ceph-mon[37040]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-06-06T07:53:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:55 smithi181 ceph-mon[36790]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-06-06T07:53:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:55 smithi087 ceph-mon[32276]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-06-06T07:53:56.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 256013 5.91 MB/sec execute 120 sec latency 18.191 ms 2024-06-06T07:53:57.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 258401 5.96 MB/sec execute 121 sec latency 16.944 ms 2024-06-06T07:53:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:57 smithi195 ceph-mon[37040]: pgmap v181: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-06-06T07:53:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:57 smithi181 ceph-mon[36790]: pgmap v181: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-06-06T07:53:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:57 smithi087 ceph-mon[32276]: pgmap v181: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-06-06T07:53:58.664 INFO:tasks.workunit.client.0.smithi087.stdout: 1 260422 6.01 MB/sec execute 122 sec latency 15.793 ms 2024-06-06T07:53:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 -- 172.21.15.87:0/4130240216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c101290 msgr2=0x7f2e8c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 --2- 172.21.15.87:0/4130240216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c101290 0x7f2e8c10d750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009cd0 tx=0x7f2e8402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 -- 172.21.15.87:0/4130240216 shutdown_connections 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 --2- 172.21.15.87:0/4130240216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c101290 0x7f2e8c10d750 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 --2- 172.21.15.87:0/4130240216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 0x7f2e8c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 --2- 172.21.15.87:0/4130240216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c106890 0x7f2e8c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 -- 172.21.15.87:0/4130240216 >> 172.21.15.87:0/4130240216 conn(0x7f2e8c077a20 msgr2=0x7f2e8c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 -- 172.21.15.87:0/4130240216 shutdown_connections 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.525+0000 7f2e9259e640 1 -- 172.21.15.87:0/4130240216 wait complete. 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 Processor -- start 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 -- start start 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 0x7f2e8c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c101290 0x7f2e8c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c110ca0 con 0x7f2e8c106890 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c110b20 con 0x7f2e8c101290 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e9259e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c110e20 con 0x7f2e8c1008d0 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e90b3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 0x7f2e8c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e90b3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33896/0 (socket says 172.21.15.87:33896) 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e90b3b640 1 -- 172.21.15.87:0/137177983 learned_addr learned my addr 172.21.15.87:0/137177983 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.526+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c101290 0x7f2e8c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 -- 172.21.15.87:0/137177983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 msgr2=0x7f2e8c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 0x7f2e8c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 -- 172.21.15.87:0/137177983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c101290 msgr2=0x7f2e8c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c101290 0x7f2e8c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 -- 172.21.15.87:0/137177983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c19d210 con 0x7f2e8c106890 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e90b3b640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005cc0 tx=0x7f2e84032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e897fa640 1 -- 172.21.15.87:0/137177983 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004040 con 0x7f2e8c106890 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e897fa640 1 -- 172.21.15.87:0/137177983 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e840046c0 con 0x7f2e8c106890 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c19d4a0 con 0x7f2e8c106890 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e897fa640 1 -- 172.21.15.87:0/137177983 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84030e00 con 0x7f2e8c106890 2024-06-06T07:53:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.527+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c1a4d90 con 0x7f2e8c106890 2024-06-06T07:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.529+0000 7f2e897fa640 1 -- 172.21.15.87:0/137177983 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2e840305a0 con 0x7f2e8c106890 2024-06-06T07:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 Processor -- start 2024-06-06T07:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 -- start start 2024-06-06T07:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c44d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c457e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c45a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56077c3b14f0 con 0x56077c44da90 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56077c34f870 con 0x56077c44d170 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e95388740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56077c3ad4b0 con 0x56077c458350 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e897fa640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 0x7f2e740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c457e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c44d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c457e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33904/0 (socket says 172.21.15.87:33904) 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8bfff640 1 -- 172.21.15.87:0/3430278759 learned_addr learned my addr 172.21.15.87:0/3430278759 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e90b3b640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 0x7f2e740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c45a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e897fa640 1 -- 172.21.15.87:0/137177983 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f2e8402e030 con 0x7f2e8c106890 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/3430278759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 msgr2=0x56077c45a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c45a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/3430278759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 msgr2=0x56077c457e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c457e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/3430278759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077c36b9b0 con 0x56077c44d170 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.530+0000 7f2e90b3b640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 0x7f2e740a5260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ee3c0 tx=0x7f2e840dd040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c44d550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009fd0 tx=0x7f2e80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb0183ea75000c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e727fc640 1 -- 172.21.15.87:0/3430278759 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80004050 con 0x56077c44d170 2024-06-06T07:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e727fc640 1 -- 172.21.15.87:0/3430278759 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e8000fc70 con 0x56077c44d170 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e727fc640 1 -- 172.21.15.87:0/3430278759 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80003950 con 0x56077c44d170 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 -- 172.21.15.87:0/3430278759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 msgr2=0x56077c44d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c44d550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009fd0 tx=0x7f2e80004730 comp rx=0 tx=0).stop 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 -- 172.21.15.87:0/3430278759 shutdown_connections 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c45a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c457e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 --2- 172.21.15.87:0/3430278759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c44d550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 -- 172.21.15.87:0/3430278759 >> 172.21.15.87:0/3430278759 conn(0x56077c449680 msgr2=0x56077c449a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 -- 172.21.15.87:0/3430278759 shutdown_connections 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.531+0000 7f2e95388740 1 -- 172.21.15.87:0/3430278759 wait complete. 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 -- start start 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 Processor -- start 2024-06-06T07:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44d170 0x56077c4552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c4557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56077c45af80 con 0x56077c44da90 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56077c33aae0 con 0x56077c458350 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56077c45b100 con 0x56077c44d170 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e95388740 10 client.0 _ll_register_callbacks cb 0x56077c3739b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.532+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c4557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:47472/0 (socket says 172.21.15.87:47472) 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/3963631774 learned_addr learned my addr 172.21.15.87:0/3963631774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44d170 0x56077c4552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 -- 172.21.15.87:0/3963631774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44d170 msgr2=0x56077c4552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44d170 0x56077c4552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 -- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 msgr2=0x56077c455d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 -- 172.21.15.87:0/3963631774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077c0c5210 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c4557f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00ba60 tx=0x7f2e7c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ba964b42d86c44 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c003d60 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f2e7c005020 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.533+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e8000fc70 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c024050 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 10 client.16587 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56077c191620 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e8000fc70 con 0x56077c44da90 2024-06-06T07:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2e7c011440 con 0x56077c44da90 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 10 client.16587 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 20 client.16587 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4685d0 0x56077c46a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 4 client.16587 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56077c3c4f10 con 0x56077c4685d0 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.534+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4685d0 0x56077c46a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4685d0 0x56077c46a9c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba85e5866f6a51 server_cookie=bc984353ee7a40ec in_seq=0 out_seq=0 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e6b7fe640 10 client.16587 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/3963631774 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56077c3c4f10 con 0x56077c4685d0 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e6b7fe640 10 client.16587 handle_command_reply: tid=0 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e95388740 1 client.16587 shutdown 2024-06-06T07:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 msgr2=0x56077c4557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.535+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c4557f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00ba60 tx=0x7f2e7c00bf20 comp rx=0 tx=0).stop 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 shutdown_connections 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4685d0 0x56077c46a9c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c458350 0x56077c455d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44da90 0x56077c4557f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e6b7fe640 0 client.16587 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 --2- 172.21.15.87:0/3963631774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44d170 0x56077c4552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 >> 172.21.15.87:0/3963631774 conn(0x56077c449680 msgr2=0x56077c44f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 shutdown_connections 2024-06-06T07:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 1 -- 172.21.15.87:0/3963631774 wait complete. 2024-06-06T07:53:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.536+0000 7f2e95388740 20 client.16587 trim_cache size 0 max 16384 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.551+0000 7f2e95388740 1 Processor -- start 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 -- start start 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44da90 0x56077c4571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c470290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b1740 con 0x56077c44d170 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1af010 con 0x56077c44da90 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e95388740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1ace60 con 0x56077c458350 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33932/0 (socket says 172.21.15.87:33932) 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/776458997 learned_addr learned my addr 172.21.15.87:0/776458997 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44da90 0x56077c4571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c470290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/776458997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 msgr2=0x56077c470290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c470290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/776458997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44da90 msgr2=0x56077c4571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44da90 0x56077c4571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 -- 172.21.15.87:0/776458997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077c467120 con 0x56077c44d170 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2e80011710 tx=0x7f2e800035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0bee23f451a0ae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e71ffb640 1 -- 172.21.15.87:0/776458997 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000f950 con 0x56077c44d170 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e71ffb640 1 -- 172.21.15.87:0/776458997 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e80034030 con 0x56077c44d170 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.552+0000 7f2e71ffb640 1 -- 172.21.15.87:0/776458997 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80004b20 con 0x56077c44d170 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- 172.21.15.87:0/776458997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 msgr2=0x56077c464900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2e80011710 tx=0x7f2e800035f0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- 172.21.15.87:0/776458997 shutdown_connections 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c458350 0x56077c470290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44da90 0x56077c4571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 --2- 172.21.15.87:0/776458997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c44d170 0x56077c464900 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- 172.21.15.87:0/776458997 >> 172.21.15.87:0/776458997 conn(0x56077c449680 msgr2=0x56077c456ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- 172.21.15.87:0/776458997 shutdown_connections 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- 172.21.15.87:0/776458997 wait complete. 2024-06-06T07:53:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.553+0000 7f2e95388740 1 -- start start 2024-06-06T07:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 Processor -- start 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c4607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c458350 0x56077c45c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1ad4d0 con 0x56077c458350 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1ae940 con 0x56077c44d170 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1adba0 con 0x56077c44da90 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e95388740 10 client.0 _ll_register_callbacks cb 0x7f2e7c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c458350 0x56077c45c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:45200/0 (socket says 172.21.15.87:45200) 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 -- 172.21.15.87:0/2258488111 learned_addr learned my addr 172.21.15.87:0/2258488111 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c4607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 -- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 msgr2=0x56077c4607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c4607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 -- 172.21.15.87:0/2258488111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c458350 msgr2=0x56077c45c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c458350 0x56077c45c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.554+0000 7f2e8bfff640 1 -- 172.21.15.87:0/2258488111 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077c45c5c0 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e8bfff640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c011810 tx=0x7f2e7c004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f253ef16ff9fac5a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c011a80 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e80034030 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f2e7c011c20 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c037050 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 10 client.25897 fetch_fsmap learned FSMap version 32 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56077c4d2fb0 con 0x56077c44da90 2024-06-06T07:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e80034030 con 0x56077c44da90 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2e7c027040 con 0x56077c44da90 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 10 client.25897 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 20 client.25897 populate_metadata read hostname 'smithi087' 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4d3750 0x56077c4d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 4 client.25897 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.555+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56077c3bec20 con 0x56077c4d3750 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.556+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4d3750 0x56077c4d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.556+0000 7f2e6b7fe640 10 client.25897 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.556+0000 7f2e8b7fe640 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4d3750 0x56077c4d5b70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23dad2e3575f107 server_cookie=dcb90870fcd1b68f in_seq=0 out_seq=0 2024-06-06T07:53:59.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 262671 6.07 MB/sec execute 123 sec latency 107.634 ms 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.741+0000 7f2e6b7fe640 1 -- 172.21.15.87:0/2258488111 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56077c3bec20 con 0x56077c4d3750 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.741+0000 7f2e6b7fe640 10 client.25897 handle_command_reply: tid=0 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.741+0000 7f2e95388740 1 client.25897 shutdown 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 msgr2=0x56077c45bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c011810 tx=0x7f2e7c004330 comp rx=0 tx=0).stop 2024-06-06T07:53:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 shutdown_connections 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56077c4d3750 0x56077c4d5b70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56077c458350 0x56077c45c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56077c44da90 0x56077c45bb40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 --2- 172.21.15.87:0/2258488111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56077c44d170 0x56077c4607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 >> 172.21.15.87:0/2258488111 conn(0x56077c449680 msgr2=0x56077c44f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e6b7fe640 0 client.25897 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 shutdown_connections 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 1 -- 172.21.15.87:0/2258488111 wait complete. 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.742+0000 7f2e95388740 20 client.25897 trim_cache size 0 max 16384 2024-06-06T07:53:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 msgr2=0x7f2e740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 0x7f2e740a5260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ee3c0 tx=0x7f2e840dd040 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 msgr2=0x7f2e8c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005cc0 tx=0x7f2e84032f40 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 shutdown_connections 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2e740a2da0 0x7f2e740a5260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e8c106890 0x7f2e8c19cae0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e8c101290 0x7f2e8c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 --2- 172.21.15.87:0/137177983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e8c1008d0 0x7f2e8c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 >> 172.21.15.87:0/137177983 conn(0x7f2e8c077a20 msgr2=0x7f2e8c069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 shutdown_connections 2024-06-06T07:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:53:59.743+0000 7f2e9259e640 1 -- 172.21.15.87:0/137177983 wait complete. 2024-06-06T07:54:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:53:59 smithi195 ceph-mon[37040]: pgmap v182: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:53:59 smithi181 ceph-mon[36790]: pgmap v182: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:53:59 smithi087 ceph-mon[32276]: pgmap v182: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:00.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 264171 6.07 MB/sec execute 124 sec latency 147.051 ms 2024-06-06T07:54:01.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 264950 6.04 MB/sec execute 125 sec latency 72.133 ms 2024-06-06T07:54:02.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:01 smithi195 ceph-mon[37040]: pgmap v183: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:01 smithi181 ceph-mon[36790]: pgmap v183: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:02.362 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:01 smithi087 ceph-mon[32276]: pgmap v183: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-06-06T07:54:02.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 266917 6.09 MB/sec execute 126 sec latency 24.435 ms 2024-06-06T07:54:03.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 269601 6.17 MB/sec execute 127 sec latency 16.943 ms 2024-06-06T07:54:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:03 smithi195 ceph-mon[37040]: pgmap v184: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 441 op/s 2024-06-06T07:54:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:03 smithi181 ceph-mon[36790]: pgmap v184: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 441 op/s 2024-06-06T07:54:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:03 smithi087 ceph-mon[32276]: pgmap v184: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 441 op/s 2024-06-06T07:54:04.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 271270 6.18 MB/sec execute 128 sec latency 20.209 ms 2024-06-06T07:54:05.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 273625 6.24 MB/sec execute 129 sec latency 16.723 ms 2024-06-06T07:54:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:05 smithi195 ceph-mon[37040]: pgmap v185: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-06-06T07:54:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:05 smithi181 ceph-mon[36790]: pgmap v185: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-06-06T07:54:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:05 smithi087 ceph-mon[32276]: pgmap v185: 129 pgs: 129 active+clean; 48 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-06-06T07:54:06.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 275908 6.27 MB/sec execute 130 sec latency 22.126 ms 2024-06-06T07:54:07.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 277835 6.30 MB/sec execute 131 sec latency 31.551 ms 2024-06-06T07:54:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:07 smithi195 ceph-mon[37040]: pgmap v186: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 415 op/s 2024-06-06T07:54:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:07 smithi181 ceph-mon[36790]: pgmap v186: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 415 op/s 2024-06-06T07:54:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:07 smithi087 ceph-mon[32276]: pgmap v186: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 415 op/s 2024-06-06T07:54:08.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 280191 6.37 MB/sec execute 132 sec latency 50.877 ms 2024-06-06T07:54:09.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 281679 6.38 MB/sec execute 133 sec latency 23.337 ms 2024-06-06T07:54:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:09 smithi195 ceph-mon[37040]: pgmap v187: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:09 smithi181 ceph-mon[36790]: pgmap v187: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:09 smithi087 ceph-mon[32276]: pgmap v187: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:10.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 283847 6.42 MB/sec execute 134 sec latency 18.574 ms 2024-06-06T07:54:11.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 285443 6.42 MB/sec execute 135 sec latency 15.675 ms 2024-06-06T07:54:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:11 smithi195 ceph-mon[37040]: pgmap v188: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:11 smithi181 ceph-mon[36790]: pgmap v188: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:11 smithi087 ceph-mon[32276]: pgmap v188: 129 pgs: 129 active+clean; 56 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-06-06T07:54:12.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 287801 6.48 MB/sec execute 136 sec latency 14.877 ms 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 -- 172.21.15.87:0/280188095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0107990 msgr2=0x7f22b0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 --2- 172.21.15.87:0/280188095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0107990 0x7f22b0107d70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009930 tx=0x7f22a002a420 comp rx=0 tx=0).stop 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 -- 172.21.15.87:0/280188095 shutdown_connections 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 --2- 172.21.15.87:0/280188095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0102340 0x7f22b010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 --2- 172.21.15.87:0/280188095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 0x7f22b0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 --2- 172.21.15.87:0/280188095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0107990 0x7f22b0107d70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 -- 172.21.15.87:0/280188095 >> 172.21.15.87:0/280188095 conn(0x7f22b00fd450 msgr2=0x7f22b00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 -- 172.21.15.87:0/280188095 shutdown_connections 2024-06-06T07:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 -- 172.21.15.87:0/280188095 wait complete. 2024-06-06T07:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.836+0000 7f22b50ee640 1 Processor -- start 2024-06-06T07:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 -- start start 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 0x7f22b0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 0x7f22b019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0111da0 con 0x7f22b0102340 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0111c20 con 0x7f22b0107990 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22b50ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0111f20 con 0x7f22b0101980 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22af59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 0x7f22b019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51186/0 (socket says 172.21.15.87:51186) 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 -- 172.21.15.87:0/518905839 learned_addr learned my addr 172.21.15.87:0/518905839 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22aed9d640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 0x7f22b0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 -- 172.21.15.87:0/518905839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 msgr2=0x7f22b0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 0x7f22b0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 -- 172.21.15.87:0/518905839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 msgr2=0x7f22b019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 0x7f22b019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 -- 172.21.15.87:0/518905839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b019d1e0 con 0x7f22b0102340 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22af59e640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 0x7f22b019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.837+0000 7f22ae59c640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f22a400e9f0 tx=0x7f22a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.838+0000 7f2297fff640 1 -- 172.21.15.87:0/518905839 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a400cd80 con 0x7f22b0102340 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.838+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b019d4d0 con 0x7f22b0102340 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.838+0000 7f2297fff640 1 -- 172.21.15.87:0/518905839 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4005500 con 0x7f22b0102340 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.838+0000 7f2297fff640 1 -- 172.21.15.87:0/518905839 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4010690 con 0x7f22b0102340 2024-06-06T07:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.838+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a4d60 con 0x7f22b0102340 2024-06-06T07:54:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.840+0000 7f2297fff640 1 -- 172.21.15.87:0/518905839 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f22a40046c0 con 0x7f22b0102340 2024-06-06T07:54:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.840+0000 7f22b7df6740 1 Processor -- start 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.840+0000 7f22b7df6740 1 -- start start 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e6f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6f3350 0x55d42e6f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d42e64c4f0 con 0x55d42e6e8170 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d42e5ea870 con 0x55d42e6f3350 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22b7df6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d42e6484b0 con 0x55d42e6e8a90 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51198/0 (socket says 172.21.15.87:51198) 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 -- 172.21.15.87:0/2933845297 learned_addr learned my addr 172.21.15.87:0/2933845297 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22aed9d640 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e6f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f2297fff640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 0x7f22980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22ae59c640 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6f3350 0x55d42e6f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 0x7f22980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 -- 172.21.15.87:0/2933845297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 msgr2=0x55d42e6f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e6f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f2297fff640 1 -- 172.21.15.87:0/518905839 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22a40cfc40 con 0x7f22b0102340 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 -- 172.21.15.87:0/2933845297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6f3350 msgr2=0x55d42e6f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6f3350 0x55d42e6f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 -- 172.21.15.87:0/2933845297 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42e6069b0 con 0x55d42e6e8170 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.841+0000 7f22af59e640 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009fd0 tx=0x7f22a800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c373e059cb4fff58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22957fa640 1 -- 172.21.15.87:0/2933845297 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8023020 con 0x55d42e6e8170 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22957fa640 1 -- 172.21.15.87:0/2933845297 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a80138a0 con 0x55d42e6e8170 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22957fa640 1 -- 172.21.15.87:0/2933845297 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8010670 con 0x55d42e6e8170 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22af59e640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 0x7f22980a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22a8024000 tx=0x7f22a8022000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 -- 172.21.15.87:0/2933845297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 msgr2=0x55d42e6e8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009fd0 tx=0x7f22a800f7d0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 -- 172.21.15.87:0/2933845297 shutdown_connections 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6f3350 0x55d42e6f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e6f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 --2- 172.21.15.87:0/2933845297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6e8550 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.842+0000 7f22b7df6740 1 -- 172.21.15.87:0/2933845297 >> 172.21.15.87:0/2933845297 conn(0x55d42e6e4680 msgr2=0x55d42e6e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- 172.21.15.87:0/2933845297 shutdown_connections 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- 172.21.15.87:0/2933845297 wait complete. 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- start start 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 Processor -- start 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e6f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d42e6f5f80 con 0x55d42e6e8a90 2024-06-06T07:54:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d42e5d5ae0 con 0x55d42e6e8170 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d42e6f6100 con 0x55d42e6f3350 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22b7df6740 10 client.0 _ll_register_callbacks cb 0x55d42e60e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51202/0 (socket says 172.21.15.87:51202) 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22ae59c640 1 -- 172.21.15.87:0/1039743455 learned_addr learned my addr 172.21.15.87:0/1039743455 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22af59e640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e6f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.843+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 -- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 msgr2=0x55d42e6f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 -- 172.21.15.87:0/1039743455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 msgr2=0x55d42e6f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 -- 172.21.15.87:0/1039743455 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42e360210 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22ae59c640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22af59e640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e6f0d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22a800fd40 tx=0x7f22a8009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229a93691a1dfe25 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8023070 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22a8012250 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d42e65ff10 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22a8039050 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22b7df6740 10 client.25909 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d42e42c620 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d42e65ff10 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.844+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f22a801f480 con 0x55d42e6f3350 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 10 client.25909 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 20 client.25909 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e703440 0x55d42e705830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 4 client.25909 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42e659c20 con 0x55d42e703440 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e703440 0x55d42e705830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e703440 0x55d42e705830 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d790c73265cbd9b4 server_cookie=1da8f4f8083217c3 in_seq=0 out_seq=0 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.845+0000 7f2292ff5640 10 client.25909 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f2292ff5640 1 -- 172.21.15.87:0/1039743455 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d42e659c20 con 0x55d42e703440 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f2292ff5640 10 client.25909 handle_command_reply: tid=0 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 client.25909 shutdown 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 msgr2=0x55d42e6f0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e6f0d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22a800fd40 tx=0x7f22a8009480 comp rx=0 tx=0).stop 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 shutdown_connections 2024-06-06T07:54:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e703440 0x55d42e705830 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e6f0d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8a90 0x55d42e6f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f2292ff5640 0 client.25909 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1039743455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 >> 172.21.15.87:0/1039743455 conn(0x55d42e6e4680 msgr2=0x55d42e6ea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 shutdown_connections 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 1 -- 172.21.15.87:0/1039743455 wait complete. 2024-06-06T07:54:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.846+0000 7f22b7df6740 20 client.25909 trim_cache size 0 max 16384 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 Processor -- start 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 -- start start 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e70b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b1770 con 0x55d42e6f3350 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01af040 con 0x55d42e6e8170 2024-06-06T07:54:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.862+0000 7f22b7df6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01ace90 con 0x55d42e6e8a90 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e70b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22af59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:52806/0 (socket says 172.21.15.87:52806) 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e70b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:44108/0 (socket says 172.21.15.87:44108) 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 -- 172.21.15.87:0/3605609512 learned_addr learned my addr 172.21.15.87:0/3605609512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22af59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51218/0 (socket says 172.21.15.87:51218) 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 -- 172.21.15.87:0/3605609512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 msgr2=0x55d42e70b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e70b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 -- 172.21.15.87:0/3605609512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 msgr2=0x55d42e70b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 -- 172.21.15.87:0/3605609512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42e6e9820 con 0x55d42e6e8170 2024-06-06T07:54:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22af59e640 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f22aed9d640 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22a002a940 tx=0x7f22a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f411d2c5ac09613 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f2294ff9640 1 -- 172.21.15.87:0/3605609512 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0035040 con 0x55d42e6e8170 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f2294ff9640 1 -- 172.21.15.87:0/3605609512 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a0005500 con 0x55d42e6e8170 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.863+0000 7f2294ff9640 1 -- 172.21.15.87:0/3605609512 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004bc0 con 0x55d42e6e8170 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- 172.21.15.87:0/3605609512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 msgr2=0x55d42e6f0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22a002a940 tx=0x7f22a0009260 comp rx=0 tx=0).stop 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- 172.21.15.87:0/3605609512 shutdown_connections 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6f3350 0x55d42e70b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6e8a90 0x55d42e70b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 --2- 172.21.15.87:0/3605609512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8170 0x55d42e6f0b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- 172.21.15.87:0/3605609512 >> 172.21.15.87:0/3605609512 conn(0x55d42e6e4680 msgr2=0x55d42e6ea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- 172.21.15.87:0/3605609512 shutdown_connections 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- 172.21.15.87:0/3605609512 wait complete. 2024-06-06T07:54:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 -- start start 2024-06-06T07:54:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.864+0000 7f22b7df6740 1 Processor -- start 2024-06-06T07:54:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8a90 0x55d42e6f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e766000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01ad500 con 0x55d42e6e8170 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01ae970 con 0x55d42e6e8a90 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01adbd0 con 0x55d42e6f3350 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22b7df6740 10 client.0 _ll_register_callbacks cb 0x7f22a409bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51220/0 (socket says 172.21.15.87:51220) 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 -- 172.21.15.87:0/1482100216 learned_addr learned my addr 172.21.15.87:0/1482100216 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22af59e640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e766000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22ae59c640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8a90 0x55d42e6f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 -- 172.21.15.87:0/1482100216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 msgr2=0x55d42e766000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e766000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 -- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8a90 msgr2=0x55d42e6f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8a90 0x55d42e6f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 -- 172.21.15.87:0/1482100216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42e766640 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22af59e640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e766000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f22aed9d640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005d50 tx=0x7f22a0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9d08013e7dd97c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.865+0000 7f2292ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0035030 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f22a0004940 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a0005500 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22a0063050 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 10 client.16611 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d42e76d9c0 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a0005500 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f22a0048940 con 0x55d42e6e8170 2024-06-06T07:54:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 10 client.16611 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 20 client.16611 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e76e0e0 0x55d42e770500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 4 client.16611 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.866+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d42e653f70 con 0x55d42e76e0e0 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.867+0000 7f22ae59c640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e76e0e0 0x55d42e770500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.867+0000 7f2292ff5640 10 client.16611 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:12.867+0000 7f22ae59c640 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e76e0e0 0x55d42e770500 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30902b7a4fecf606 server_cookie=cdb63f9fb94231b in_seq=0 out_seq=0 2024-06-06T07:54:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f2292ff5640 1 -- 172.21.15.87:0/1482100216 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d42e653f70 con 0x55d42e76e0e0 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f2292ff5640 10 client.16611 handle_command_reply: tid=0 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 client.16611 shutdown 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 msgr2=0x55d42e6f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005d50 tx=0x7f22a0005d80 comp rx=0 tx=0).stop 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 shutdown_connections 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d42e76e0e0 0x55d42e770500 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d42e6f3350 0x55d42e766000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f2292ff5640 0 client.16611 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d42e6e8a90 0x55d42e6f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 --2- 172.21.15.87:0/1482100216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d42e6e8170 0x55d42e6f6ae0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 >> 172.21.15.87:0/1482100216 conn(0x55d42e6e4680 msgr2=0x55d42e6fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 shutdown_connections 2024-06-06T07:54:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 1 -- 172.21.15.87:0/1482100216 wait complete. 2024-06-06T07:54:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.094+0000 7f22b7df6740 20 client.16611 trim_cache size 0 max 16384 2024-06-06T07:54:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:54:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:54:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 msgr2=0x7f22980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 0x7f22980a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22a8024000 tx=0x7f22a8022000 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 msgr2=0x7f22b0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f22a400e9f0 tx=0x7f22a400edf0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 shutdown_connections 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f22980a2da0 0x7f22980a5260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b0107990 0x7f22b019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22b0102340 0x7f22b0198780 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 --2- 172.21.15.87:0/518905839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b0101980 0x7f22b0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 >> 172.21.15.87:0/518905839 conn(0x7f22b00fd450 msgr2=0x7f22b00fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 shutdown_connections 2024-06-06T07:54:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:13.095+0000 7f22b50ee640 1 -- 172.21.15.87:0/518905839 wait complete. 2024-06-06T07:54:13.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 289024 6.47 MB/sec execute 137 sec latency 132.377 ms 2024-06-06T07:54:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:13 smithi195 ceph-mon[37040]: pgmap v189: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 414 op/s 2024-06-06T07:54:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:13 smithi181 ceph-mon[36790]: pgmap v189: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 414 op/s 2024-06-06T07:54:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:13 smithi087 ceph-mon[32276]: pgmap v189: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 414 op/s 2024-06-06T07:54:14.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 291216 6.52 MB/sec execute 138 sec latency 37.609 ms 2024-06-06T07:54:15.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 293075 6.52 MB/sec execute 139 sec latency 22.892 ms 2024-06-06T07:54:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:15 smithi195 ceph-mon[37040]: pgmap v190: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-06-06T07:54:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:15 smithi181 ceph-mon[36790]: pgmap v190: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-06-06T07:54:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:15 smithi087 ceph-mon[32276]: pgmap v190: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-06-06T07:54:16.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 295204 6.57 MB/sec execute 140 sec latency 17.601 ms 2024-06-06T07:54:17.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 297725 6.62 MB/sec execute 141 sec latency 32.588 ms 2024-06-06T07:54:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:17 smithi195 ceph-mon[37040]: pgmap v191: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-06-06T07:54:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:17 smithi181 ceph-mon[36790]: pgmap v191: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-06-06T07:54:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:17 smithi087 ceph-mon[32276]: pgmap v191: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-06-06T07:54:18.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 299220 6.62 MB/sec execute 142 sec latency 87.537 ms 2024-06-06T07:54:19.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 301487 6.69 MB/sec execute 143 sec latency 84.746 ms 2024-06-06T07:54:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:19 smithi195 ceph-mon[37040]: pgmap v192: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:19 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:54:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:19 smithi181 ceph-mon[36790]: pgmap v192: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:19 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:54:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:19 smithi087 ceph-mon[32276]: pgmap v192: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:19 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:54:20.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 303028 6.69 MB/sec execute 144 sec latency 19.234 ms 2024-06-06T07:54:21.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:20 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:54:21.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:20 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:54:21.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:20 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:54:21.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:20 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:54:21.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:20 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:54:21.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:20 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:54:21.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:20 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:54:21.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:20 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:54:21.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:20 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:54:21.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 305386 6.73 MB/sec execute 145 sec latency 29.602 ms 2024-06-06T07:54:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:21 smithi195 ceph-mon[37040]: pgmap v193: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:21 smithi181 ceph-mon[36790]: pgmap v193: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:22.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:21 smithi087 ceph-mon[32276]: pgmap v193: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-06-06T07:54:22.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 306757 6.73 MB/sec execute 146 sec latency 18.248 ms 2024-06-06T07:54:23.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 309160 6.78 MB/sec execute 147 sec latency 14.321 ms 2024-06-06T07:54:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:23 smithi195 ceph-mon[37040]: pgmap v194: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-06-06T07:54:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:23 smithi181 ceph-mon[36790]: pgmap v194: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-06-06T07:54:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:23 smithi087 ceph-mon[32276]: pgmap v194: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-06-06T07:54:24.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 310797 6.77 MB/sec execute 148 sec latency 78.337 ms 2024-06-06T07:54:25.665 INFO:tasks.workunit.client.0.smithi087.stdout: 1 312895 6.81 MB/sec execute 149 sec latency 21.749 ms 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 -- 172.21.15.87:0/2576560654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 msgr2=0x7f8f08100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/2576560654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f08100d50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009930 tx=0x7f8efc02a420 comp rx=0 tx=0).stop 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 -- 172.21.15.87:0/2576560654 shutdown_connections 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/2576560654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08101290 0x7f8f0810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/2576560654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f08100d50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/2576560654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08106890 0x7f8f08106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 -- 172.21.15.87:0/2576560654 >> 172.21.15.87:0/2576560654 conn(0x7f8f08077a20 msgr2=0x7f8f08077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 -- 172.21.15.87:0/2576560654 shutdown_connections 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.186+0000 7f8f0dd29640 1 -- 172.21.15.87:0/2576560654 wait complete. 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 Processor -- start 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 -- start start 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08101290 0x7f8f08197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08106890 0x7f8f0819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08110ca0 con 0x7f8f081008d0 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08110b20 con 0x7f8f08101290 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f0dd29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08110e20 con 0x7f8f08106890 2024-06-06T07:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52728/0 (socket says 172.21.15.87:52728) 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 -- 172.21.15.87:0/3360086729 learned_addr learned my addr 172.21.15.87:0/3360086729 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f07fff640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08106890 0x7f8f0819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08101290 0x7f8f08197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 -- 172.21.15.87:0/3360086729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08106890 msgr2=0x7f8f0819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08106890 0x7f8f0819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 -- 172.21.15.87:0/3360086729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08101290 msgr2=0x7f8f08197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08101290 0x7f8f08197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.187+0000 7f8f077fe640 1 -- 172.21.15.87:0/3360086729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0819c0d0 con 0x7f8f081008d0 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f077fe640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ecd0 tx=0x7f8ef800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f04ff9640 1 -- 172.21.15.87:0/3360086729 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef801c070 con 0x7f8f081008d0 2024-06-06T07:54:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0819c3c0 con 0x7f8f081008d0 2024-06-06T07:54:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f04ff9640 1 -- 172.21.15.87:0/3360086729 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef80178f0 con 0x7f8f081008d0 2024-06-06T07:54:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f04ff9640 1 -- 172.21.15.87:0/3360086729 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8016960 con 0x7f8f081008d0 2024-06-06T07:54:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.188+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f08069050 con 0x7f8f081008d0 2024-06-06T07:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.190+0000 7f8f04ff9640 1 -- 172.21.15.87:0/3360086729 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8ef801f070 con 0x7f8f081008d0 2024-06-06T07:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.190+0000 7f8f10a39740 1 Processor -- start 2024-06-06T07:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.190+0000 7f8f10a39740 1 -- start start 2024-06-06T07:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f04ff9640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 0x7f8ef00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55813a6354f0 con 0x55813a6d19b0 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55813a5d3870 con 0x55813a6d1090 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f10a39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55813a6314b0 con 0x55813a6e0760 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 0x7f8ef00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:47720/0 (socket says 172.21.15.87:47720) 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52740/0 (socket says 172.21.15.87:52740) 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 -- 172.21.15.87:0/3028687315 learned_addr learned my addr 172.21.15.87:0/3028687315 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f04ff9640 1 -- 172.21.15.87:0/3360086729 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f8ef80d11a0 con 0x7f8f081008d0 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 -- 172.21.15.87:0/3028687315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 msgr2=0x55813a6e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 -- 172.21.15.87:0/3028687315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 msgr2=0x55813a6e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 -- 172.21.15.87:0/3028687315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813a5ef9b0 con 0x55813a6d1090 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.191+0000 7f8f07fff640 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bb50 tx=0x7f8f000042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3abd9644004491 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 0x7f8ef00a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009900 tx=0x7f8efc0058d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8eedffb640 1 -- 172.21.15.87:0/3028687315 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00010460 con 0x55813a6d1090 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8eedffb640 1 -- 172.21.15.87:0/3028687315 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00010ae0 con 0x55813a6d1090 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8eedffb640 1 -- 172.21.15.87:0/3028687315 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000faa0 con 0x55813a6d1090 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 -- 172.21.15.87:0/3028687315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 msgr2=0x55813a6d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bb50 tx=0x7f8f000042b0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 -- 172.21.15.87:0/3028687315 shutdown_connections 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 --2- 172.21.15.87:0/3028687315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6d1470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 -- 172.21.15.87:0/3028687315 >> 172.21.15.87:0/3028687315 conn(0x55813a6cd680 msgr2=0x55813a6cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 -- 172.21.15.87:0/3028687315 shutdown_connections 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.192+0000 7f8f10a39740 1 -- 172.21.15.87:0/3028687315 wait complete. 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 -- start start 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 Processor -- start 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 0x55813a6dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6e0760 0x55813a6de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55813a6e3330 con 0x55813a6d19b0 2024-06-06T07:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55813a5beae0 con 0x55813a6e0760 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55813a6e34b0 con 0x55813a6d1090 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f10a39740 10 client.0 _ll_register_callbacks cb 0x55813a5f79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 0x55813a6dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52756/0 (socket says 172.21.15.87:52756) 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f06ffd640 1 -- 172.21.15.87:0/2077420816 learned_addr learned my addr 172.21.15.87:0/2077420816 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.193+0000 7f8f07fff640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6e0760 0x55813a6de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 -- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6e0760 msgr2=0x55813a6de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6e0760 0x55813a6de190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 -- 172.21.15.87:0/2077420816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 msgr2=0x55813a6ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 -- 172.21.15.87:0/2077420816 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813a349210 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 0x55813a6dd710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ce520 tx=0x7f8ef809cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a48718ecb0d6d5c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a5f60 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f8ef80a5ad0 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f00010ae0 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ef80a5d10 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f10a39740 10 client.25927 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55813a415620 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00010ae0 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.194+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8ef809c980 con 0x55813a6d1090 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 20 client.25927 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a6f08a0 0x55813a6f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 4 client.25927 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55813a648f10 con 0x55813a6f08a0 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a6f08a0 0x55813a6f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8ee6ffd640 10 client.25927 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.195+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a6f08a0 0x55813a6f2c90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5c683ea60f7384 server_cookie=3eec8b563bae4e59 in_seq=0 out_seq=0 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2077420816 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55813a648f10 con 0x55813a6f08a0 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8ee6ffd640 10 client.25927 handle_command_reply: tid=0 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 client.25927 shutdown 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 msgr2=0x55813a6dd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 0x55813a6dd710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ce520 tx=0x7f8ef809cfa0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 shutdown_connections 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a6f08a0 0x55813a6f2c90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6e0760 0x55813a6de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8ee6ffd640 0 client.25927 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2077420816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6d1090 0x55813a6dd710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 >> 172.21.15.87:0/2077420816 conn(0x55813a6cd680 msgr2=0x55813a6d34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 shutdown_connections 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 1 -- 172.21.15.87:0/2077420816 wait complete. 2024-06-06T07:54:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.196+0000 7f8f10a39740 20 client.25927 trim_cache size 0 max 16384 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.212+0000 7f8f10a39740 1 Processor -- start 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.212+0000 7f8f10a39740 1 -- start start 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.212+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.212+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d19b0 0x55813a6f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f10a39740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081af500 con 0x55813a6d1090 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f10a39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081acdd0 con 0x55813a6d19b0 2024-06-06T07:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f10a39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081aac20 con 0x55813a6e0760 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52768/0 (socket says 172.21.15.87:52768) 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 -- 172.21.15.87:0/1496131024 learned_addr learned my addr 172.21.15.87:0/1496131024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d19b0 0x55813a6f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f07fff640 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 -- 172.21.15.87:0/1496131024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 msgr2=0x55813a6dd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6dd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 -- 172.21.15.87:0/1496131024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d19b0 msgr2=0x55813a6f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d19b0 0x55813a6f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 -- 172.21.15.87:0/1496131024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813a6d17e0 con 0x55813a6d1090 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8f077fe640 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8ef809d0a0 tx=0x7f8ef80163b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec07976e202c6b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8eed7fa640 1 -- 172.21.15.87:0/1496131024 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8099640 con 0x55813a6d1090 2024-06-06T07:54:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8eed7fa640 1 -- 172.21.15.87:0/1496131024 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef80a5f60 con 0x55813a6d1090 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.213+0000 7f8eed7fa640 1 -- 172.21.15.87:0/1496131024 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80bd0f0 con 0x55813a6d1090 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- 172.21.15.87:0/1496131024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 msgr2=0x55813a6f8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8ef809d0a0 tx=0x7f8ef80163b0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- 172.21.15.87:0/1496131024 shutdown_connections 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6dd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d19b0 0x55813a6f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 --2- 172.21.15.87:0/1496131024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d1090 0x55813a6f8560 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- 172.21.15.87:0/1496131024 >> 172.21.15.87:0/1496131024 conn(0x55813a6cd680 msgr2=0x55813a6d89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- 172.21.15.87:0/1496131024 shutdown_connections 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- 172.21.15.87:0/1496131024 wait complete. 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 -- start start 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.214+0000 7f8f10a39740 1 Processor -- start 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081ab290 con 0x55813a6d19b0 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081ac700 con 0x55813a6d1090 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081ab960 con 0x55813a6e0760 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 10 client.0 _ll_register_callbacks cb 0x7f8ef80099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52774/0 (socket says 172.21.15.87:52774) 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 -- 172.21.15.87:0/2820022606 learned_addr learned my addr 172.21.15.87:0/2820022606 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f07fff640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 -- 172.21.15.87:0/2820022606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 msgr2=0x55813a6e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 -- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 msgr2=0x55813a6e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 -- 172.21.15.87:0/2820022606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813a6e4af0 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f07fff640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f06ffd640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8efc004770 tx=0x7f8efc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a294d2f026f87d55 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc035040 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f8efc003770 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.215+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef80a5f60 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8efc061050 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 10 client.16629 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55813a75b370 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef80a5f60 con 0x55813a6d19b0 2024-06-06T07:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8efc05ac20 con 0x55813a6d19b0 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 10 client.16629 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 20 client.16629 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a75bad0 0x55813a75def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 4 client.16629 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55813a642c20 con 0x55813a75bad0 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.216+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a75bad0 0x55813a75def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.217+0000 7f8ee6ffd640 10 client.16629 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.217+0000 7f8f077fe640 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a75bad0 0x55813a75def0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db23bb48a12def60 server_cookie=5f391ad23f3ce09f in_seq=0 out_seq=0 2024-06-06T07:54:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:25 smithi195 ceph-mon[37040]: pgmap v195: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2024-06-06T07:54:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:25 smithi181 ceph-mon[36790]: pgmap v195: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2024-06-06T07:54:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:25 smithi087 ceph-mon[32276]: pgmap v195: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.435+0000 7f8ee6ffd640 1 -- 172.21.15.87:0/2820022606 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55813a642c20 con 0x55813a75bad0 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.435+0000 7f8ee6ffd640 10 client.16629 handle_command_reply: tid=0 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.435+0000 7f8f10a39740 1 client.16629 shutdown 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 msgr2=0x55813a6e3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8efc004770 tx=0x7f8efc02a420 comp rx=0 tx=0).stop 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 shutdown_connections 2024-06-06T07:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55813a75bad0 0x55813a75def0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55813a6e0760 0x55813a6e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55813a6d19b0 0x55813a6e3ff0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 --2- 172.21.15.87:0/2820022606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55813a6d1090 0x55813a6e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 >> 172.21.15.87:0/2820022606 conn(0x55813a6cd680 msgr2=0x55813a6cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8ee6ffd640 0 client.16629 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 shutdown_connections 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 1 -- 172.21.15.87:0/2820022606 wait complete. 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.436+0000 7f8f10a39740 20 client.16629 trim_cache size 0 max 16384 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:54:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:54:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 msgr2=0x7f8ef00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 0x7f8ef00a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009900 tx=0x7f8efc0058d0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 msgr2=0x7f8f081970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ecd0 tx=0x7f8ef800b690 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 shutdown_connections 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8ef00a2da0 0x7f8ef00a5260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f08106890 0x7f8f0819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f08101290 0x7f8f08197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 --2- 172.21.15.87:0/3360086729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f081008d0 0x7f8f081970e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 >> 172.21.15.87:0/3360086729 conn(0x7f8f08077a20 msgr2=0x7f8f08103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 shutdown_connections 2024-06-06T07:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:26.437+0000 7f8f0dd29640 1 -- 172.21.15.87:0/3360086729 wait complete. 2024-06-06T07:54:26.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 314903 6.84 MB/sec execute 150 sec latency 99.869 ms 2024-06-06T07:54:27.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 316887 6.86 MB/sec execute 151 sec latency 15.617 ms 2024-06-06T07:54:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:27 smithi195 ceph-mon[37040]: pgmap v196: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 611 op/s 2024-06-06T07:54:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:27 smithi181 ceph-mon[36790]: pgmap v196: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 611 op/s 2024-06-06T07:54:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:27 smithi087 ceph-mon[32276]: pgmap v196: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 611 op/s 2024-06-06T07:54:28.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 319485 6.93 MB/sec execute 152 sec latency 30.971 ms 2024-06-06T07:54:29.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 320895 6.92 MB/sec execute 153 sec latency 89.234 ms 2024-06-06T07:54:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:29 smithi195 ceph-mon[37040]: pgmap v197: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:29 smithi181 ceph-mon[36790]: pgmap v197: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:29 smithi087 ceph-mon[32276]: pgmap v197: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:30.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 323086 6.96 MB/sec execute 154 sec latency 62.769 ms 2024-06-06T07:54:31.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 325163 6.96 MB/sec execute 155 sec latency 15.244 ms 2024-06-06T07:54:32.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:31 smithi195 ceph-mon[37040]: pgmap v198: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:31 smithi181 ceph-mon[36790]: pgmap v198: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:31 smithi087 ceph-mon[32276]: pgmap v198: 129 pgs: 129 active+clean; 54 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-06-06T07:54:32.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 327211 7.00 MB/sec execute 156 sec latency 15.258 ms 2024-06-06T07:54:33.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 329883 7.06 MB/sec execute 157 sec latency 35.390 ms 2024-06-06T07:54:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:33 smithi195 ceph-mon[37040]: pgmap v199: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-06-06T07:54:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:33 smithi181 ceph-mon[36790]: pgmap v199: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-06-06T07:54:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:33 smithi087 ceph-mon[32276]: pgmap v199: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-06-06T07:54:34.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 331320 7.06 MB/sec execute 158 sec latency 48.590 ms 2024-06-06T07:54:35.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 333640 7.10 MB/sec execute 159 sec latency 36.473 ms 2024-06-06T07:54:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:35 smithi195 ceph-mon[37040]: pgmap v200: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 361 op/s 2024-06-06T07:54:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:35 smithi181 ceph-mon[36790]: pgmap v200: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 361 op/s 2024-06-06T07:54:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:35 smithi087 ceph-mon[32276]: pgmap v200: 129 pgs: 129 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 361 op/s 2024-06-06T07:54:36.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 335407 7.10 MB/sec execute 160 sec latency 16.631 ms 2024-06-06T07:54:37.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 337655 7.14 MB/sec execute 161 sec latency 15.995 ms 2024-06-06T07:54:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:37 smithi195 ceph-mon[37040]: pgmap v201: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-06-06T07:54:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:37 smithi181 ceph-mon[36790]: pgmap v201: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-06-06T07:54:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:37 smithi087 ceph-mon[32276]: pgmap v201: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-06-06T07:54:38.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 339896 7.17 MB/sec execute 162 sec latency 42.870 ms 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 -- 172.21.15.87:0/261555584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5408103df0 msgr2=0x7f54081041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 --2- 172.21.15.87:0/261555584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5408103df0 0x7f54081041d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f802a410 comp rx=0 tx=0).stop 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 -- 172.21.15.87:0/261555584 shutdown_connections 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 --2- 172.21.15.87:0/261555584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f5408110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 --2- 172.21.15.87:0/261555584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54081047a0 0x7f54081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 --2- 172.21.15.87:0/261555584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5408103df0 0x7f54081041d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 -- 172.21.15.87:0/261555584 >> 172.21.15.87:0/261555584 conn(0x7f54080ff630 msgr2=0x7f5408101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 -- 172.21.15.87:0/261555584 shutdown_connections 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.537+0000 7f540cafc640 1 -- 172.21.15.87:0/261555584 wait complete. 2024-06-06T07:54:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.538+0000 7f540cafc640 1 Processor -- start 2024-06-06T07:54:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.538+0000 7f540cafc640 1 -- start start 2024-06-06T07:54:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408103df0 0x7f540819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f540819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5408113f70 con 0x7f540819ece0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5408113df0 con 0x7f5408103df0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54081140f0 con 0x7f5408108ec0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58052/0 (socket says 172.21.15.87:58052) 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f53fdd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f540819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 -- 172.21.15.87:0/3740391600 learned_addr learned my addr 172.21.15.87:0/3740391600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408103df0 0x7f540819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 -- 172.21.15.87:0/3740391600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 msgr2=0x7f540819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f540819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 -- 172.21.15.87:0/3740391600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408103df0 msgr2=0x7f540819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408103df0 0x7f540819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 -- 172.21.15.87:0/3740391600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540819f7d0 con 0x7f540819ece0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f540819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f5406d9d640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f540000eb10 tx=0x7f540000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f53fffff640 1 -- 172.21.15.87:0/3740391600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400004b20 con 0x7f540819ece0 2024-06-06T07:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f53fffff640 1 -- 172.21.15.87:0/3740391600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400004cc0 con 0x7f540819ece0 2024-06-06T07:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.539+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540819fac0 con 0x7f540819ece0 2024-06-06T07:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.540+0000 7f53fffff640 1 -- 172.21.15.87:0/3740391600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400010760 con 0x7f540819ece0 2024-06-06T07:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.540+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081a72b0 con 0x7f540819ece0 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.542+0000 7f53fffff640 1 -- 172.21.15.87:0/3740391600 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5400005020 con 0x7f540819ece0 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.542+0000 7f540f809740 1 Processor -- start 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.542+0000 7f540f809740 1 -- start start 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f53fffff640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 0x7f53f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564039d7b4f0 con 0x564039e17170 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564039d19870 con 0x564039e22350 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540f809740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564039d774b0 con 0x564039e17a90 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f53fffff640 1 -- 172.21.15.87:0/3740391600 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54000cfe50 con 0x7f540819ece0 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 0x7f53f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58054/0 (socket says 172.21.15.87:58054) 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 -- 172.21.15.87:0/456009364 learned_addr learned my addr 172.21.15.87:0/456009364 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f5406d9d640 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 -- 172.21.15.87:0/456009364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 msgr2=0x564039e21e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e21e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 -- 172.21.15.87:0/456009364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 msgr2=0x564039e24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 -- 172.21.15.87:0/456009364 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564039d359b0 con 0x564039e17170 2024-06-06T07:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 0x7f53f40a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564039e18710 tx=0x7f53f00096c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.543+0000 7f540659c640 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680b4ac05a858e77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f53fcd99640 1 -- 172.21.15.87:0/456009364 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8040070 con 0x564039e17170 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f53fcd99640 1 -- 172.21.15.87:0/456009364 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f53fcd99640 1 -- 172.21.15.87:0/456009364 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8035040 con 0x564039e17170 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 -- 172.21.15.87:0/456009364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 msgr2=0x564039e17550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f8005be0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 -- 172.21.15.87:0/456009364 shutdown_connections 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e21e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 --2- 172.21.15.87:0/456009364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e17550 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 -- 172.21.15.87:0/456009364 >> 172.21.15.87:0/456009364 conn(0x564039e13680 msgr2=0x564039e13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 -- 172.21.15.87:0/456009364 shutdown_connections 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.544+0000 7f540f809740 1 -- 172.21.15.87:0/456009364 wait complete. 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 -- start start 2024-06-06T07:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 Processor -- start 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e1f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e1fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564039e24f20 con 0x564039e17170 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564039d04ae0 con 0x564039e22350 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564039e250a0 con 0x564039e17a90 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540f809740 10 client.0 _ll_register_callbacks cb 0x564039d3d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f53fdd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e1f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f5406d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e1fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58058/0 (socket says 172.21.15.87:58058) 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 -- 172.21.15.87:0/1307225461 learned_addr learned my addr 172.21.15.87:0/1307225461 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 -- 172.21.15.87:0/1307225461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 msgr2=0x564039e1f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e1f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 -- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 msgr2=0x564039e1fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e1fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.545+0000 7f540659c640 1 -- 172.21.15.87:0/1307225461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564039a8f210 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f540659c640 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009670 tx=0x7f53f80052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ea57f374c1c051 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f53edffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f804b020 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f53f8004050 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53f8056050 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f540f809740 10 client.16641 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564039b5b620 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.546+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f53f8003710 con 0x564039e17170 2024-06-06T07:54:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 10 client.16641 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 20 client.16641 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 4 client.16641 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564039d8ef10 con 0x564039e32280 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f53edffb640 10 client.16641 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.547+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e34670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf2cf7a7e5cfe70 server_cookie=7b497c773df0e7c0 in_seq=0 out_seq=0 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f53edffb640 1 -- 172.21.15.87:0/1307225461 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564039d8ef10 con 0x564039e32280 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f53edffb640 10 client.16641 handle_command_reply: tid=0 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f540f809740 1 client.16641 shutdown 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 msgr2=0x564039e1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009670 tx=0x7f53f80052f0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.549+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 shutdown_connections 2024-06-06T07:54:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e34670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f53edffb640 0 client.16641 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e22350 0x564039e1fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e1f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 --2- 172.21.15.87:0/1307225461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e1f260 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 >> 172.21.15.87:0/1307225461 conn(0x564039e13680 msgr2=0x564039e193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 shutdown_connections 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 1 -- 172.21.15.87:0/1307225461 wait complete. 2024-06-06T07:54:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.550+0000 7f540f809740 20 client.16641 trim_cache size 0 max 16384 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 Processor -- start 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 -- start start 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17a90 0x564039e3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e22350 0x564039e18b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54081b3a10 con 0x564039e17170 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54081b12e0 con 0x564039e17a90 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540f809740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54081af130 con 0x564039e22350 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58062/0 (socket says 172.21.15.87:58062) 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 -- 172.21.15.87:0/706086823 learned_addr learned my addr 172.21.15.87:0/706086823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17a90 0x564039e3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f5406d9d640 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e22350 0x564039e18b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 -- 172.21.15.87:0/706086823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e22350 msgr2=0x564039e18b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e22350 0x564039e18b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 -- 172.21.15.87:0/706086823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17a90 msgr2=0x564039e3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17a90 0x564039e3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 -- 172.21.15.87:0/706086823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564039e178c0 con 0x564039e17170 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f540659c640 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f53f80052c0 tx=0x7f53f80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edd5b1c519da5b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f53effff640 1 -- 172.21.15.87:0/706086823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f804b070 con 0x564039e17170 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.566+0000 7f53effff640 1 -- 172.21.15.87:0/706086823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f53effff640 1 -- 172.21.15.87:0/706086823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8035070 con 0x564039e17170 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 -- 172.21.15.87:0/706086823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 msgr2=0x564039e39cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f53f80052c0 tx=0x7f53f80058d0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 -- 172.21.15.87:0/706086823 shutdown_connections 2024-06-06T07:54:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e22350 0x564039e18b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17a90 0x564039e3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 --2- 172.21.15.87:0/706086823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e17170 0x564039e39cd0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 -- 172.21.15.87:0/706086823 >> 172.21.15.87:0/706086823 conn(0x564039e13680 msgr2=0x564039e13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 -- 172.21.15.87:0/706086823 shutdown_connections 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.567+0000 7f540f809740 1 -- 172.21.15.87:0/706086823 wait complete. 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 -- start start 2024-06-06T07:54:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 Processor -- start 2024-06-06T07:54:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e25f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e22350 0x564039e26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54081af7a0 con 0x564039e22350 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54081b0c10 con 0x564039e17170 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54081afe70 con 0x564039e17a90 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540f809740 10 client.0 _ll_register_callbacks cb 0x7f53f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:55104/0 (socket says 172.21.15.87:55104) 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f540659c640 1 -- 172.21.15.87:0/1888436466 learned_addr learned my addr 172.21.15.87:0/1888436466 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f5406d9d640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e22350 0x564039e26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.568+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e25f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 -- 172.21.15.87:0/1888436466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 msgr2=0x564039e25f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e25f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 -- 172.21.15.87:0/1888436466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e22350 msgr2=0x564039e26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e22350 0x564039e26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 -- 172.21.15.87:0/1888436466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564039e9bca0 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540659c640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f8005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb87f1fdc624c191 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f53edffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f804b070 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f53f8003800 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53f806a050 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540f809740 10 client.25865 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.569+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564039e9ce50 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f80317b0 con 0x564039e17170 2024-06-06T07:54:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f53f803c440 con 0x564039e17170 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 10 client.25865 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 20 client.25865 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 4 client.25865 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564039d88c20 con 0x564039e32280 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f53edffb640 10 client.25865 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.570+0000 7f53fdd9b640 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e9f4f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2212a6d064c14e5f server_cookie=205558f605323d93 in_seq=0 out_seq=0 2024-06-06T07:54:39.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 341278 7.17 MB/sec execute 163 sec latency 51.679 ms 2024-06-06T07:54:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f53edffb640 1 -- 172.21.15.87:0/1888436466 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564039d88c20 con 0x564039e32280 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f53edffb640 10 client.25865 handle_command_reply: tid=0 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 client.25865 shutdown 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 msgr2=0x564039e2abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f8005430 comp rx=0 tx=0).stop 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 shutdown_connections 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564039e32280 0x564039e9f4f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564039e22350 0x564039e26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564039e17a90 0x564039e25f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f53edffb640 0 client.25865 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 --2- 172.21.15.87:0/1888436466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564039e17170 0x564039e2abf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 >> 172.21.15.87:0/1888436466 conn(0x564039e13680 msgr2=0x564039e1f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 shutdown_connections 2024-06-06T07:54:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.783+0000 7f540f809740 1 -- 172.21.15.87:0/1888436466 wait complete. 2024-06-06T07:54:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540f809740 20 client.25865 trim_cache size 0 max 16384 2024-06-06T07:54:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:54:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:54:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 msgr2=0x7f53f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 0x7f53f40a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564039e18710 tx=0x7f53f00096c0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 msgr2=0x7f540819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f540000eb10 tx=0x7f540000c2d0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 shutdown_connections 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53f40a2da0 0x7f53f40a5260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f540819ece0 0x7f540819f190 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408108ec0 0x7f540819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 --2- 172.21.15.87:0/3740391600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408103df0 0x7f540819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.784+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 >> 172.21.15.87:0/3740391600 conn(0x7f54080ff630 msgr2=0x7f5408101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.785+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 shutdown_connections 2024-06-06T07:54:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:39.785+0000 7f540cafc640 1 -- 172.21.15.87:0/3740391600 wait complete. 2024-06-06T07:54:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:39 smithi195 ceph-mon[37040]: pgmap v202: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:39 smithi181 ceph-mon[36790]: pgmap v202: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:39 smithi087 ceph-mon[32276]: pgmap v202: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:40.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 343128 7.17 MB/sec execute 164 sec latency 106.988 ms 2024-06-06T07:54:41.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 345255 7.21 MB/sec execute 165 sec latency 18.710 ms 2024-06-06T07:54:42.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:41 smithi195 ceph-mon[37040]: pgmap v203: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:41 smithi181 ceph-mon[36790]: pgmap v203: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:41 smithi087 ceph-mon[32276]: pgmap v203: 129 pgs: 129 active+clean; 67 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-06-06T07:54:42.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 347854 7.26 MB/sec execute 166 sec latency 16.668 ms 2024-06-06T07:54:43.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 349589 7.26 MB/sec execute 167 sec latency 15.025 ms 2024-06-06T07:54:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:43 smithi195 ceph-mon[37040]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-06-06T07:54:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:43 smithi181 ceph-mon[36790]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-06-06T07:54:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:43 smithi087 ceph-mon[32276]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-06-06T07:54:44.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 351661 7.29 MB/sec execute 168 sec latency 67.746 ms 2024-06-06T07:54:45.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 353394 7.29 MB/sec execute 169 sec latency 23.078 ms 2024-06-06T07:54:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:45 smithi195 ceph-mon[37040]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 355 op/s 2024-06-06T07:54:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:45 smithi181 ceph-mon[36790]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 355 op/s 2024-06-06T07:54:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:45 smithi087 ceph-mon[32276]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 355 op/s 2024-06-06T07:54:46.666 INFO:tasks.workunit.client.0.smithi087.stdout: 1 355460 7.32 MB/sec execute 170 sec latency 18.978 ms 2024-06-06T07:54:47.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 357102 7.32 MB/sec execute 171 sec latency 45.922 ms 2024-06-06T07:54:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:47 smithi195 ceph-mon[37040]: pgmap v206: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T07:54:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:47 smithi181 ceph-mon[36790]: pgmap v206: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T07:54:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:47 smithi087 ceph-mon[32276]: pgmap v206: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T07:54:48.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 359162 7.35 MB/sec execute 172 sec latency 15.966 ms 2024-06-06T07:54:49.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 361416 7.38 MB/sec execute 173 sec latency 42.281 ms 2024-06-06T07:54:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:49 smithi195 ceph-mon[37040]: pgmap v207: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:49 smithi181 ceph-mon[36790]: pgmap v207: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:49 smithi087 ceph-mon[32276]: pgmap v207: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:50.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 362859 7.38 MB/sec execute 174 sec latency 43.538 ms 2024-06-06T07:54:51.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 365454 7.43 MB/sec execute 175 sec latency 16.716 ms 2024-06-06T07:54:51.747 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.6 2024-06-06T07:54:51.748 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:54:52.002 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:54:52.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:51 smithi087 ceph-mon[32276]: pgmap v208: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:51 smithi195 ceph-mon[37040]: pgmap v208: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:51 smithi181 ceph-mon[36790]: pgmap v208: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2024-06-06T07:54:52.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 367044 7.43 MB/sec execute 176 sec latency 19.970 ms 2024-06-06T07:54:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 -- 172.21.15.87:0/2385972801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 msgr2=0x7f7000103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 --2- 172.21.15.87:0/2385972801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000103010 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009900 tx=0x7f6ff002a480 comp rx=0 tx=0).stop 2024-06-06T07:54:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 -- 172.21.15.87:0/2385972801 shutdown_connections 2024-06-06T07:54:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 --2- 172.21.15.87:0/2385972801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 0x7f700010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 --2- 172.21.15.87:0/2385972801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f700010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 --2- 172.21.15.87:0/2385972801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000103010 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 -- 172.21.15.87:0/2385972801 >> 172.21.15.87:0/2385972801 conn(0x7f70000779f0 msgr2=0x7f7000077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 -- 172.21.15.87:0/2385972801 shutdown_connections 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 -- 172.21.15.87:0/2385972801 wait complete. 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.882+0000 7f7005710640 1 Processor -- start 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 -- start start 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f7000198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 0x7f700019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111de0 con 0x7f7000102c30 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111c60 con 0x7f7000103550 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f7005710640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111f60 con 0x7f700010c000 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48712/0 (socket says 172.21.15.87:48712) 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 -- 172.21.15.87:0/2428897810 learned_addr learned my addr 172.21.15.87:0/2428897810 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f7000198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 0x7f700019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 -- 172.21.15.87:0/2428897810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 msgr2=0x7f700019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 0x7f700019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 -- 172.21.15.87:0/2428897810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 msgr2=0x7f7000198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f7000198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffeffd640 1 -- 172.21.15.87:0/2428897810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700019d1e0 con 0x7f7000102c30 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.883+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f7000198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004770 tx=0x7f6ff0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f6fe7fff640 1 -- 172.21.15.87:0/2428897810 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff003f070 con 0x7f7000102c30 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700019d470 con 0x7f7000102c30 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70001a4d60 con 0x7f7000102c30 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f6fe7fff640 1 -- 172.21.15.87:0/2428897810 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0031ca0 con 0x7f7000102c30 2024-06-06T07:54:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.884+0000 7f6fe7fff640 1 -- 172.21.15.87:0/2428897810 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0049060 con 0x7f7000102c30 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.886+0000 7f6fe7fff640 1 -- 172.21.15.87:0/2428897810 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6ff0042070 con 0x7f7000102c30 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.886+0000 7f700841d740 1 Processor -- start 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.886+0000 7f700841d740 1 -- start start 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372152550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 0x55f37215ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f3720b64f0 con 0x55f372152170 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f372054870 con 0x55f37215d350 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f700841d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3720b24b0 con 0x55f372152a90 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6fff7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 0x55f37215ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 0x55f37215ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49978/0 (socket says 172.21.15.87:49978) 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffeffd640 1 -- 172.21.15.87:0/2966085526 learned_addr learned my addr 172.21.15.87:0/2966085526 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6fe7fff640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 0x7f6fe80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/2966085526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 msgr2=0x55f37215ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 0x55f37215ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/2966085526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 msgr2=0x55f372152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/2966085526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3720709b0 con 0x55f37215d350 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6fe7fff640 1 -- 172.21.15.87:0/2428897810 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6ff00ef6c0 con 0x7f7000102c30 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 0x7f6fe80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.887+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215f740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009fa0 tx=0x7f6ff40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17656487a5c100f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f6fe57fa640 1 -- 172.21.15.87:0/2966085526 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004020 con 0x55f37215d350 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f6fe57fa640 1 -- 172.21.15.87:0/2966085526 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4005500 con 0x55f37215d350 2024-06-06T07:54:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f6fe57fa640 1 -- 172.21.15.87:0/2966085526 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004d40 con 0x55f37215d350 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 -- 172.21.15.87:0/2966085526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 msgr2=0x55f37215f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 0x7f6fe80a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8005fd0 tx=0x7f6ff8014040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215f740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009fa0 tx=0x7f6ff40118d0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 -- 172.21.15.87:0/2966085526 shutdown_connections 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215f740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152a90 0x55f37215ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 --2- 172.21.15.87:0/2966085526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.888+0000 7f700841d740 1 -- 172.21.15.87:0/2966085526 >> 172.21.15.87:0/2966085526 conn(0x55f37214e680 msgr2=0x55f37214ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- 172.21.15.87:0/2966085526 shutdown_connections 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- 172.21.15.87:0/2966085526 wait complete. 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- start start 2024-06-06T07:54:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 Processor -- start 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f37215a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f37215ff80 con 0x55f372152a90 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f37203fae0 con 0x55f37215d350 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f372160100 con 0x55f372152170 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f700841d740 10 client.0 _ll_register_callbacks cb 0x55f3720789b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f6ffe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f6ffe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48728/0 (socket says 172.21.15.87:48728) 2024-06-06T07:54:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/3192045531 learned_addr learned my addr 172.21.15.87:0/3192045531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f37215a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.889+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/3192045531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 msgr2=0x55f37215a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f37215a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 msgr2=0x55f37215ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/3192045531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f371dca210 con 0x55f372152a90 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f37215a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6ff401b040 tx=0x7f6ff40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2536589794c478 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6fe2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4028020 con 0x55f372152a90 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6ff4005300 con 0x55f372152a90 2024-06-06T07:54:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff4005500 con 0x55f372152a90 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ff4033050 con 0x55f372152a90 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f700841d740 10 client.16659 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f371e96620 con 0x55f372152a90 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.890+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4005500 con 0x55f372152a90 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6ff4023470 con 0x55f372152a90 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 10 client.16659 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 20 client.16659 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f37216d3b0 0x55f37216f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 4 client.16659 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3720c9f10 con 0x55f37216d3b0 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f37216d3b0 0x55f37216f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f6fe2ff5640 10 client.16659 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.891+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f37216d3b0 0x55f37216f7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eefc3a7f26dbacfc server_cookie=511dc5c4aed0121a in_seq=0 out_seq=0 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/3192045531 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f3720c9f10 con 0x55f37216d3b0 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f6fe2ff5640 10 client.16659 handle_command_reply: tid=0 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 client.16659 shutdown 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 msgr2=0x55f37215a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6ff401b040 tx=0x7f6ff40044c0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 shutdown_connections 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f37216d3b0 0x55f37216f7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f37215ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f37215a740 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 --2- 172.21.15.87:0/3192045531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f37215a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 >> 172.21.15.87:0/3192045531 conn(0x55f37214e680 msgr2=0x55f372154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f6fe2ff5640 0 client.16659 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 shutdown_connections 2024-06-06T07:54:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 1 -- 172.21.15.87:0/3192045531 wait complete. 2024-06-06T07:54:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.893+0000 7f700841d740 20 client.16659 trim_cache size 0 max 16384 2024-06-06T07:54:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 Processor -- start 2024-06-06T07:54:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 -- start start 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f3721591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f372175560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0128b0 con 0x55f372152a90 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec010180 con 0x55f37215d350 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f700841d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec00dfd0 con 0x55f372152170 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f372175560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48736/0 (socket says 172.21.15.87:48736) 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffe7fc640 1 -- 172.21.15.87:0/1290234766 learned_addr learned my addr 172.21.15.87:0/1290234766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f3721591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 -- 172.21.15.87:0/1290234766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 msgr2=0x55f3721591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f3721591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 -- 172.21.15.87:0/1290234766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 msgr2=0x55f372175020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6fff7fe640 1 -- 172.21.15.87:0/1290234766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f372149fe0 con 0x55f37215d350 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f3721591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.911+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.912+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f372175560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80104b0 tx=0x7f6ff8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662deba588dd7f65 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.912+0000 7f6fe4ff9640 1 -- 172.21.15.87:0/1290234766 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8010de0 con 0x55f37215d350 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.912+0000 7f6fe4ff9640 1 -- 172.21.15.87:0/1290234766 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff801c960 con 0x55f37215d350 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.912+0000 7f6fe4ff9640 1 -- 172.21.15.87:0/1290234766 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff801d9e0 con 0x55f37215d350 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- 172.21.15.87:0/1290234766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 msgr2=0x55f372175560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f372175560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80104b0 tx=0x7f6ff8017000 comp rx=0 tx=0).stop 2024-06-06T07:54:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- 172.21.15.87:0/1290234766 shutdown_connections 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f37215d350 0x55f372175560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152a90 0x55f372175020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- 172.21.15.87:0/1290234766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f372152170 0x55f3721591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- 172.21.15.87:0/1290234766 >> 172.21.15.87:0/1290234766 conn(0x55f37214e680 msgr2=0x55f372158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- 172.21.15.87:0/1290234766 shutdown_connections 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- 172.21.15.87:0/1290234766 wait complete. 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- start start 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 Processor -- start 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f372152a90 0x55f372165380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f37215d350 0x55f3721658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec00e640 con 0x55f372152170 2024-06-06T07:54:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec00fab0 con 0x55f372152a90 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec00ed10 con 0x55f37215d350 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f700841d740 10 client.0 _ll_register_callbacks cb 0x7f6ff00bb7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f6ffeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48738/0 (socket says 172.21.15.87:48738) 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.913+0000 7f6ffeffd640 1 -- 172.21.15.87:0/1678099664 learned_addr learned my addr 172.21.15.87:0/1678099664 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fff7fe640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f37215d350 0x55f3721658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f372152a90 0x55f372165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 -- 172.21.15.87:0/1678099664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f37215d350 msgr2=0x55f3721658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f37215d350 0x55f3721658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 -- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f372152a90 msgr2=0x55f372165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f372152a90 0x55f372165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 -- 172.21.15.87:0/1678099664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f372165f60 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6ffeffd640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00b7de0 tx=0x7f6ff00b7580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280b2953c9605902 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fe2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00c4540 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f6ff00b7ff0 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff801c960 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ff00bb980 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f700841d740 10 client.16665 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3721d7050 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.914+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff801c960 con 0x55f372152170 2024-06-06T07:54:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6ff00bbbe0 con 0x55f372152170 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 10 client.16665 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 20 client.16665 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3721d7720 0x55f3721d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 4 client.16665 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3720c3c20 con 0x55f3721d7720 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3721d7720 0x55f3721d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f6fe2ff5640 10 client.16665 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:52.915+0000 7f6ffe7fc640 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3721d7720 0x55f3721d9b10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60a094fc1ae6fa3 server_cookie=914e3dad8a2ffd3e in_seq=0 out_seq=0 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f6fe2ff5640 1 -- 172.21.15.87:0/1678099664 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f3720c3c20 con 0x55f3721d7720 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f6fe2ff5640 10 client.16665 handle_command_reply: tid=0 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 client.16665 shutdown 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 msgr2=0x55f372157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00b7de0 tx=0x7f6ff00b7580 comp rx=0 tx=0).stop 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 shutdown_connections 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3721d7720 0x55f3721d9b10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f37215d350 0x55f3721658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f372152a90 0x55f372165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f6fe2ff5640 0 client.16665 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 --2- 172.21.15.87:0/1678099664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f372152170 0x55f372157c20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.123+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 >> 172.21.15.87:0/1678099664 conn(0x55f37214e680 msgr2=0x55f372159360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 shutdown_connections 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f700841d740 1 -- 172.21.15.87:0/1678099664 wait complete. 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f700841d740 20 client.16665 trim_cache size 0 max 16384 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:54:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:54:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 msgr2=0x7f6fe80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 0x7f6fe80a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8005fd0 tx=0x7f6ff8014040 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 msgr2=0x7f7000198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004770 tx=0x7f6ff0003c10 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 shutdown_connections 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6fe80a2da0 0x7f6fe80a5260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f700010c000 0x7f700019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7000103550 0x7f7000198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 --2- 172.21.15.87:0/2428897810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7000102c30 0x7f7000198240 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.124+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 >> 172.21.15.87:0/2428897810 conn(0x7f70000779f0 msgr2=0x7f70000fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.125+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 shutdown_connections 2024-06-06T07:54:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:54:53.125+0000 7f7005710640 1 -- 172.21.15.87:0/2428897810 wait complete. 2024-06-06T07:54:53.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 369178 7.46 MB/sec execute 177 sec latency 96.641 ms 2024-06-06T07:54:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 -- 172.21.15.87:0/3615294004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006aac0 msgr2=0x7fddd010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 --2- 172.21.15.87:0/3615294004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006aac0 0x7fddd010ea00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009990 tx=0x7fddbc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 -- 172.21.15.87:0/3615294004 shutdown_connections 2024-06-06T07:54:53.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 --2- 172.21.15.87:0/3615294004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006a1a0 0x7fddd006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 --2- 172.21.15.87:0/3615294004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 0x7fddd0111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 --2- 172.21.15.87:0/3615294004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006aac0 0x7fddd010ea00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 -- 172.21.15.87:0/3615294004 >> 172.21.15.87:0/3615294004 conn(0x7fddd0078340 msgr2=0x7fddd0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:53.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 -- 172.21.15.87:0/3615294004 shutdown_connections 2024-06-06T07:54:53.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.878+0000 7fddd6c84700 1 -- 172.21.15.87:0/3615294004 wait complete. 2024-06-06T07:54:53.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 Processor -- start 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 -- start start 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 0x7fddd0103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 0x7fddd01040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01149d0 con 0x7fddd006a1a0 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0114850 con 0x7fddd006aac0 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.879+0000 7fddd6c84700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0114b50 con 0x7fddd010ef40 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddd4a20700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 0x7fddd0103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:48264/0 (socket says 172.21.15.87:48264) 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddd4a20700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 0x7fddd0103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48762/0 (socket says 172.21.15.87:48762) 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 -- 172.21.15.87:0/881613350 learned_addr learned my addr 172.21.15.87:0/881613350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:53.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddd5221700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 0x7fddd01040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 -- 172.21.15.87:0/881613350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 msgr2=0x7fddd01040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 0x7fddd01040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 -- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 msgr2=0x7fddd0103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 0x7fddd0103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 -- 172.21.15.87:0/881613350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd01ab610 con 0x7fddd006aac0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.880+0000 7fddcffff700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00bab0 tx=0x7fddbc00bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.881+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc027070 con 0x7fddd006aac0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.881+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd01ab8a0 con 0x7fddd006aac0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.881+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc003d70 con 0x7fddd006aac0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.881+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00f460 con 0x7fddd006aac0 2024-06-06T07:54:53.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.881+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd01abd50 con 0x7fddd006aac0 2024-06-06T07:54:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.882+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd01019b0 con 0x7fddd006aac0 2024-06-06T07:54:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.883+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fddbc02a070 con 0x7fddd006aac0 2024-06-06T07:54:53.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.884+0000 7fddcdffb700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 0x7fddc00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.884+0000 7fddd4a20700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 0x7fddc00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.884+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7fddbc0d0990 con 0x7fddd006aac0 2024-06-06T07:54:53.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.884+0000 7fddd4a20700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 0x7fddc00a52d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009c60 tx=0x7fddc4009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:53.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:53.888+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fddbc013070 con 0x7fddd006aac0 2024-06-06T07:54:54.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:53 smithi087 ceph-mon[32276]: pgmap v209: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-06-06T07:54:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.190+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd002dca0 con 0x7fddd006aac0 2024-06-06T07:54:54.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.191+0000 7fddcdffb700 1 -- 172.21.15.87:0/881613350 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7fddbc09d270 con 0x7fddd006aac0 2024-06-06T07:54:54.191 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:54:54.192 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:54:54.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 msgr2=0x7fddc00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:54.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 0x7fddc00a52d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009c60 tx=0x7fddc4009260 comp rx=0 tx=0).stop 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 msgr2=0x7fddd0103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00bab0 tx=0x7fddbc00bae0 comp rx=0 tx=0).stop 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 shutdown_connections 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fddc00a2e10 0x7fddc00a52d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd010ef40 0x7fddd01040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddd006a1a0 0x7fddd0103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 --2- 172.21.15.87:0/881613350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd006aac0 0x7fddd0103bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:54.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 >> 172.21.15.87:0/881613350 conn(0x7fddd0078340 msgr2=0x7fddd0110490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:54.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 shutdown_connections 2024-06-06T07:54:54.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:54.193+0000 7fddd6c84700 1 -- 172.21.15.87:0/881613350 wait complete. 2024-06-06T07:54:54.195 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:54:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:53 smithi195 ceph-mon[37040]: pgmap v209: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-06-06T07:54:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:53 smithi181 ceph-mon[36790]: pgmap v209: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-06-06T07:54:54.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 370532 7.45 MB/sec execute 178 sec latency 50.223 ms 2024-06-06T07:54:54.968 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 132.6 --await-for 45.0' 2024-06-06T07:54:54.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 132.6 --await-for 45.0 2024-06-06T07:54:55.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:54 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/881613350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:54:55.264 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:54:55.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:54 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/881613350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:54:55.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:54 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/881613350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:54:55.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 372832 7.49 MB/sec execute 179 sec latency 18.908 ms 2024-06-06T07:54:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:55 smithi195 ceph-mon[37040]: pgmap v210: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-06T07:54:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:55 smithi181 ceph-mon[36790]: pgmap v210: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-06T07:54:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:55 smithi087 ceph-mon[32276]: pgmap v210: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-06-06T07:54:56.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 374886 7.48 MB/sec execute 180 sec latency 26.355 ms 2024-06-06T07:54:57.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 -- 172.21.15.87:0/4098756352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101af0 msgr2=0x7f54841057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 --2- 172.21.15.87:0/4098756352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101af0 0x7f54841057a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f54740099c0 tx=0x7f5474009cd0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 -- 172.21.15.87:0/4098756352 shutdown_connections 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 --2- 172.21.15.87:0/4098756352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f548410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 --2- 172.21.15.87:0/4098756352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101140 0x7f5484101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 --2- 172.21.15.87:0/4098756352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101af0 0x7f54841057a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 -- 172.21.15.87:0/4098756352 >> 172.21.15.87:0/4098756352 conn(0x7f5484076630 msgr2=0x7f5484076a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 -- 172.21.15.87:0/4098756352 shutdown_connections 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 -- 172.21.15.87:0/4098756352 wait complete. 2024-06-06T07:54:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.314+0000 7f548b792700 1 Processor -- start 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 -- start start 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101af0 0x7f548419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f5484196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54841105d0 con 0x7f5484101140 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5484110450 con 0x7f5484105e80 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548b792700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5484110750 con 0x7f5484101af0 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60966/0 (socket says 172.21.15.87:60966) 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 -- 172.21.15.87:0/1331828608 learned_addr learned my addr 172.21.15.87:0/1331828608 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f5489d2f700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f5484196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f5488d2d700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101af0 0x7f548419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 -- 172.21.15.87:0/1331828608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101af0 msgr2=0x7f548419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101af0 0x7f548419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 -- 172.21.15.87:0/1331828608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 msgr2=0x7f5484196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f5484196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f548952e700 1 -- 172.21.15.87:0/1331828608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5484197070 con 0x7f5484101140 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.315+0000 7f5489d2f700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f5484196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f548952e700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f548000fec0 tx=0x7f548000d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f547a7fc700 1 -- 172.21.15.87:0/1331828608 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480004220 con 0x7f5484101140 2024-06-06T07:54:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f547a7fc700 1 -- 172.21.15.87:0/1331828608 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54800043c0 con 0x7f5484101140 2024-06-06T07:54:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5484197300 con 0x7f5484101140 2024-06-06T07:54:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f547a7fc700 1 -- 172.21.15.87:0/1331828608 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480017930 con 0x7f5484101140 2024-06-06T07:54:57.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.316+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841a3420 con 0x7f5484101140 2024-06-06T07:54:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.318+0000 7f547a7fc700 1 -- 172.21.15.87:0/1331828608 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5480017bd0 con 0x7f5484101140 2024-06-06T07:54:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.319+0000 7f547a7fc700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 0x7f54700a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.319+0000 7f5488d2d700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 0x7f54700a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.319+0000 7f547a7fc700 1 -- 172.21.15.87:0/1331828608 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54800cfa80 con 0x7f5484101140 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5488d2d700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 0x7f54700a9780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5474000c00 tx=0x7f5474005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 Processor -- start 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 -- start start 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f3060e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f3060f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559f305010f0 con 0x559f30614590 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f3055f9e0 con 0x559f30614eb0 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5496015b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f304c75f0 con 0x559f3060ece0 2024-06-06T07:54:57.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60972/0 (socket says 172.21.15.87:60972) 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f548952e700 1 -- 172.21.15.87:0/4281202795 learned_addr learned my addr 172.21.15.87:0/4281202795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.320+0000 7f5488d2d700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f3060e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f5489d2f700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f3060f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 -- 172.21.15.87:0/4281202795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 msgr2=0x559f3060f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f3060f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 -- 172.21.15.87:0/4281202795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 msgr2=0x559f3060e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f3060e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 -- 172.21.15.87:0/4281202795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f304aea60 con 0x559f30614590 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f5489d2f700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f3060f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f548952e700 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f54800985a0 tx=0x7f548009b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5021895d52fde652 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f546f7fe700 1 -- 172.21.15.87:0/4281202795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548009bbd0 con 0x559f30614590 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f546f7fe700 1 -- 172.21.15.87:0/4281202795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548009ff80 con 0x559f30614590 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.321+0000 7f546f7fe700 1 -- 172.21.15.87:0/4281202795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800a43b0 con 0x559f30614590 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- 172.21.15.87:0/4281202795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 msgr2=0x559f30614970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f54800985a0 tx=0x7f548009b830 comp rx=0 tx=0).stop 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- 172.21.15.87:0/4281202795 shutdown_connections 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f3060f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30614970 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 --2- 172.21.15.87:0/4281202795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f3060e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- 172.21.15.87:0/4281202795 >> 172.21.15.87:0/4281202795 conn(0x559f3053b210 msgr2=0x559f30609a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- 172.21.15.87:0/4281202795 shutdown_connections 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- 172.21.15.87:0/4281202795 wait complete. 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 -- start start 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.322+0000 7f5496015b80 1 Processor -- start 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30617ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f30614eb0 0x559f30618430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559f30500150 con 0x559f3060ece0 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559f3055b9a0 con 0x559f30614590 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f3061b740 con 0x559f30614eb0 2024-06-06T07:54:57.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5496015b80 10 client.0 _ll_register_callbacks cb 0x559f304f5740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60984/0 (socket says 172.21.15.87:60984) 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 -- 172.21.15.87:0/2070543507 learned_addr learned my addr 172.21.15.87:0/2070543507 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5488d2d700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30617ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f5489d2f700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f30614eb0 0x559f30618430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 -- 172.21.15.87:0/2070543507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f30614eb0 msgr2=0x559f30618430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f30614eb0 0x559f30618430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 -- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 msgr2=0x559f30617ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30617ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.323+0000 7f548952e700 1 -- 172.21.15.87:0/2070543507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f305258e0 con 0x559f3060ece0 2024-06-06T07:54:57.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f548952e700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f548009bb60 tx=0x7f548009c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cee12c9b07138e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548007e020 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54800a45c0 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548009ff80 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54800ba050 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 10 client.16683 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f3062e6a0 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548009ff80 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f548000b040 con 0x559f3060ece0 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 10 client.16683 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 10 client.16683 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 20 client.16683 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f30630b70 0x559f30632f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 4 client.16683 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.324+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f3056cee0 con 0x559f30630b70 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.325+0000 7f5488d2d700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f30630b70 0x559f30632f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.325+0000 7f546cff9700 10 client.16683 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:57.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.325+0000 7f5488d2d700 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f30630b70 0x559f30632f60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2863be22b75afd2 server_cookie=6187356fe16f2d03 in_seq=0 out_seq=0 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f546cff9700 1 -- 172.21.15.87:0/2070543507 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f3056cee0 con 0x559f30630b70 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f546cff9700 10 client.16683 handle_command_reply: tid=0 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 client.16683 shutdown 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 msgr2=0x559f3061abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f548009bb60 tx=0x7f548009c0e0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 shutdown_connections 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f30630b70 0x559f30632f60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f30614eb0 0x559f30618430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f3060ece0 0x559f3061abf0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f546cff9700 0 client.16683 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 --2- 172.21.15.87:0/2070543507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30617ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 >> 172.21.15.87:0/2070543507 conn(0x559f3053b210 msgr2=0x559f30608c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 shutdown_connections 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 1 -- 172.21.15.87:0/2070543507 wait complete. 2024-06-06T07:54:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.326+0000 7f5496015b80 20 client.16683 trim_cache size 0 max 16384 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 Processor -- start 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 -- start start 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f306093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54841aec20 con 0x559f30614eb0 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54841ac4b0 con 0x559f30614590 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.348+0000 7f5496015b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54841aa300 con 0x559f3060ece0 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5489d2f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5489d2f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60988/0 (socket says 172.21.15.87:60988) 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:57662/0 (socket says 172.21.15.87:57662) 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5489d2f700 1 -- 172.21.15.87:0/215950939 learned_addr learned my addr 172.21.15.87:0/215950939 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f548952e700 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f306093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 -- 172.21.15.87:0/215950939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 msgr2=0x559f306093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f306093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 -- 172.21.15.87:0/215950939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 msgr2=0x559f30635f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 -- 172.21.15.87:0/215950939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f30616480 con 0x559f30614590 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5489d2f700 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f5488d2d700 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54740098e0 tx=0x7f547401a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2e86e6927b370a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f546effd700 1 -- 172.21.15.87:0/215950939 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740050c0 con 0x559f30614590 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f546effd700 1 -- 172.21.15.87:0/215950939 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547400b790 con 0x559f30614590 2024-06-06T07:54:57.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.349+0000 7f546effd700 1 -- 172.21.15.87:0/215950939 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740236c0 con 0x559f30614590 2024-06-06T07:54:57.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- 172.21.15.87:0/215950939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 msgr2=0x559f30609900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54740098e0 tx=0x7f547401a070 comp rx=0 tx=0).stop 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- 172.21.15.87:0/215950939 shutdown_connections 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614eb0 0x559f30635f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f306093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 --2- 172.21.15.87:0/215950939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614590 0x559f30609900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- 172.21.15.87:0/215950939 >> 172.21.15.87:0/215950939 conn(0x559f3053b210 msgr2=0x559f30619fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- 172.21.15.87:0/215950939 shutdown_connections 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- 172.21.15.87:0/215950939 wait complete. 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 -- start start 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.350+0000 7f5496015b80 1 Processor -- start 2024-06-06T07:54:57.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30636500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f306345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54841aa970 con 0x559f30614590 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54841abde0 con 0x559f30614eb0 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54841ab040 con 0x559f3060ece0 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5496015b80 10 client.0 _ll_register_callbacks cb 0x559f304f5740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5488d2d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30636500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58466/0 (socket says 172.21.15.87:58466) 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5488d2d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30636500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32772/0 (socket says 172.21.15.87:32772) 2024-06-06T07:54:57.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 -- 172.21.15.87:0/612809682 learned_addr learned my addr 172.21.15.87:0/612809682 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f5489d2f700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f306345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 -- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 msgr2=0x559f306345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f306345e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 -- 172.21.15.87:0/612809682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 msgr2=0x559f30636500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30636500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.351+0000 7f548952e700 1 -- 172.21.15.87:0/612809682 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f30634c90 con 0x559f3060ece0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f548952e700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5480099e80 tx=0x7f548009c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546d40d1cfc18fe7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548007e040 con 0x559f3060ece0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9495+0+0 (secure 0 0 0) 0x7f54800a1250 con 0x559f3060ece0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547400b790 con 0x559f3060ece0 2024-06-06T07:54:57.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f548000d240 con 0x559f3060ece0 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 10 client.25990 fetch_fsmap learned FSMap version 32 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f30634c90 con 0x559f3060ece0 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547400b790 con 0x559f3060ece0 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f548009b020 con 0x559f3060ece0 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 10 client.25990 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 20 client.25990 populate_metadata read hostname 'smithi087' 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f3069c2f0 0x559f3069e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 4 client.25990 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.6, "await_for": 45.0}] 2024-06-06T07:54:57.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.352+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.6, "await_for": 45.0}) -- 0x559f30567460 con 0x559f3069c2f0 2024-06-06T07:54:57.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.353+0000 7f5488d2d700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f3069c2f0 0x559f3069e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:54:57.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.353+0000 7f546cff9700 10 client.25990 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:57.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.353+0000 7f5488d2d700 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f3069c2f0 0x559f3069e6e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3ec51abf059ea8 server_cookie=1b440b93b39a3e0 in_seq=0 out_seq=0 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.471+0000 7f546cff9700 1 -- 172.21.15.87:0/612809682 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f30567460 con 0x559f3069c2f0 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.471+0000 7f546cff9700 10 client.25990 handle_command_reply: tid=0 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.471+0000 7f5496015b80 1 client.25990 shutdown 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 msgr2=0x559f30635fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5480099e80 tx=0x7f548009c040 comp rx=0 tx=0).stop 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 shutdown_connections 2024-06-06T07:54:57.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559f3069c2f0 0x559f3069e6e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559f30614eb0 0x559f306345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f3060ece0 0x559f30635fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f546cff9700 0 client.25990 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 --2- 172.21.15.87:0/612809682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559f30614590 0x559f30636500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 >> 172.21.15.87:0/612809682 conn(0x559f3053b210 msgr2=0x559f30613050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 shutdown_connections 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 1 -- 172.21.15.87:0/612809682 wait complete. 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.472+0000 7f5496015b80 20 client.25990 trim_cache size 0 max 16384 2024-06-06T07:54:57.473 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 18, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "cf783428": { 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 18, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:54:57.474 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 132.6, 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:54:57.475 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:54:57.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 msgr2=0x7f54700a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 0x7f54700a9780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5474000c00 tx=0x7f5474005fb0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 msgr2=0x7f548419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:54:57.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f548000fec0 tx=0x7f548000d6c0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 shutdown_connections 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5484105e80 0x7f5484196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f54700a72c0 0x7f54700a9780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5484101140 0x7f548419c670 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 --2- 172.21.15.87:0/1331828608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5484101af0 0x7f548419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 >> 172.21.15.87:0/1331828608 conn(0x7f5484076630 msgr2=0x7f54840fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 shutdown_connections 2024-06-06T07:54:57.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:54:57.473+0000 7f548b792700 1 -- 172.21.15.87:0/1331828608 wait complete. 2024-06-06T07:54:57.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.51 MB/sec execute 181 sec latency 296.765 ms 2024-06-06T07:54:58.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:57 smithi195 ceph-mon[37040]: pgmap v211: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 447 op/s 2024-06-06T07:54:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:57 smithi181 ceph-mon[36790]: pgmap v211: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 447 op/s 2024-06-06T07:54:58.331 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:57 smithi087 ceph-mon[32276]: pgmap v211: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 447 op/s 2024-06-06T07:54:58.366 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cf783428', quiesce took 6.6 seconds. Will release after: 6.0 2024-06-06T07:54:58.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.47 MB/sec execute 182 sec latency 1296.796 ms 2024-06-06T07:54:59.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.43 MB/sec execute 183 sec latency 2296.835 ms 2024-06-06T07:55:00.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:54:59 smithi195 ceph-mon[37040]: pgmap v212: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:54:59 smithi181 ceph-mon[36790]: pgmap v212: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:54:59 smithi087 ceph-mon[32276]: pgmap v212: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:00.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.39 MB/sec execute 184 sec latency 3296.878 ms 2024-06-06T07:55:01.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.35 MB/sec execute 185 sec latency 4296.974 ms 2024-06-06T07:55:02.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:01 smithi195 ceph-mon[37040]: pgmap v213: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:01 smithi181 ceph-mon[36790]: pgmap v213: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:02.403 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:01 smithi087 ceph-mon[32276]: pgmap v213: 129 pgs: 129 active+clean; 58 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-06-06T07:55:02.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.31 MB/sec execute 186 sec latency 5297.043 ms 2024-06-06T07:55:03.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.27 MB/sec execute 187 sec latency 6297.132 ms 2024-06-06T07:55:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:04 smithi195 ceph-mon[37040]: pgmap v214: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-06-06T07:55:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:04 smithi181 ceph-mon[36790]: pgmap v214: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-06-06T07:55:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:04 smithi087 ceph-mon[32276]: pgmap v214: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-06-06T07:55:04.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:55:04.618 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:55:04.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.23 MB/sec execute 188 sec latency 7297.202 ms 2024-06-06T07:55:05.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.19 MB/sec execute 189 sec latency 8297.306 ms 2024-06-06T07:55:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:06 smithi195 ceph-mon[37040]: pgmap v215: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-06-06T07:55:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 -- 172.21.15.87:0/2471699518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c10d2f0 msgr2=0x7f388c114ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 --2- 172.21.15.87:0/2471699518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c10d2f0 0x7f388c114ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3884009be0 tx=0x7f388402a6e0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 -- 172.21.15.87:0/2471699518 shutdown_connections 2024-06-06T07:55:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 --2- 172.21.15.87:0/2471699518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c10d2f0 0x7f388c114ec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 --2- 172.21.15.87:0/2471699518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c108bd0 0x7f388c10cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 --2- 172.21.15.87:0/2471699518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.275+0000 7f3893a93640 1 -- 172.21.15.87:0/2471699518 >> 172.21.15.87:0/2471699518 conn(0x7f388c075f20 msgr2=0x7f388c078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.276+0000 7f3893a93640 1 -- 172.21.15.87:0/2471699518 shutdown_connections 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.276+0000 7f3893a93640 1 -- 172.21.15.87:0/2471699518 wait complete. 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 Processor -- start 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 -- start start 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c10d2f0 0x7f388c1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 0x7f388c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f388c118400 con 0x7f388c108220 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f388c118280 con 0x7f388c19e6a0 2024-06-06T07:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.277+0000 7f3893a93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f388c118580 con 0x7f388c10d2f0 2024-06-06T07:55:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.278+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.278+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32788/0 (socket says 172.21.15.87:32788) 2024-06-06T07:55:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.278+0000 7f389182f640 1 -- 172.21.15.87:0/287905395 learned_addr learned my addr 172.21.15.87:0/287905395 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.278+0000 7f389102e640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c10d2f0 0x7f388c1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.278+0000 7f3892030640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 0x7f388c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.279+0000 7f389182f640 1 -- 172.21.15.87:0/287905395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c10d2f0 msgr2=0x7f388c1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.279+0000 7f389182f640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c10d2f0 0x7f388c1a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.280+0000 7f389182f640 1 -- 172.21.15.87:0/287905395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 msgr2=0x7f388c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.280+0000 7f389182f640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 0x7f388c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.280+0000 7f389182f640 1 -- 172.21.15.87:0/287905395 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c19f2a0 con 0x7f388c108220 2024-06-06T07:55:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.280+0000 7f389182f640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f388800e9c0 tx=0x7f388800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.280+0000 7f3892030640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 0x7f388c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.281+0000 7f3882ffd640 1 -- 172.21.15.87:0/287905395 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388800cde0 con 0x7f388c108220 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.281+0000 7f3882ffd640 1 -- 172.21.15.87:0/287905395 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38880035f0 con 0x7f388c108220 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.281+0000 7f3882ffd640 1 -- 172.21.15.87:0/287905395 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888010690 con 0x7f388c108220 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.281+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1ab380 con 0x7f388c108220 2024-06-06T07:55:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.281+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1ab730 con 0x7f388c108220 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 Processor -- start 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 -- start start 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5592430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c55b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c3c54f64f0 con 0x55c3c5592970 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3c5494870 con 0x55c3c5592050 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.285+0000 7f389679d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3c54f24b0 con 0x55c3c55ae360 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.286+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.287+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32790/0 (socket says 172.21.15.87:32790) 2024-06-06T07:55:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.287+0000 7f389182f640 1 -- 172.21.15.87:0/617597812 learned_addr learned my addr 172.21.15.87:0/617597812 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f3882ffd640 1 -- 172.21.15.87:0/287905395 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3888004020 con 0x7f388c108220 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f389182f640 1 -- 172.21.15.87:0/617597812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 msgr2=0x55c3c55b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f389182f640 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c55b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f389182f640 1 -- 172.21.15.87:0/617597812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 msgr2=0x55c3c5592430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f389182f640 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5592430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.289+0000 7f389182f640 1 -- 172.21.15.87:0/617597812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c54b09b0 con 0x55c3c5592970 2024-06-06T07:55:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.290+0000 7f3882ffd640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 0x7f38780a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.290+0000 7f3882ffd640 1 -- 172.21.15.87:0/287905395 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f38880f70f0 con 0x7f388c108220 2024-06-06T07:55:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.292+0000 7f3892030640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 0x7f38780a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.292+0000 7f389182f640 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f38880f43c0 tx=0x7f38880f85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e261f887929c099 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.292+0000 7f3877fff640 1 -- 172.21.15.87:0/617597812 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880bf3a0 con 0x55c3c5592970 2024-06-06T07:55:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.292+0000 7f3877fff640 1 -- 172.21.15.87:0/617597812 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38880c3890 con 0x55c3c5592970 2024-06-06T07:55:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.293+0000 7f3877fff640 1 -- 172.21.15.87:0/617597812 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880bf3a0 con 0x55c3c5592970 2024-06-06T07:55:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.293+0000 7f389679d740 1 -- 172.21.15.87:0/617597812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 msgr2=0x55c3c55ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.293+0000 7f389679d740 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f38880f43c0 tx=0x7f38880f85a0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.293+0000 7f3892030640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 0x7f38780a5410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3884004af0 tx=0x7f3884005bd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.294+0000 7f389679d740 1 -- 172.21.15.87:0/617597812 shutdown_connections 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.294+0000 7f389679d740 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c55b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.294+0000 7f389679d740 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592970 0x55c3c55ade20 secure :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f38880f43c0 tx=0x7f38880f85a0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.294+0000 7f389679d740 1 --2- 172.21.15.87:0/617597812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5592430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.294+0000 7f389679d740 1 -- 172.21.15.87:0/617597812 >> 172.21.15.87:0/617597812 conn(0x55c3c558e680 msgr2=0x55c3c558ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.296+0000 7f389679d740 1 -- 172.21.15.87:0/617597812 shutdown_connections 2024-06-06T07:55:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.297+0000 7f389679d740 1 -- 172.21.15.87:0/617597812 wait complete. 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.297+0000 7f389679d740 1 -- start start 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.297+0000 7f389679d740 1 Processor -- start 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ae360 0x55c3c55ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c55ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c3c5594fe0 con 0x55c3c5592050 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3c547fae0 con 0x55c3c55ae360 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389679d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3c5595160 con 0x55c3c55ad340 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.300+0000 7f389679d740 10 client.0 _ll_register_callbacks cb 0x55c3c54b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.298+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.300+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32794/0 (socket says 172.21.15.87:32794) 2024-06-06T07:55:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.300+0000 7f389182f640 1 -- 172.21.15.87:0/3591234606 learned_addr learned my addr 172.21.15.87:0/3591234606 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.300+0000 7f3892030640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c55ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.300+0000 7f389102e640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ae360 0x55c3c55ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 -- 172.21.15.87:0/3591234606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 msgr2=0x55c3c55ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c55ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 -- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ae360 msgr2=0x55c3c55ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ae360 0x55c3c55ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 -- 172.21.15.87:0/3591234606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c520a210 con 0x55c3c5592050 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f389182f640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f38880f49c0 tx=0x7f38880f4870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11861ae5b0878a77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.301+0000 7f38757fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.302+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880c12d0 con 0x55c3c5592050 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.302+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38880c3890 con 0x55c3c5592050 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.302+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f38880e30d0 con 0x55c3c5592050 2024-06-06T07:55:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.302+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38880c2af0 con 0x55c3c5592050 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 10 client.16701 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c3c52d6620 con 0x55c3c5592050 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38880c3890 con 0x55c3c5592050 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f38880c7160 con 0x55c3c5592050 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 10 client.16701 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 20 client.16701 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c559d5c0 0x55c3c559fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 4 client.16701 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.303+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c5509f10 con 0x55c3c559d5c0 2024-06-06T07:55:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.304+0000 7f389102e640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c559d5c0 0x55c3c559fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:06 smithi181 ceph-mon[36790]: pgmap v215: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-06-06T07:55:06.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.309+0000 7f389102e640 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c559d5c0 0x55c3c559fa20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff4e6e00ef1076c server_cookie=a32ba1c4fe674682 in_seq=0 out_seq=0 2024-06-06T07:55:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.310+0000 7f38757fa640 10 client.16701 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.312+0000 7f38757fa640 1 -- 172.21.15.87:0/3591234606 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3c5509f10 con 0x55c3c559d5c0 2024-06-06T07:55:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.312+0000 7f38757fa640 10 client.16701 handle_command_reply: tid=0 2024-06-06T07:55:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.313+0000 7f389679d740 1 client.16701 shutdown 2024-06-06T07:55:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.317+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 msgr2=0x55c3c55abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.317+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f38880f49c0 tx=0x7f38880f4870 comp rx=0 tx=0).stop 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 shutdown_connections 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c559d5c0 0x55c3c559fa20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c55ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ae360 0x55c3c55ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 --2- 172.21.15.87:0/3591234606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55abde0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 >> 172.21.15.87:0/3591234606 conn(0x55c3c558e680 msgr2=0x55c3c5594490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f38757fa640 0 client.16701 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 shutdown_connections 2024-06-06T07:55:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 1 -- 172.21.15.87:0/3591234606 wait complete. 2024-06-06T07:55:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.321+0000 7f389679d740 20 client.16701 trim_cache size 0 max 16384 2024-06-06T07:55:06.326 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:06 smithi087 ceph-mon[32276]: pgmap v215: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-06-06T07:55:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.355+0000 7f389679d740 1 Processor -- start 2024-06-06T07:55:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.355+0000 7f389679d740 1 -- start start 2024-06-06T07:55:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5595d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c5596280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3870012bc0 con 0x55c3c55ae360 2024-06-06T07:55:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3870010490 con 0x55c3c5592050 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.356+0000 7f389679d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f387000e300 con 0x55c3c55ad340 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32806/0 (socket says 172.21.15.87:32806) 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 -- 172.21.15.87:0/728637519 learned_addr learned my addr 172.21.15.87:0/728637519 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 -- 172.21.15.87:0/728637519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 msgr2=0x55c3c5596280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c5596280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 -- 172.21.15.87:0/728637519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 msgr2=0x55c3c5595d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:55:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5595d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 -- 172.21.15.87:0/728637519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c558e4b0 con 0x55c3c55ae360 2024-06-06T07:55:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.357+0000 7f3892030640 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f388402acd0 tx=0x7f388403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d4a7157ef9ae6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.360+0000 7f38777fe640 1 -- 172.21.15.87:0/728637519 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388404e070 con 0x55c3c55ae360 2024-06-06T07:55:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.360+0000 7f38777fe640 1 -- 172.21.15.87:0/728637519 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884004520 con 0x55c3c55ae360 2024-06-06T07:55:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.362+0000 7f38777fe640 1 -- 172.21.15.87:0/728637519 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884036040 con 0x55c3c55ae360 2024-06-06T07:55:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.363+0000 7f389679d740 1 -- 172.21.15.87:0/728637519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 msgr2=0x55c3c55993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.363+0000 7f389679d740 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f388402acd0 tx=0x7f388403d000 comp rx=0 tx=0).stop 2024-06-06T07:55:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.364+0000 7f389679d740 1 -- 172.21.15.87:0/728637519 shutdown_connections 2024-06-06T07:55:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.364+0000 7f389679d740 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c55ae360 0x55c3c55993d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.364+0000 7f389679d740 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ad340 0x55c3c5596280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.364+0000 7f389679d740 1 --2- 172.21.15.87:0/728637519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c5592050 0x55c3c5595d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.364+0000 7f389679d740 1 -- 172.21.15.87:0/728637519 >> 172.21.15.87:0/728637519 conn(0x55c3c558e680 msgr2=0x55c3c558ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.367+0000 7f389679d740 1 -- 172.21.15.87:0/728637519 shutdown_connections 2024-06-06T07:55:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.370+0000 7f389679d740 1 -- 172.21.15.87:0/728637519 wait complete. 2024-06-06T07:55:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.371+0000 7f389679d740 1 -- start start 2024-06-06T07:55:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.371+0000 7f389679d740 1 Processor -- start 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c56380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f387000e950 con 0x55c3c5592050 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f387000fdc0 con 0x55c3c55ad340 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f387000f020 con 0x55c3c55ae360 2024-06-06T07:55:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389679d740 10 client.0 _ll_register_callbacks cb 0x7f38880f9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389102e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389182f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389102e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:57724/0 (socket says 172.21.15.87:57724) 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389102e640 1 -- 172.21.15.87:0/2321383440 learned_addr learned my addr 172.21.15.87:0/2321383440 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f389182f640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32812/0 (socket says 172.21.15.87:32812) 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.372+0000 7f3892030640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c56380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 -- 172.21.15.87:0/2321383440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 msgr2=0x55c3c56380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c56380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 -- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 msgr2=0x55c3c55b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 -- 172.21.15.87:0/2321383440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c5638730 con 0x55c3c5592050 2024-06-06T07:55:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389102e640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.373+0000 7f389182f640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f38880cb8d0 tx=0x7f38880f8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a43b8d33f4646c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.374+0000 7f38757fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.374+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880c2cd0 con 0x55c3c5592050 2024-06-06T07:55:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.374+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f38880c25e0 con 0x55c3c5592050 2024-06-06T07:55:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.375+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884004520 con 0x55c3c5592050 2024-06-06T07:55:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.375+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38880c2f60 con 0x55c3c5592050 2024-06-06T07:55:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.375+0000 7f389679d740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.375+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c3c5639a40 con 0x55c3c5592050 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884004520 con 0x55c3c5592050 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f38880a8020 con 0x55c3c5592050 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 20 client.16707 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c5639e60 0x55c3c563c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 4 client.16707 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:55:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.377+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3c5503c20 con 0x55c3c5639e60 2024-06-06T07:55:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.379+0000 7f389102e640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c5639e60 0x55c3c563c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.384+0000 7f38757fa640 10 client.16707 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.384+0000 7f389102e640 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c5639e60 0x55c3c563c280 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8842f7ef283d4f66 server_cookie=a2a90d6c06bed942 in_seq=0 out_seq=0 2024-06-06T07:55:06.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.517+0000 7f9c489e1700 1 -- 172.21.15.87:0/2695975800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110ff0 msgr2=0x7f9c441099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.517+0000 7f9c489e1700 1 --2- 172.21.15.87:0/2695975800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110ff0 0x7f9c441099d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009990 tx=0x7f9c38009ca0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.526+0000 7f9c489e1700 1 -- 172.21.15.87:0/2695975800 shutdown_connections 2024-06-06T07:55:06.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.526+0000 7f9c489e1700 1 --2- 172.21.15.87:0/2695975800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4411bc70 0x7f9c4411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.526+0000 7f9c489e1700 1 --2- 172.21.15.87:0/2695975800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c44110b80 0x7f9c44109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.526+0000 7f9c489e1700 1 --2- 172.21.15.87:0/2695975800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110ff0 0x7f9c441099d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.526+0000 7f9c489e1700 1 -- 172.21.15.87:0/2695975800 >> 172.21.15.87:0/2695975800 conn(0x7f9c4406d470 msgr2=0x7f9c4406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.529+0000 7f9c489e1700 1 -- 172.21.15.87:0/2695975800 shutdown_connections 2024-06-06T07:55:06.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.530+0000 7f9c489e1700 1 -- 172.21.15.87:0/2695975800 wait complete. 2024-06-06T07:55:06.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 Processor -- start 2024-06-06T07:55:06.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 -- start start 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110b80 0x7f9c441ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 0x7f9c441adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 0x7f9c441a7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441215c0 con 0x7f9c44110b80 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44121440 con 0x7f9c44110ff0 2024-06-06T07:55:06.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.531+0000 7f9c489e1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44121740 con 0x7f9c4411bc70 2024-06-06T07:55:06.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.535+0000 7f9c42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 0x7f9c441adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.536+0000 7f9c42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 0x7f9c441adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:57738/0 (socket says 172.21.15.87:57738) 2024-06-06T07:55:06.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.536+0000 7f9c42ffd700 1 -- 172.21.15.87:0/1026071597 learned_addr learned my addr 172.21.15.87:0/1026071597 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:06.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.537+0000 7f9c437fe700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110b80 0x7f9c441ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.537+0000 7f9c43fff700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 0x7f9c441a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 -- 172.21.15.87:0/1026071597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 msgr2=0x7f9c441adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 0x7f9c441adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 -- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110b80 msgr2=0x7f9c441ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110b80 0x7f9c441ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 -- 172.21.15.87:0/1026071597 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441a7fe0 con 0x7f9c4411bc70 2024-06-06T07:55:06.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c43fff700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 0x7f9c441a7880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00e8d0 tx=0x7f9c3c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.538+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c004220 con 0x7f9c4411bc70 2024-06-06T07:55:06.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.539+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441a82d0 con 0x7f9c4411bc70 2024-06-06T07:55:06.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.539+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c44111450 con 0x7f9c4411bc70 2024-06-06T07:55:06.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.539+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c3c0043c0 con 0x7f9c4411bc70 2024-06-06T07:55:06.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.539+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c010430 con 0x7f9c4411bc70 2024-06-06T07:55:06.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.544+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c30005330 con 0x7f9c4411bc70 2024-06-06T07:55:06.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.544+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9c3c00c810 con 0x7f9c4411bc70 2024-06-06T07:55:06.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.545+0000 7f9c40ff9700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 0x7f9c2c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:06.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.545+0000 7f9c437fe700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 0x7f9c2c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:06.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.546+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f9c3c0cf8b0 con 0x7f9c4411bc70 2024-06-06T07:55:06.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.549+0000 7f9c437fe700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 0x7f9c2c0a53d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c441a89c0 tx=0x7f9c3400b040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:06.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:06.554+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c3c09bff0 con 0x7f9c4411bc70 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.558+0000 7f38757fa640 1 -- 172.21.15.87:0/2321383440 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3c5503c20 con 0x55c3c5639e60 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.558+0000 7f38757fa640 10 client.16707 handle_command_reply: tid=0 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.558+0000 7f389679d740 1 client.16707 shutdown 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 msgr2=0x55c3c55b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f38880cb8d0 tx=0x7f38880f8b20 comp rx=0 tx=0).stop 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 shutdown_connections 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3c5639e60 0x55c3c563c280 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3c55ae360 0x55c3c56380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3c55ad340 0x55c3c55b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 --2- 172.21.15.87:0/2321383440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3c5592050 0x55c3c55b5330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 >> 172.21.15.87:0/2321383440 conn(0x55c3c558e680 msgr2=0x55c3c5595d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f38757fa640 0 client.16707 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 shutdown_connections 2024-06-06T07:55:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.559+0000 7f389679d740 1 -- 172.21.15.87:0/2321383440 wait complete. 2024-06-06T07:55:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.560+0000 7f389679d740 20 client.16707 trim_cache size 0 max 16384 2024-06-06T07:55:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:55:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:55:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.560+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 msgr2=0x7f38780a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.560+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 0x7f38780a5410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3884004af0 tx=0x7f3884005bd0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.560+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 msgr2=0x7f388c1a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.560+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f388800e9c0 tx=0x7f388800ee80 comp rx=0 tx=0).stop 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 shutdown_connections 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f38780a2f50 0x7f38780a5410 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c19e6a0 0x7f388c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388c10d2f0 0x7f388c1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 --2- 172.21.15.87:0/287905395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f388c108220 0x7f388c1a45e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 >> 172.21.15.87:0/287905395 conn(0x7f388c075f20 msgr2=0x7f388c078250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 shutdown_connections 2024-06-06T07:55:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:06.561+0000 7f3893a93640 1 -- 172.21.15.87:0/287905395 wait complete. 2024-06-06T07:55:06.667 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.15 MB/sec execute 190 sec latency 9297.358 ms 2024-06-06T07:55:07.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.053+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c300058f0 con 0x7f9c4411bc70 2024-06-06T07:55:07.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.057+0000 7f9c40ff9700 1 -- 172.21.15.87:0/1026071597 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7f9c3c09b8f0 con 0x7f9c4411bc70 2024-06-06T07:55:07.058 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:55:07.058 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:55:07.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.061+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 msgr2=0x7f9c2c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:07.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.061+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 0x7f9c2c0a53d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c441a89c0 tx=0x7f9c3400b040 comp rx=0 tx=0).stop 2024-06-06T07:55:07.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.061+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 msgr2=0x7f9c441a7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:07.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.061+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 0x7f9c441a7880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00e8d0 tx=0x7f9c3c00ebe0 comp rx=0 tx=0).stop 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 shutdown_connections 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9c2c0a2f10 0x7f9c2c0a53d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4411bc70 0x7f9c441a7880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c44110b80 0x7f9c441ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 --2- 172.21.15.87:0/1026071597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c44110ff0 0x7f9c441adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:07.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 >> 172.21.15.87:0/1026071597 conn(0x7f9c4406d470 msgr2=0x7f9c4410e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:07.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 shutdown_connections 2024-06-06T07:55:07.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:07.062+0000 7f9c489e1700 1 -- 172.21.15.87:0/1026071597 wait complete. 2024-06-06T07:55:07.066 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:55:07.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:07 smithi195 ceph-mon[37040]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:55:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:07 smithi181 ceph-mon[36790]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:55:07.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:07 smithi087 ceph-mon[32276]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:55:07.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.12 MB/sec execute 191 sec latency 10297.396 ms 2024-06-06T07:55:07.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id cf783428 --release --await' 2024-06-06T07:55:07.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id cf783428 --release --await 2024-06-06T07:55:08.010 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:55:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:08 smithi181 ceph-mon[36790]: pgmap v217: 129 pgs: 19 active+clean+snaptrim, 63 active+clean, 47 active+clean+snaptrim_wait; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-06-06T07:55:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:08 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1026071597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:08 smithi087 ceph-mon[32276]: pgmap v217: 129 pgs: 19 active+clean+snaptrim, 63 active+clean, 47 active+clean+snaptrim_wait; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-06-06T07:55:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:08 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1026071597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:08 smithi195 ceph-mon[37040]: pgmap v217: 129 pgs: 19 active+clean+snaptrim, 63 active+clean, 47 active+clean+snaptrim_wait; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-06-06T07:55:08.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:08 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1026071597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:08.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.08 MB/sec execute 192 sec latency 11297.482 ms 2024-06-06T07:55:09.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.04 MB/sec execute 193 sec latency 12297.527 ms 2024-06-06T07:55:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:10 smithi087 ceph-mon[32276]: pgmap v218: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:10 smithi181 ceph-mon[36790]: pgmap v218: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:10 smithi195 ceph-mon[37040]: pgmap v218: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:10.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 376350 7.01 MB/sec execute 194 sec latency 13297.580 ms 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 -- 172.21.15.87:0/3045213007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c108500 msgr2=0x7f4e7c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/3045213007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c108500 0x7f4e7c1088e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b0a0 tx=0x7f4e7800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 -- 172.21.15.87:0/3045213007 shutdown_connections 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/3045213007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/3045213007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c108500 0x7f4e7c1088e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/3045213007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c06b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 -- 172.21.15.87:0/3045213007 >> 172.21.15.87:0/3045213007 conn(0x7f4e7c078360 msgr2=0x7f4e7c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 -- 172.21.15.87:0/3045213007 shutdown_connections 2024-06-06T07:55:10.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.704+0000 7f4e82dc4700 1 -- 172.21.15.87:0/3045213007 wait complete. 2024-06-06T07:55:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 Processor -- start 2024-06-06T07:55:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 -- start start 2024-06-06T07:55:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 0x7f4e7c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1127e0 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c112660 con 0x7f4e7c06abe0 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e82dc4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c112960 con 0x7f4e7c108500 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32780/0 (socket says 172.21.15.87:32780) 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 -- 172.21.15.87:0/4284406908 learned_addr learned my addr 172.21.15.87:0/4284406908 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e81361700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 0x7f4e7c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e80b60700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 -- 172.21.15.87:0/4284406908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 msgr2=0x7f4e7c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 0x7f4e7c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 -- 172.21.15.87:0/4284406908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 msgr2=0x7f4e7c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.705+0000 7f4e73fff700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e73fff700 1 -- 172.21.15.87:0/4284406908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c199240 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e81361700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 0x7f4e7c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e80b60700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e73fff700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b530 tx=0x7f4e6c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e71ffb700 1 -- 172.21.15.87:0/4284406908 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c015020 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c199530 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e71ffb700 1 -- 172.21.15.87:0/4284406908 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c011d10 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e71ffb700 1 -- 172.21.15.87:0/4284406908 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c010e50 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.706+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a57f0 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.708+0000 7f4e71ffb700 1 -- 172.21.15.87:0/4284406908 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4e6c018020 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.709+0000 7f4e71ffb700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 0x7f4e640a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.709+0000 7f4e71ffb700 1 -- 172.21.15.87:0/4284406908 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f4e6c0cefe0 con 0x7f4e7c06b5a0 2024-06-06T07:55:10.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.709+0000 7f4e80b60700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 0x7f4e640a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e80b60700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 0x7f4e640a53f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b830 tx=0x7f4e78009470 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 Processor -- start 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 -- start start 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed98068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed9801070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558ed96f3060 con 0x558ed9806e20 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558ed9751950 con 0x558ed9806500 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ed96b9560 con 0x558ed9800c50 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32792/0 (socket says 172.21.15.87:32792) 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e73fff700 1 -- 172.21.15.87:0/2828008223 learned_addr learned my addr 172.21.15.87:0/2828008223 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e81361700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed9801070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.710+0000 7f4e80b60700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed98068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 -- 172.21.15.87:0/2828008223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 msgr2=0x558ed9801070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed9801070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 -- 172.21.15.87:0/2828008223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 msgr2=0x558ed98068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed98068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 -- 172.21.15.87:0/2828008223 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed96a09d0 con 0x558ed9806e20 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e80b60700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed98068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cc8b0 tx=0x7f4e6c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28ce84c979cff8a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e62ffd700 1 -- 172.21.15.87:0/2828008223 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09b190 con 0x558ed9806e20 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e62ffd700 1 -- 172.21.15.87:0/2828008223 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c09f540 con 0x558ed9806e20 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.711+0000 7f4e62ffd700 1 -- 172.21.15.87:0/2828008223 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09b3f0 con 0x558ed9806e20 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2828008223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 msgr2=0x558ed9800710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cc8b0 tx=0x7f4e6c004860 comp rx=0 tx=0).stop 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2828008223 shutdown_connections 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed98068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed9801070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2828008223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9806e20 0x558ed9800710 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2828008223 >> 172.21.15.87:0/2828008223 conn(0x558ed972d180 msgr2=0x558ed97fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2828008223 shutdown_connections 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2828008223 wait complete. 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 -- start start 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 Processor -- start 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.712+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed9809eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806e20 0x558ed980a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558ed96f20c0 con 0x558ed9800c50 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558ed974d910 con 0x558ed9806e20 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ed9811c10 con 0x558ed9806500 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 10 client.0 _ll_register_callbacks cb 0x558ed96e76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32796/0 (socket says 172.21.15.87:32796) 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 -- 172.21.15.87:0/2469615588 learned_addr learned my addr 172.21.15.87:0/2469615588 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed9809eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e81361700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806e20 0x558ed980a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 -- 172.21.15.87:0/2469615588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 msgr2=0x558ed9809eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed9809eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 -- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806e20 msgr2=0x558ed980a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806e20 0x558ed980a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 -- 172.21.15.87:0/2469615588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed9717850 con 0x558ed9800c50 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed9809eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:10.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e80b60700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b0a0 tx=0x7f4e780099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5112d94494a2350f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:10.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7802d030 con 0x558ed9800c50 2024-06-06T07:55:10.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f4e7801c650 con 0x558ed9800c50 2024-06-06T07:55:10.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.713+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6c09f540 con 0x558ed9800c50 2024-06-06T07:55:10.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e7801c850 con 0x558ed9800c50 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 10 client.16725 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ed981c330 con 0x558ed9800c50 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c09f540 con 0x558ed9800c50 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4e78011070 con 0x558ed9800c50 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 10 client.16725 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 20 client.16725 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed981e970 0x558ed9820d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 4 client.16725 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:10.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ed975ee50 con 0x558ed981e970 2024-06-06T07:55:10.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.714+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed981e970 0x558ed9820d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.715+0000 7f4e73fff700 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed981e970 0x558ed9820d60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e5716d555e88a2 server_cookie=1b8cdfed8d4fa110 in_seq=0 out_seq=0 2024-06-06T07:55:10.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.715+0000 7f4e5bfff700 10 client.16725 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e5bfff700 1 -- 172.21.15.87:0/2469615588 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ed975ee50 con 0x558ed981e970 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e5bfff700 10 client.16725 handle_command_reply: tid=0 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 client.16725 shutdown 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 msgr2=0x558ed980cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b0a0 tx=0x7f4e780099e0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 shutdown_connections 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed981e970 0x558ed9820d60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806e20 0x558ed980a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed9800c50 0x558ed980cbb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e5bfff700 0 client.16725 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/2469615588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed9809eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 >> 172.21.15.87:0/2469615588 conn(0x558ed972d180 msgr2=0x558ed97fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 shutdown_connections 2024-06-06T07:55:10.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 1 -- 172.21.15.87:0/2469615588 wait complete. 2024-06-06T07:55:10.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.716+0000 7f4e8d647b80 20 client.16725 trim_cache size 0 max 16384 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 Processor -- start 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 -- start start 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9800c50 0x558ed98078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed980c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1af050 con 0x558ed972d180 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ac8e0 con 0x558ed9800c50 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1aa750 con 0x558ed9806500 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32804/0 (socket says 172.21.15.87:32804) 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e80b60700 1 -- 172.21.15.87:0/3296354117 learned_addr learned my addr 172.21.15.87:0/3296354117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e73fff700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9800c50 0x558ed98078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.736+0000 7f4e81361700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed980c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 -- 172.21.15.87:0/3296354117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 msgr2=0x558ed980c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed980c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 -- 172.21.15.87:0/3296354117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9800c50 msgr2=0x558ed98078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9800c50 0x558ed98078b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 -- 172.21.15.87:0/3296354117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed980ae20 con 0x558ed972d180 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e81361700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed980c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e80b60700 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b000 tx=0x7f4e7800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83273f0bfe9daf8b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e627fc700 1 -- 172.21.15.87:0/3296354117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78040070 con 0x558ed972d180 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e627fc700 1 -- 172.21.15.87:0/3296354117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78005e60 con 0x558ed972d180 2024-06-06T07:55:10.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.737+0000 7f4e627fc700 1 -- 172.21.15.87:0/3296354117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7802d070 con 0x558ed972d180 2024-06-06T07:55:10.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- 172.21.15.87:0/3296354117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 msgr2=0x558ed9800710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b000 tx=0x7f4e7800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- 172.21.15.87:0/3296354117 shutdown_connections 2024-06-06T07:55:10.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9806500 0x558ed980c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9800710 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/3296354117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9800c50 0x558ed98078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- 172.21.15.87:0/3296354117 >> 172.21.15.87:0/3296354117 conn(0x558ed97fbcc0 msgr2=0x558ed97fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- 172.21.15.87:0/3296354117 shutdown_connections 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- 172.21.15.87:0/3296354117 wait complete. 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 -- start start 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.738+0000 7f4e8d647b80 1 Processor -- start 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed98175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed9813e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1aada0 con 0x558ed972d180 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ac210 con 0x558ed9806500 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ab470 con 0x558ed9800c50 2024-06-06T07:55:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 10 client.0 _ll_register_callbacks cb 0x558ed96e76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32808/0 (socket says 172.21.15.87:32808) 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 -- 172.21.15.87:0/811519672 learned_addr learned my addr 172.21.15.87:0/811519672 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e73fff700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed98175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e81361700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed9813e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 -- 172.21.15.87:0/811519672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 msgr2=0x558ed98175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed98175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 -- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 msgr2=0x558ed9813e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed9813e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 -- 172.21.15.87:0/811519672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed98146b0 con 0x558ed972d180 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e80b60700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4e7803e040 tx=0x7f4e780099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7112e4eb5d623e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78040030 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7f4e78005c60 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.739+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ed97593d0 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e7804f050 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 10 client.16731 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ed988a000 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558ed97593d0 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4e78011040 con 0x558ed972d180 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 10 client.16731 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 10 client.16731 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 20 client.16731 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed9806e20 0x558ed988c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 4 client.16731 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cf783428", "release": true, "await": true}] 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cf783428", "release": true, "await": true}) -- 0x558ed970d050 con 0x558ed9806e20 2024-06-06T07:55:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.740+0000 7f4e73fff700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed9806e20 0x558ed988c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:10.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.741+0000 7f4e73fff700 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed9806e20 0x558ed988c4d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5b5eb509e36067 server_cookie=e8d0e945b3c5bf89 in_seq=0 out_seq=0 2024-06-06T07:55:10.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.741+0000 7f4e5bfff700 10 client.16731 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e5bfff700 1 -- 172.21.15.87:0/811519672 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558ed970d050 con 0x558ed9806e20 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e5bfff700 10 client.16731 handle_command_reply: tid=0 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 client.16731 shutdown 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 msgr2=0x558ed9817080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4e7803e040 tx=0x7f4e780099e0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 shutdown_connections 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558ed9806e20 0x558ed988c4d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558ed9806500 0x558ed9813e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558ed972d180 0x558ed9817080 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 --2- 172.21.15.87:0/811519672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ed9800c50 0x558ed98175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 >> 172.21.15.87:0/811519672 conn(0x558ed97fbcc0 msgr2=0x558ed97fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.754+0000 7f4e5bfff700 0 client.16731 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 shutdown_connections 2024-06-06T07:55:10.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e8d647b80 1 -- 172.21.15.87:0/811519672 wait complete. 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e8d647b80 20 client.16731 trim_cache size 0 max 16384 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 20, 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "cf783428": { 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 20, 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:55:10.756 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 132.6, 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:10.757 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:55:10.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 msgr2=0x7f4e640a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 0x7f4e640a53f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b830 tx=0x7f4e78009470 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 msgr2=0x7f4e7c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b530 tx=0x7f4e6c00b8f0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 shutdown_connections 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e7c06abe0 0x7f4e7c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4e640a2f30 0x7f4e640a53f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e7c108500 0x7f4e7c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 --2- 172.21.15.87:0/4284406908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7c06b5a0 0x7f4e7c19ee60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 >> 172.21.15.87:0/4284406908 conn(0x7f4e7c078360 msgr2=0x7f4e7c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.755+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 shutdown_connections 2024-06-06T07:55:10.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:10.756+0000 7f4e82dc4700 1 -- 172.21.15.87:0/4284406908 wait complete. 2024-06-06T07:55:11.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cf783428', total seconds elapsed: 19.8 2024-06-06T07:55:11.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 377004 6.98 MB/sec execute 195 sec latency 13387.384 ms 2024-06-06T07:55:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:12 smithi181 ceph-mon[36790]: pgmap v219: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:12 smithi087 ceph-mon[32276]: pgmap v219: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:12.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:12 smithi195 ceph-mon[37040]: pgmap v219: 129 pgs: 21 active+clean+snaptrim, 55 active+clean, 53 active+clean+snaptrim_wait; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-06-06T07:55:12.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 378986 7.00 MB/sec execute 196 sec latency 55.018 ms 2024-06-06T07:55:13.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 379511 6.99 MB/sec execute 197 sec latency 843.806 ms 2024-06-06T07:55:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:14 smithi181 ceph-mon[36790]: pgmap v220: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:14 smithi087 ceph-mon[32276]: pgmap v220: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:14.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:14 smithi195 ceph-mon[37040]: pgmap v220: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:14.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 379511 6.95 MB/sec execute 198 sec latency 1843.898 ms 2024-06-06T07:55:15.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 379511 6.92 MB/sec execute 199 sec latency 2844.009 ms 2024-06-06T07:55:16.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:15 smithi195 ceph-mon[37040]: pgmap v221: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:16.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:15 smithi195 ceph-mon[37040]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:55:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:15 smithi181 ceph-mon[36790]: pgmap v221: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:15 smithi181 ceph-mon[36790]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:55:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:15 smithi087 ceph-mon[32276]: pgmap v221: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-06-06T07:55:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:15 smithi087 ceph-mon[32276]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:55:16.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 379511 6.88 MB/sec execute 200 sec latency 3844.078 ms 2024-06-06T07:55:17.668 INFO:tasks.workunit.client.0.smithi087.stdout: 1 379511 6.85 MB/sec execute 201 sec latency 4844.383 ms 2024-06-06T07:55:18.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:17 smithi195 ceph-mon[37040]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-06-06T07:55:18.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:17 smithi181 ceph-mon[36790]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-06-06T07:55:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:17 smithi087 ceph-mon[32276]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-06-06T07:55:18.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 380821 6.83 MB/sec execute 202 sec latency 5017.424 ms 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 -- 172.21.15.87:0/1871249147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 msgr2=0x7f114c10d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 --2- 172.21.15.87:0/1871249147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c10d6f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1144009ca0 tx=0x7f114402a8c0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 -- 172.21.15.87:0/1871249147 shutdown_connections 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 --2- 172.21.15.87:0/1871249147 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c10d6f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 --2- 172.21.15.87:0/1871249147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 --2- 172.21.15.87:0/1871249147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 0x7f114c105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 -- 172.21.15.87:0/1871249147 >> 172.21.15.87:0/1871249147 conn(0x7f114c077680 msgr2=0x7f114c077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 -- 172.21.15.87:0/1871249147 shutdown_connections 2024-06-06T07:55:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.658+0000 7f11508d4640 1 -- 172.21.15.87:0/1871249147 wait complete. 2024-06-06T07:55:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 Processor -- start 2024-06-06T07:55:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 -- start start 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 0x7f114c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f114c110c60 con 0x7f114c101540 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f114c110ae0 con 0x7f114c100b80 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.659+0000 7f11508d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f114c110de0 con 0x7f114c105870 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f114a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54986/0 (socket says 172.21.15.87:54986) 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f114a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:43904/0 (socket says 172.21.15.87:43904) 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 -- 172.21.15.87:0/627143101 learned_addr learned my addr 172.21.15.87:0/627143101 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f114ad9d640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 0x7f114c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 -- 172.21.15.87:0/627143101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 msgr2=0x7f114c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 0x7f114c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 -- 172.21.15.87:0/627143101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 msgr2=0x7f114c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 -- 172.21.15.87:0/627143101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c19d0c0 con 0x7f114c101540 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f114a59c640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1141d9b640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9f0 tx=0x7f113400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1143fff640 1 -- 172.21.15.87:0/627143101 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113400cd80 con 0x7f114c101540 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1143fff640 1 -- 172.21.15.87:0/627143101 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134010910 con 0x7f114c101540 2024-06-06T07:55:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c19d3b0 con 0x7f114c101540 2024-06-06T07:55:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f1143fff640 1 -- 172.21.15.87:0/627143101 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134018990 con 0x7f114c101540 2024-06-06T07:55:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.660+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a4c40 con 0x7f114c101540 2024-06-06T07:55:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.662+0000 7f1143fff640 1 -- 172.21.15.87:0/627143101 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1134010ac0 con 0x7f114c101540 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 Processor -- start 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 -- start start 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f1143fff640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 0x7f11380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f8413740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5592f836a4f0 con 0x5592f8406a90 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5592f8308870 con 0x5592f8411350 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.663+0000 7f11535e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592f83664b0 con 0x5592f8406170 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1143fff640 1 -- 172.21.15.87:0/627143101 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f11340cfc10 con 0x7f114c101540 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114ad9d640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 0x7f11380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114a59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114a59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:54628/0 (socket says 172.21.15.87:54628) 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55002/0 (socket says 172.21.15.87:55002) 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114a59c640 1 -- 172.21.15.87:0/414286600 learned_addr learned my addr 172.21.15.87:0/414286600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114ad9d640 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f8413740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 -- 172.21.15.87:0/414286600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 msgr2=0x5592f8406550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 -- 172.21.15.87:0/414286600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 msgr2=0x5592f8413740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f8413740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 -- 172.21.15.87:0/414286600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f83249b0 con 0x5592f8406a90 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114a59c640 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1141d9b640 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11340ccf40 tx=0x7f11340980e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92867723a9d7fc9a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f114ad9d640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 0x7f11380a5260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1144009a60 tx=0x7f114402f330 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1140d99640 1 -- 172.21.15.87:0/414286600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113409bd10 con 0x5592f8406a90 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1140d99640 1 -- 172.21.15.87:0/414286600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11340a00c0 con 0x5592f8406a90 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.664+0000 7f1140d99640 1 -- 172.21.15.87:0/414286600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113407fbd0 con 0x5592f8406a90 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- 172.21.15.87:0/414286600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 msgr2=0x5592f8410e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11340ccf40 tx=0x7f11340980e0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- 172.21.15.87:0/414286600 shutdown_connections 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f8413740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8410e10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 --2- 172.21.15.87:0/414286600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406170 0x5592f8406550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- 172.21.15.87:0/414286600 >> 172.21.15.87:0/414286600 conn(0x5592f8402680 msgr2=0x5592f8402a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- 172.21.15.87:0/414286600 shutdown_connections 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- 172.21.15.87:0/414286600 wait complete. 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 -- start start 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.665+0000 7f11535e9740 1 Processor -- start 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f840e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f840ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5592f8413ec0 con 0x5592f8406170 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5592f82f3ae0 con 0x5592f8411350 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592f8414040 con 0x5592f8406a90 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 10 client.0 _ll_register_callbacks cb 0x5592f832c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55010/0 (socket says 172.21.15.87:55010) 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 -- 172.21.15.87:0/2926750086 learned_addr learned my addr 172.21.15.87:0/2926750086 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114ad9d640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f840ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f1141d9b640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f840e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 -- 172.21.15.87:0/2926750086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 msgr2=0x5592f840e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f840e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 -- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 msgr2=0x5592f840ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f840ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 -- 172.21.15.87:0/2926750086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f807e210 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f114a59c640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f113c0044f0 tx=0x7f113c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaa5d00a2d931f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f1131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c021060 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11340a00c0 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.666+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f113c003cd0 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f113c028050 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 10 client.16743 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5592f814a620 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11340a00c0 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f113c01b070 con 0x5592f8406170 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 10 client.16743 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 20 client.16743 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f84215a0 0x5592f8423990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 4 client.16743 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592f837df10 con 0x5592f84215a0 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.667+0000 7f1141d9b640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f84215a0 0x5592f8423990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f1141d9b640 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f84215a0 0x5592f8423990 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3f36b1d945a24d server_cookie=4a5be89b0699e0ad in_seq=0 out_seq=0 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f1131ffb640 10 client.16743 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f1131ffb640 1 -- 172.21.15.87:0/2926750086 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5592f837df10 con 0x5592f84215a0 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f1131ffb640 10 client.16743 handle_command_reply: tid=0 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 client.16743 shutdown 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 msgr2=0x5592f840e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f113c0044f0 tx=0x7f113c012b30 comp rx=0 tx=0).stop 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 shutdown_connections 2024-06-06T07:55:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f84215a0 0x5592f8423990 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f840ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f840e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f1131ffb640 0 client.16743 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 --2- 172.21.15.87:0/2926750086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f840e3b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.668+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 >> 172.21.15.87:0/2926750086 conn(0x5592f8402680 msgr2=0x5592f8408400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.669+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 shutdown_connections 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.669+0000 7f11535e9740 1 -- 172.21.15.87:0/2926750086 wait complete. 2024-06-06T07:55:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.669+0000 7f11535e9740 20 client.16743 trim_cache size 0 max 16384 2024-06-06T07:55:19.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 383113 6.88 MB/sec execute 203 sec latency 20.801 ms 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 Processor -- start 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 -- start start 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8406170 0x5592f8408b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8411350 0x5592f8429280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1af5f0 con 0x5592f8406a90 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1acec0 con 0x5592f8406170 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f11535e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1aad30 con 0x5592f8411350 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.685+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55016/0 (socket says 172.21.15.87:55016) 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 -- 172.21.15.87:0/115124619 learned_addr learned my addr 172.21.15.87:0/115124619 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f114a59c640 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8406170 0x5592f8408b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f114ad9d640 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8411350 0x5592f8429280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 -- 172.21.15.87:0/115124619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8411350 msgr2=0x5592f8429280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8411350 0x5592f8429280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 -- 172.21.15.87:0/115124619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8406170 msgr2=0x5592f8408b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8406170 0x5592f8408b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 -- 172.21.15.87:0/115124619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f84068c0 con 0x5592f8406a90 2024-06-06T07:55:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1141d9b640 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f113409ba10 tx=0x7f1134097910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab393c63c8acc1d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1133fff640 1 -- 172.21.15.87:0/115124619 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134082120 con 0x5592f8406a90 2024-06-06T07:55:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1133fff640 1 -- 172.21.15.87:0/115124619 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11340cd1c0 con 0x5592f8406a90 2024-06-06T07:55:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.686+0000 7f1133fff640 1 -- 172.21.15.87:0/115124619 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134082440 con 0x5592f8406a90 2024-06-06T07:55:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- 172.21.15.87:0/115124619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 msgr2=0x5592f8401660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f113409ba10 tx=0x7f1134097910 comp rx=0 tx=0).stop 2024-06-06T07:55:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- 172.21.15.87:0/115124619 shutdown_connections 2024-06-06T07:55:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8411350 0x5592f8429280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406a90 0x5592f8401660 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 --2- 172.21.15.87:0/115124619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8406170 0x5592f8408b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- 172.21.15.87:0/115124619 >> 172.21.15.87:0/115124619 conn(0x5592f8402680 msgr2=0x5592f8400f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- 172.21.15.87:0/115124619 shutdown_connections 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- 172.21.15.87:0/115124619 wait complete. 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 -- start start 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.687+0000 7f11535e9740 1 Processor -- start 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f8414e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f84153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1ab380 con 0x5592f8406170 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1ac7f0 con 0x5592f8411350 2024-06-06T07:55:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1aba50 con 0x5592f8406a90 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 10 client.0 _ll_register_callbacks cb 0x7f113c00bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55028/0 (socket says 172.21.15.87:55028) 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 -- 172.21.15.87:0/19770292 learned_addr learned my addr 172.21.15.87:0/19770292 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f1141d9b640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f8414e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114ad9d640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f84153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 -- 172.21.15.87:0/19770292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 msgr2=0x5592f8414e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f8414e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 -- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 msgr2=0x5592f84153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f84153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 -- 172.21.15.87:0/19770292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f8415bf0 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f1141d9b640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f8414e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f114a59c640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f113c014010 tx=0x7f113c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a30894a286cf19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f1131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00b4c0 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f113c005040 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.688+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11340cd1c0 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f113c03d050 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 10 client.16749 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5592f848c330 con 0x5592f8406170 2024-06-06T07:55:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11340cd1c0 con 0x5592f8406170 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f113c01b040 con 0x5592f8406170 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 10 client.16749 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 20 client.16749 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f848ca20 0x5592f848ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 4 client.16749 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592f8377c20 con 0x5592f848ca20 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1141d9b640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f848ca20 0x5592f848ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1141d9b640 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f848ca20 0x5592f848ee40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44f9bdedbc8f19b server_cookie=a89c4b5637bd8300 in_seq=0 out_seq=0 2024-06-06T07:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.689+0000 7f1131ffb640 10 client.16749 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f1131ffb640 1 -- 172.21.15.87:0/19770292 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5592f8377c20 con 0x5592f848ca20 2024-06-06T07:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f1131ffb640 10 client.16749 handle_command_reply: tid=0 2024-06-06T07:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 client.16749 shutdown 2024-06-06T07:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 msgr2=0x5592f8419ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f113c014010 tx=0x7f113c003ce0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 shutdown_connections 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5592f848ca20 0x5592f848ee40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592f8411350 0x5592f84153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f1131ffb640 0 client.16749 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592f8406a90 0x5592f8414e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 --2- 172.21.15.87:0/19770292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592f8406170 0x5592f8419ab0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 >> 172.21.15.87:0/19770292 conn(0x5592f8402680 msgr2=0x5592f84017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 shutdown_connections 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 1 -- 172.21.15.87:0/19770292 wait complete. 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.783+0000 7f11535e9740 20 client.16749 trim_cache size 0 max 16384 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 msgr2=0x7f11380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 0x7f11380a5260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1144009a60 tx=0x7f114402f330 comp rx=0 tx=0).stop 2024-06-06T07:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 msgr2=0x7f114c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9f0 tx=0x7f113400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 shutdown_connections 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f11380a2da0 0x7f11380a5260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f114c105870 0x7f114c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f114c101540 0x7f114c198610 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 --2- 172.21.15.87:0/627143101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f114c100b80 0x7f114c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 >> 172.21.15.87:0/627143101 conn(0x7f114c077680 msgr2=0x7f114c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 shutdown_connections 2024-06-06T07:55:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:19.784+0000 7f11508d4640 1 -- 172.21.15.87:0/627143101 wait complete. 2024-06-06T07:55:20.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:19 smithi195 ceph-mon[37040]: pgmap v224: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:20.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:19 smithi181 ceph-mon[36790]: pgmap v224: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:19 smithi087 ceph-mon[32276]: pgmap v224: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:20.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 383865 6.85 MB/sec execute 204 sec latency 111.309 ms 2024-06-06T07:55:21.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:20 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:55:21.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:20 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:55:21.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:20 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:55:21.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 385530 6.85 MB/sec execute 205 sec latency 40.541 ms 2024-06-06T07:55:22.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:21 smithi195 ceph-mon[37040]: pgmap v225: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:22.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:21 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:55:22.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:21 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:55:22.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:21 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:55:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:21 smithi181 ceph-mon[36790]: pgmap v225: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:21 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:55:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:21 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:55:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:21 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:55:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:21 smithi087 ceph-mon[32276]: pgmap v225: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-06-06T07:55:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:21 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:55:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:21 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:55:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:21 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:55:22.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 387560 6.88 MB/sec execute 206 sec latency 18.537 ms 2024-06-06T07:55:23.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 390209 6.93 MB/sec execute 207 sec latency 26.529 ms 2024-06-06T07:55:24.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:23 smithi195 ceph-mon[37040]: pgmap v226: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:23 smithi181 ceph-mon[36790]: pgmap v226: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:23 smithi087 ceph-mon[32276]: pgmap v226: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:24.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 391781 6.93 MB/sec execute 208 sec latency 28.442 ms 2024-06-06T07:55:25.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 393987 6.96 MB/sec execute 209 sec latency 32.365 ms 2024-06-06T07:55:26.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:25 smithi181 ceph-mon[36790]: pgmap v227: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:25 smithi087 ceph-mon[32276]: pgmap v227: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:25 smithi195 ceph-mon[37040]: pgmap v227: 129 pgs: 129 active+clean; 51 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-06-06T07:55:26.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 395471 6.95 MB/sec execute 210 sec latency 21.240 ms 2024-06-06T07:55:27.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 397903 6.99 MB/sec execute 211 sec latency 15.391 ms 2024-06-06T07:55:28.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:27 smithi181 ceph-mon[36790]: pgmap v228: 129 pgs: 129 active+clean; 57 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-06-06T07:55:28.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:27 smithi087 ceph-mon[32276]: pgmap v228: 129 pgs: 129 active+clean; 57 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-06-06T07:55:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:27 smithi195 ceph-mon[37040]: pgmap v228: 129 pgs: 129 active+clean; 57 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-06-06T07:55:28.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 399900 6.99 MB/sec execute 212 sec latency 34.890 ms 2024-06-06T07:55:29.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 401848 7.01 MB/sec execute 213 sec latency 18.436 ms 2024-06-06T07:55:30.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:29 smithi181 ceph-mon[36790]: pgmap v229: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-06-06T07:55:30.061 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:29 smithi195 ceph-mon[37040]: pgmap v229: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-06-06T07:55:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:29 smithi087 ceph-mon[32276]: pgmap v229: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-06-06T07:55:30.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 404384 7.06 MB/sec execute 214 sec latency 37.283 ms 2024-06-06T07:55:31.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 405987 7.06 MB/sec execute 215 sec latency 18.024 ms 2024-06-06T07:55:32.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:31 smithi181 ceph-mon[36790]: pgmap v230: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-06-06T07:55:32.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:31 smithi087 ceph-mon[32276]: pgmap v230: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-06-06T07:55:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:31 smithi195 ceph-mon[37040]: pgmap v230: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 250 op/s 2024-06-06T07:55:32.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 408395 7.09 MB/sec execute 216 sec latency 13.011 ms 2024-06-06T07:55:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 -- 172.21.15.87:0/2708972424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b8106890 msgr2=0x7fa1b8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 --2- 172.21.15.87:0/2708972424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b8106890 0x7fa1b8106c70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009930 tx=0x7fa1a802a420 comp rx=0 tx=0).stop 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 -- 172.21.15.87:0/2708972424 shutdown_connections 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 --2- 172.21.15.87:0/2708972424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 --2- 172.21.15.87:0/2708972424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b81008d0 0x7fa1b8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 --2- 172.21.15.87:0/2708972424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b8106890 0x7fa1b8106c70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 -- 172.21.15.87:0/2708972424 >> 172.21.15.87:0/2708972424 conn(0x7fa1b8077a20 msgr2=0x7fa1b8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 -- 172.21.15.87:0/2708972424 shutdown_connections 2024-06-06T07:55:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.879+0000 7fa1bce72640 1 -- 172.21.15.87:0/2708972424 wait complete. 2024-06-06T07:55:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 Processor -- start 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 -- start start 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b80fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b8106890 0x7fa1b80fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8110c60 con 0x7fa1b81008d0 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8110ae0 con 0x7fa1b8101290 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1bce72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8110de0 con 0x7fa1b8106890 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.880+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57088/0 (socket says 172.21.15.87:57088) 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 -- 172.21.15.87:0/3694419691 learned_addr learned my addr 172.21.15.87:0/3694419691 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b5d9b640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b80fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b8106890 0x7fa1b80fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 -- 172.21.15.87:0/3694419691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b8106890 msgr2=0x7fa1b80fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b8106890 0x7fa1b80fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 -- 172.21.15.87:0/3694419691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 msgr2=0x7fa1b80fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b80fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 -- 172.21.15.87:0/3694419691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b81a67a0 con 0x7fa1b81008d0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b5d9b640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b80fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009230 tx=0x7fa1a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1a77fe640 1 -- 172.21.15.87:0/3694419691 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a803f030 con 0x7fa1b81008d0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b81a6a30 con 0x7fa1b81008d0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1a77fe640 1 -- 172.21.15.87:0/3694419691 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8004440 con 0x7fa1b81008d0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1a77fe640 1 -- 172.21.15.87:0/3694419691 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8035070 con 0x7fa1b81008d0 2024-06-06T07:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.881+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a6f70 con 0x7fa1b81008d0 2024-06-06T07:55:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.884+0000 7fa1a77fe640 1 -- 172.21.15.87:0/3694419691 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa1a8042030 con 0x7fa1b81008d0 2024-06-06T07:55:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.884+0000 7fa1bfb80740 1 Processor -- start 2024-06-06T07:55:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.884+0000 7fa1bfb80740 1 -- start start 2024-06-06T07:55:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd6c350 0x563e9fd6e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563e9fcc54f0 con 0x563e9fd61a90 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563e9fc63870 con 0x563e9fd61170 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e9fcc14b0 con 0x563e9fd6c350 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57096/0 (socket says 172.21.15.87:57096) 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 -- 172.21.15.87:0/3274122299 learned_addr learned my addr 172.21.15.87:0/3274122299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1a77fe640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 0x7fa1a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 -- 172.21.15.87:0/3274122299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd6c350 msgr2=0x563e9fd6e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd6c350 0x563e9fd6e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 -- 172.21.15.87:0/3274122299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 msgr2=0x563e9fd61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b659c640 1 -- 172.21.15.87:0/3274122299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9fc7f9b0 con 0x563e9fd61a90 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.885+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 0x7fa1a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.886+0000 7fa1a77fe640 1 -- 172.21.15.87:0/3694419691 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa1a80efb90 con 0x7fa1b81008d0 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.886+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ece30 tx=0x7fa1a80b11d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e799698473654f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.886+0000 7fa1a4ff9640 1 -- 172.21.15.87:0/3274122299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80bb6a0 con 0x563e9fd61a90 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.886+0000 7fa1a4ff9640 1 -- 172.21.15.87:0/3274122299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a808aa80 con 0x563e9fd61a90 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.886+0000 7fa1a4ff9640 1 -- 172.21.15.87:0/3274122299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a808dfb0 con 0x563e9fd61a90 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3274122299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 msgr2=0x563e9fd6be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 0x7fa1a00a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0005fd0 tx=0x7fa1b0013040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ece30 tx=0x7fa1a80b11d0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3274122299 shutdown_connections 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd6c350 0x563e9fd6e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6be10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3274122299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3274122299 >> 172.21.15.87:0/3274122299 conn(0x563e9fd5d680 msgr2=0x563e9fd5da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3274122299 shutdown_connections 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3274122299 wait complete. 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 -- start start 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.887+0000 7fa1bfb80740 1 Processor -- start 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd69250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd69790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563e9fd6ed40 con 0x563e9fd6c350 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563e9fc4eae0 con 0x563e9fd61170 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e9fd6eec0 con 0x563e9fd61a90 2024-06-06T07:55:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1bfb80740 10 client.0 _ll_register_callbacks cb 0x563e9fc879b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57100/0 (socket says 172.21.15.87:57100) 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd69790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/4276084213 learned_addr learned my addr 172.21.15.87:0/4276084213 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b659c640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd69250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/4276084213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 msgr2=0x563e9fd69790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd69790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 msgr2=0x563e9fd69250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd69250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/4276084213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9f9d9210 con 0x563e9fd6c350 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0004a80 tx=0x7fa1b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31b1f1306c068fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.888+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0029060 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa1b001c980 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a808aa80 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1b001cb80 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e9faa5620 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a808aa80 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa1b0024410 con 0x563e9fd6c350 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 20 client.16761 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fd7c180 0x563e9fd7e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 4 client.16761 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.889+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e9fcd8f10 con 0x563e9fd7c180 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.890+0000 7fa1b659c640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fd7c180 0x563e9fd7e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.890+0000 7fa19dffb640 10 client.16761 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.890+0000 7fa1b659c640 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fd7c180 0x563e9fd7e590 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88350540812febc server_cookie=4bd982efbae1bc21 in_seq=0 out_seq=0 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa19dffb640 1 -- 172.21.15.87:0/4276084213 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563e9fcd8f10 con 0x563e9fd7c180 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa19dffb640 10 client.16761 handle_command_reply: tid=0 2024-06-06T07:55:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 client.16761 shutdown 2024-06-06T07:55:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 msgr2=0x563e9fd69cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0004a80 tx=0x7fa1b0004ab0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 shutdown_connections 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fd7c180 0x563e9fd7e590 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd69cd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd69790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/4276084213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd69250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 >> 172.21.15.87:0/4276084213 conn(0x563e9fd5d680 msgr2=0x563e9fd63400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa19dffb640 0 client.16761 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 shutdown_connections 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.891+0000 7fa1bfb80740 1 -- 172.21.15.87:0/4276084213 wait complete. 2024-06-06T07:55:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.892+0000 7fa1bfb80740 20 client.16761 trim_cache size 0 max 16384 2024-06-06T07:55:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.910+0000 7fa1bfb80740 1 Processor -- start 2024-06-06T07:55:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.910+0000 7fa1bfb80740 1 -- start start 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81b3a10 con 0x563e9fd6c350 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81b12e0 con 0x563e9fd61170 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81af150 con 0x563e9fd61a90 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:42284/0 (socket says 172.21.15.87:42284) 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57102/0 (socket says 172.21.15.87:57102) 2024-06-06T07:55:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b659c640 1 -- 172.21.15.87:0/3070249274 learned_addr learned my addr 172.21.15.87:0/3070249274 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/3070249274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 msgr2=0x563e9fd83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/3070249274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 msgr2=0x563e9fd63b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 -- 172.21.15.87:0/3070249274 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9fd677a0 con 0x563e9fd6c350 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b659c640 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.911+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0013010 tx=0x7fa1b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414dbbf527a7fd67 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.912+0000 7fa19ffff640 1 -- 172.21.15.87:0/3070249274 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0033080 con 0x563e9fd6c350 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.912+0000 7fa19ffff640 1 -- 172.21.15.87:0/3070249274 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b001c7d0 con 0x563e9fd6c350 2024-06-06T07:55:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.912+0000 7fa19ffff640 1 -- 172.21.15.87:0/3070249274 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b001ae00 con 0x563e9fd6c350 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.912+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3070249274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 msgr2=0x563e9fd843e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.912+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0013010 tx=0x7fa1b0009b50 comp rx=0 tx=0).stop 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3070249274 shutdown_connections 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd6c350 0x563e9fd843e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61a90 0x563e9fd83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/3070249274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd61170 0x563e9fd63b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3070249274 >> 172.21.15.87:0/3070249274 conn(0x563e9fd5d680 msgr2=0x563e9fd64530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3070249274 shutdown_connections 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- 172.21.15.87:0/3070249274 wait complete. 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- start start 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 Processor -- start 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61170 0x563e9fd6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd6c350 0x563e9fde4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81af7a0 con 0x563e9fd61a90 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81b0c10 con 0x563e9fd6c350 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81afe70 con 0x563e9fd61170 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1bfb80740 10 client.0 _ll_register_callbacks cb 0x7fa1b0009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1b5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1b5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57104/0 (socket says 172.21.15.87:57104) 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1b5d9b640 1 -- 172.21.15.87:0/2837604034 learned_addr learned my addr 172.21.15.87:0/2837604034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1b659c640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61170 0x563e9fd6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.913+0000 7fa1b6d9d640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd6c350 0x563e9fde4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 -- 172.21.15.87:0/2837604034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61170 msgr2=0x563e9fd6f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61170 0x563e9fd6f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 -- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd6c350 msgr2=0x563e9fde4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd6c350 0x563e9fde4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 -- 172.21.15.87:0/2837604034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9fde53f0 con 0x563e9fd61a90 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1b5d9b640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac010cd0 tx=0x7fa1ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bd228fdf5c62eb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac004460 con 0x563e9fd61a90 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b001c7d0 con 0x563e9fd61a90 2024-06-06T07:55:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa1ac0035f0 con 0x563e9fd61a90 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1ac00fe60 con 0x563e9fd61a90 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1bfb80740 10 client.16767 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e9fde67e0 con 0x563e9fd61a90 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b001c7d0 con 0x563e9fd61a90 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.914+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa1ac01d960 con 0x563e9fd61a90 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 10 client.16767 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 20 client.16767 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fde6fc0 0x563e9fde93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 4 client.16767 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e9fcd2c20 con 0x563e9fde6fc0 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1b659c640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fde6fc0 0x563e9fde93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa19dffb640 10 client.16767 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:32.915+0000 7fa1b659c640 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fde6fc0 0x563e9fde93e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfecb8859cf0382 server_cookie=134cba5057856238 in_seq=0 out_seq=0 2024-06-06T07:55:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa19dffb640 1 -- 172.21.15.87:0/2837604034 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563e9fcd2c20 con 0x563e9fde6fc0 2024-06-06T07:55:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa19dffb640 10 client.16767 handle_command_reply: tid=0 2024-06-06T07:55:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 client.16767 shutdown 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 msgr2=0x563e9fd6faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac010cd0 tx=0x7fa1ac00f790 comp rx=0 tx=0).stop 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 shutdown_connections 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563e9fde6fc0 0x563e9fde93e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e9fd6c350 0x563e9fde4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa19dffb640 0 client.16767 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563e9fd61a90 0x563e9fd6faf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 --2- 172.21.15.87:0/2837604034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e9fd61170 0x563e9fd6f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 >> 172.21.15.87:0/2837604034 conn(0x563e9fd5d680 msgr2=0x563e9fd69210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 shutdown_connections 2024-06-06T07:55:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.149+0000 7fa1bfb80740 1 -- 172.21.15.87:0/2837604034 wait complete. 2024-06-06T07:55:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bfb80740 20 client.16767 trim_cache size 0 max 16384 2024-06-06T07:55:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:55:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:55:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:55:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 msgr2=0x7fa1a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 0x7fa1a00a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0005fd0 tx=0x7fa1b0013040 comp rx=0 tx=0).stop 2024-06-06T07:55:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 msgr2=0x7fa1b80fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009230 tx=0x7fa1a8004040 comp rx=0 tx=0).stop 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 shutdown_connections 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa1a00a2da0 0x7fa1a00a5260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1b8106890 0x7fa1b80fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b8101290 0x7fa1b80fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 --2- 172.21.15.87:0/3694419691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1b81008d0 0x7fa1b80fd860 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 >> 172.21.15.87:0/3694419691 conn(0x7fa1b8077a20 msgr2=0x7fa1b8069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 shutdown_connections 2024-06-06T07:55:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:33.150+0000 7fa1bce72640 1 -- 172.21.15.87:0/3694419691 wait complete. 2024-06-06T07:55:33.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 409608 7.08 MB/sec execute 217 sec latency 169.085 ms 2024-06-06T07:55:34.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:33 smithi181 ceph-mon[36790]: pgmap v231: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-06-06T07:55:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:33 smithi087 ceph-mon[32276]: pgmap v231: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-06-06T07:55:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:33 smithi195 ceph-mon[37040]: pgmap v231: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-06-06T07:55:34.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 411970 7.11 MB/sec execute 218 sec latency 14.198 ms 2024-06-06T07:55:34.934 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T07:55:34.934 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-06-06T07:55:34.935 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:55:35.183 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:55:35.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 413911 7.11 MB/sec execute 219 sec latency 30.385 ms 2024-06-06T07:55:35.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:35 smithi087 ceph-mon[32276]: pgmap v232: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-06-06T07:55:36.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:35 smithi181 ceph-mon[36790]: pgmap v232: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-06-06T07:55:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:35 smithi195 ceph-mon[37040]: pgmap v232: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-06-06T07:55:36.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 415850 7.14 MB/sec execute 220 sec latency 22.484 ms 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 -- 172.21.15.87:0/1604734725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101d50 msgr2=0x7f822010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 --2- 172.21.15.87:0/1604734725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101d50 0x7f822010e240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f821800b0a0 tx=0x7f821800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 -- 172.21.15.87:0/1604734725 shutdown_connections 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 --2- 172.21.15.87:0/1604734725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 --2- 172.21.15.87:0/1604734725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101d50 0x7f822010e240 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 --2- 172.21.15.87:0/1604734725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101390 0x7f8220101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 -- 172.21.15.87:0/1604734725 >> 172.21.15.87:0/1604734725 conn(0x7f8220078340 msgr2=0x7f8220078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 -- 172.21.15.87:0/1604734725 shutdown_connections 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 -- 172.21.15.87:0/1604734725 wait complete. 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.595+0000 7f8225bdd700 1 Processor -- start 2024-06-06T07:55:37.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 -- start start 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 0x7f8220104c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201036d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82201116f0 con 0x7f8220101390 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8220111570 con 0x7f8220107400 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f8225bdd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8220111870 con 0x7f8220101d50 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 0x7f8220104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58296/0 (socket says 172.21.15.87:58296) 2024-06-06T07:55:37.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 -- 172.21.15.87:0/3408278199 learned_addr learned my addr 172.21.15.87:0/3408278199 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821ffff700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 -- 172.21.15.87:0/3408278199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 msgr2=0x7f82201036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 -- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 msgr2=0x7f8220104c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 0x7f8220104c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821effd700 1 -- 172.21.15.87:0/3408278199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82201ab5e0 con 0x7f8220101d50 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821f7fe700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 0x7f8220104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.596+0000 7f821ffff700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f821effd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9c0 tx=0x7f821400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821400cbd0 con 0x7f8220101d50 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82201ab870 con 0x7f8220101d50 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201abd80 con 0x7f8220101d50 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821400ee60 con 0x7f8220101d50 2024-06-06T07:55:37.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.597+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214005630 con 0x7f8220101d50 2024-06-06T07:55:37.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.598+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c005360 con 0x7f8220101d50 2024-06-06T07:55:37.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.599+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8214044bb0 con 0x7f8220101d50 2024-06-06T07:55:37.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.600+0000 7f821cff9700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 0x7f82080a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:37.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.600+0000 7f821f7fe700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 0x7f82080a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:37.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.600+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f82140cfa20 con 0x7f8220101d50 2024-06-06T07:55:37.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.600+0000 7f821f7fe700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 0x7f82080a52d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8210009fd0 tx=0x7f8210009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:37.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.603+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f821409c1c0 con 0x7f8220101d50 2024-06-06T07:55:37.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 418237 7.16 MB/sec execute 221 sec latency 32.630 ms 2024-06-06T07:55:37.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.870+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f820c005650 con 0x7f8220101d50 2024-06-06T07:55:37.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.871+0000 7f821cff9700 1 -- 172.21.15.87:0/3408278199 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7f821409bac0 con 0x7f8220101d50 2024-06-06T07:55:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:55:37.872 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 msgr2=0x7f82080a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 0x7f82080a52d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8210009fd0 tx=0x7f8210009260 comp rx=0 tx=0).stop 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 msgr2=0x7f8220103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9c0 tx=0x7f821400ecd0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 shutdown_connections 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8220107400 0x7f82201036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82080a2e10 0x7f82080a52d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8220101390 0x7f8220104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 --2- 172.21.15.87:0/3408278199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8220101d50 0x7f8220103070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 >> 172.21.15.87:0/3408278199 conn(0x7f8220078340 msgr2=0x7f82201005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 shutdown_connections 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:37.874+0000 7f8225bdd700 1 -- 172.21.15.87:0/3408278199 wait complete. 2024-06-06T07:55:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:55:38.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:37 smithi181 ceph-mon[36790]: pgmap v233: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-06-06T07:55:38.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:37 smithi087 ceph-mon[32276]: pgmap v233: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-06-06T07:55:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:37 smithi195 ceph-mon[37040]: pgmap v233: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 375 op/s 2024-06-06T07:55:38.613 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0' 2024-06-06T07:55:38.613 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0 2024-06-06T07:55:38.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 420113 7.18 MB/sec execute 222 sec latency 15.865 ms 2024-06-06T07:55:38.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:38 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3408278199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:38.875 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:55:39.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:38 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3408278199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:39.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:38 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3408278199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:55:39.669 INFO:tasks.workunit.client.0.smithi087.stdout: 1 422503 7.21 MB/sec execute 223 sec latency 16.427 ms 2024-06-06T07:55:40.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:39 smithi181 ceph-mon[36790]: pgmap v234: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-06-06T07:55:40.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:39 smithi087 ceph-mon[32276]: pgmap v234: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-06-06T07:55:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:39 smithi195 ceph-mon[37040]: pgmap v234: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-06-06T07:55:40.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 424479 7.21 MB/sec execute 224 sec latency 19.761 ms 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 -- 172.21.15.87:0/2124257703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d0107850 msgr2=0x7f82d010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/2124257703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d0107850 0x7f82d010f3d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f82c000b0d0 tx=0x7f82c000b3e0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 -- 172.21.15.87:0/2124257703 shutdown_connections 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/2124257703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d0107850 0x7f82d010f3d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/2124257703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d006a3d0 0x7f82d006a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/2124257703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006ad80 0x7f82d0107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 -- 172.21.15.87:0/2124257703 >> 172.21.15.87:0/2124257703 conn(0x7f82d0078330 msgr2=0x7f82d0078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:40.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 -- 172.21.15.87:0/2124257703 shutdown_connections 2024-06-06T07:55:40.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.924+0000 7f82d5e0f700 1 -- 172.21.15.87:0/2124257703 wait complete. 2024-06-06T07:55:40.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 Processor -- start 2024-06-06T07:55:40.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 -- start start 2024-06-06T07:55:40.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006a3d0 0x7f82d019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d0107850 0x7f82d0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112790 con 0x7f82d006ad80 2024-06-06T07:55:40.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112610 con 0x7f82d006a3d0 2024-06-06T07:55:40.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82d5e0f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112910 con 0x7f82d0107850 2024-06-06T07:55:40.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.925+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45164/0 (socket says 172.21.15.87:45164) 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 -- 172.21.15.87:0/3938013082 learned_addr learned my addr 172.21.15.87:0/3938013082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006a3d0 0x7f82d019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82cffff700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d0107850 0x7f82d0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 -- 172.21.15.87:0/3938013082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d0107850 msgr2=0x7f82d0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d0107850 0x7f82d0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 -- 172.21.15.87:0/3938013082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006a3d0 msgr2=0x7f82d019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006a3d0 0x7f82d019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 -- 172.21.15.87:0/3938013082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0198ff0 con 0x7f82d006ad80 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e9a0 tx=0x7f82b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ccff9700 1 -- 172.21.15.87:0/3938013082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b800ca10 con 0x7f82d006ad80 2024-06-06T07:55:40.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ccff9700 1 -- 172.21.15.87:0/3938013082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b800cbb0 con 0x7f82d006ad80 2024-06-06T07:55:40.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d01992e0 con 0x7f82d006ad80 2024-06-06T07:55:40.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82ccff9700 1 -- 172.21.15.87:0/3938013082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8018680 con 0x7f82d006ad80 2024-06-06T07:55:40.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.926+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01a55a0 con 0x7f82d006ad80 2024-06-06T07:55:40.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.929+0000 7f82ccff9700 1 -- 172.21.15.87:0/3938013082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f82b8018820 con 0x7f82d006ad80 2024-06-06T07:55:40.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.930+0000 7f82ccff9700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 0x7f82bc0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.930+0000 7f82ccff9700 1 -- 172.21.15.87:0/3938013082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f82b80cfaf0 con 0x7f82d006ad80 2024-06-06T07:55:40.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.930+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 0x7f82bc0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 Processor -- start 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 -- start start 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 0x7f82bc0a52f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82d01999d0 tx=0x7f82c800b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b64d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559e35a4d0f0 con 0x559e35b649a0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e35aab9e0 con 0x559e35b5f0f0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82e0692b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e35a135f0 con 0x559e35b652c0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82cf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b64d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.931+0000 7f82cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:45708/0 (socket says 172.21.15.87:45708) 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82cffff700 1 -- 172.21.15.87:0/62104884 learned_addr learned my addr 172.21.15.87:0/62104884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58316/0 (socket says 172.21.15.87:58316) 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 -- 172.21.15.87:0/62104884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 msgr2=0x559e35b5f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 -- 172.21.15.87:0/62104884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 msgr2=0x559e35b64d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b64d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 -- 172.21.15.87:0/62104884 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e359faa60 con 0x559e35b652c0 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82cffff700 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82ceffd700 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cce20 tx=0x7f82b809c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e667e4e4442c9461 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82c5ffb700 1 -- 172.21.15.87:0/62104884 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80a4860 con 0x559e35b652c0 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82c5ffb700 1 -- 172.21.15.87:0/62104884 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b8099ef0 con 0x559e35b652c0 2024-06-06T07:55:40.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.932+0000 7f82c5ffb700 1 -- 172.21.15.87:0/62104884 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b809b2a0 con 0x559e35b652c0 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 -- 172.21.15.87:0/62104884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 msgr2=0x559e35b5ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cce20 tx=0x7f82b809c440 comp rx=0 tx=0).stop 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 -- 172.21.15.87:0/62104884 shutdown_connections 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b5f0f0 0x559e35b5f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b64d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 --2- 172.21.15.87:0/62104884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b652c0 0x559e35b5ebb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 -- 172.21.15.87:0/62104884 >> 172.21.15.87:0/62104884 conn(0x559e35a87210 msgr2=0x559e35b55a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 -- 172.21.15.87:0/62104884 shutdown_connections 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.933+0000 7f82e0692b80 1 -- 172.21.15.87:0/62104884 wait complete. 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 -- start start 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 Processor -- start 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b68860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559e35a4c150 con 0x559e35b649a0 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e35aa79a0 con 0x559e35b652c0 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e35b6bca0 con 0x559e35b5f0f0 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82e0692b80 10 client.0 _ll_register_callbacks cb 0x559e35a41740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45180/0 (socket says 172.21.15.87:45180) 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82ceffd700 1 -- 172.21.15.87:0/3034284360 learned_addr learned my addr 172.21.15.87:0/3034284360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.934+0000 7f82cffff700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b68860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 -- 172.21.15.87:0/3034284360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 msgr2=0x559e35b6b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 -- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 msgr2=0x559e35b68860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b68860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 -- 172.21.15.87:0/3034284360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e35a718e0 con 0x559e35b649a0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82ceffd700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cce80 tx=0x7f82b8009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5405be2c39fb1b26 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80b8070 con 0x559e35b649a0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f82b809c050 con 0x559e35b649a0 2024-06-06T07:55:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b8099ef0 con 0x559e35b649a0 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82b809b040 con 0x559e35b649a0 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82e0692b80 10 client.16785 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e35b76420 con 0x559e35b649a0 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.935+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b8099ef0 con 0x559e35b649a0 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f82b80979d0 con 0x559e35b649a0 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 10 client.16785 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 20 client.16785 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35b7ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 4 client.16785 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e35ab8ee0 con 0x559e35b788f0 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35b7ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35b7ace0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef48dbeb73d3bcf server_cookie=87b49823a4107b81 in_seq=0 out_seq=0 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.936+0000 7f82aeffd700 10 client.16785 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82aeffd700 1 -- 172.21.15.87:0/3034284360 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e35ab8ee0 con 0x559e35b788f0 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82aeffd700 10 client.16785 handle_command_reply: tid=0 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82e0692b80 1 client.16785 shutdown 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 msgr2=0x559e35b68320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cce80 tx=0x7f82b8009790 comp rx=0 tx=0).stop 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 shutdown_connections 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.937+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35b7ace0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b68860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82aeffd700 0 client.16785 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 --2- 172.21.15.87:0/3034284360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b68320 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 >> 172.21.15.87:0/3034284360 conn(0x559e35a87210 msgr2=0x559e35b54c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 shutdown_connections 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 1 -- 172.21.15.87:0/3034284360 wait complete. 2024-06-06T07:55:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.938+0000 7f82e0692b80 20 client.16785 trim_cache size 0 max 16384 2024-06-06T07:55:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 Processor -- start 2024-06-06T07:55:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 -- start start 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01aee10 con 0x559e35b649a0 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ac6a0 con 0x559e35b652c0 2024-06-06T07:55:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82e0692b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01aa510 con 0x559e35b5f0f0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.959+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82ceffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45188/0 (socket says 172.21.15.87:45188) 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82ceffd700 1 -- 172.21.15.87:0/2437545034 learned_addr learned my addr 172.21.15.87:0/2437545034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cffff700 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2437545034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 msgr2=0x559e35b55900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b55900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2437545034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 msgr2=0x559e35b553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2437545034 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e35b5eb40 con 0x559e35b5f0f0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82ceffd700 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6ab60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x559e35b6bdf0 tx=0x7f82c8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977b08b3bf0706f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82c57fa700 1 -- 172.21.15.87:0/2437545034 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c802a6f0 con 0x559e35b5f0f0 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82c57fa700 1 -- 172.21.15.87:0/2437545034 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8020540 con 0x559e35b5f0f0 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.960+0000 7f82c57fa700 1 -- 172.21.15.87:0/2437545034 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80116a0 con 0x559e35b5f0f0 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- 172.21.15.87:0/2437545034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 msgr2=0x559e35b6ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6ab60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x559e35b6bdf0 tx=0x7f82c8013000 comp rx=0 tx=0).stop 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- 172.21.15.87:0/2437545034 shutdown_connections 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b55900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b5f0f0 0x559e35b6ab60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2437545034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b649a0 0x559e35b553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- 172.21.15.87:0/2437545034 >> 172.21.15.87:0/2437545034 conn(0x559e35a87210 msgr2=0x559e35b5e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- 172.21.15.87:0/2437545034 shutdown_connections 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- 172.21.15.87:0/2437545034 wait complete. 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.961+0000 7f82e0692b80 1 -- start start 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 Processor -- start 2024-06-06T07:55:40.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b649a0 0x559e35b7c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b7cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01aab60 con 0x559e35b5f0f0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01abfd0 con 0x559e35b652c0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ab230 con 0x559e35b649a0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82e0692b80 10 client.0 _ll_register_callbacks cb 0x559e35a41740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b7cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45190/0 (socket says 172.21.15.87:45190) 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2725720632 learned_addr learned my addr 172.21.15.87:0/2725720632 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82ceffd700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b649a0 0x559e35b7c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2725720632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b649a0 msgr2=0x559e35b7c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b649a0 0x559e35b7c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 msgr2=0x559e35b7cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b7cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cf7fe700 1 -- 172.21.15.87:0/2725720632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e35b7d2c0 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.962+0000 7f82cffff700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b7cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82cf7fe700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f82c800b4c0 tx=0x7f82c8020990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450318f088a99468 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004950 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c8020540 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f82c8004af0 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82c8048050 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 10 client.16791 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e35be40d0 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8020540 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f82c8043030 con 0x559e35b5f0f0 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 10 client.16791 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 20 client.16791 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35be67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 4 client.16791 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}] 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.963+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}) -- 0x559e35ab3460 con 0x559e35b788f0 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.964+0000 7f82ceffd700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35be67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.964+0000 7f82aeffd700 10 client.16791 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:40.964+0000 7f82ceffd700 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35be67a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0c8f8bdf18df1d server_cookie=a8ba5a9a223db8c7 in_seq=0 out_seq=0 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82aeffd700 1 -- 172.21.15.87:0/2725720632 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e35ab3460 con 0x559e35b788f0 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82aeffd700 10 client.16791 handle_command_reply: tid=0 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 client.16791 shutdown 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 msgr2=0x559e35b7e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f82c800b4c0 tx=0x7f82c8020990 comp rx=0 tx=0).stop 2024-06-06T07:55:41.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 shutdown_connections 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x559e35b788f0 0x559e35be67a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82aeffd700 0 client.16791 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e35b652c0 0x559e35b7cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e35b5f0f0 0x559e35b7e300 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 --2- 172.21.15.87:0/2725720632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e35b649a0 0x559e35b7c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 >> 172.21.15.87:0/2725720632 conn(0x559e35a87210 msgr2=0x559e35b66890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 shutdown_connections 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 1 -- 172.21.15.87:0/2725720632 wait complete. 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.065+0000 7f82e0692b80 20 client.16791 trim_cache size 0 max 16384 2024-06-06T07:55:41.066 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 22, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "1c04fd38": { 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 22, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.6, 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:55:41.067 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:55:41.068 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 msgr2=0x7f82bc0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 0x7f82bc0a52f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82d01999d0 tx=0x7f82c800b3c0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 msgr2=0x7f82d019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e9a0 tx=0x7f82b800ecb0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 shutdown_connections 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82d006a3d0 0x7f82d019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82bc0a2e30 0x7f82bc0a52f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82d0107850 0x7f82d0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 --2- 172.21.15.87:0/3938013082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d006ad80 0x7f82d019ec10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 >> 172.21.15.87:0/3938013082 conn(0x7f82d0078330 msgr2=0x7f82d00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 shutdown_connections 2024-06-06T07:55:41.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:55:41.066+0000 7f82d5e0f700 1 -- 172.21.15.87:0/3938013082 wait complete. 2024-06-06T07:55:41.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.20 MB/sec execute 225 sec latency 605.635 ms 2024-06-06T07:55:42.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:41 smithi181 ceph-mon[36790]: pgmap v235: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-06-06T07:55:42.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:41 smithi087 ceph-mon[32276]: pgmap v235: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-06-06T07:55:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:41 smithi195 ceph-mon[37040]: pgmap v235: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-06-06T07:55:42.439 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1c04fd38', quiesce took 7.5 seconds. Will release after: 18.1 2024-06-06T07:55:42.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.17 MB/sec execute 226 sec latency 1605.747 ms 2024-06-06T07:55:43.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.14 MB/sec execute 227 sec latency 2605.844 ms 2024-06-06T07:55:44.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:43 smithi181 ceph-mon[36790]: pgmap v236: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 376 op/s 2024-06-06T07:55:44.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:43 smithi087 ceph-mon[32276]: pgmap v236: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 376 op/s 2024-06-06T07:55:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:43 smithi195 ceph-mon[37040]: pgmap v236: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 376 op/s 2024-06-06T07:55:44.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.11 MB/sec execute 228 sec latency 3605.923 ms 2024-06-06T07:55:45.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.08 MB/sec execute 229 sec latency 4605.967 ms 2024-06-06T07:55:46.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:45 smithi181 ceph-mon[36790]: pgmap v237: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 289 op/s 2024-06-06T07:55:46.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:45 smithi087 ceph-mon[32276]: pgmap v237: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 289 op/s 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 -- 172.21.15.87:0/3125573133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c4100b80 msgr2=0x7fc1c4101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/3125573133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c4100b80 0x7fc1c4101000 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009930 tx=0x7fc1b802a420 comp rx=0 tx=0).stop 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 -- 172.21.15.87:0/3125573133 shutdown_connections 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/3125573133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 0x7fc1c410d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/3125573133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c4100b80 0x7fc1c4101000 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/3125573133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c41069c0 0x7fc1c4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 -- 172.21.15.87:0/3125573133 >> 172.21.15.87:0/3125573133 conn(0x7fc1c40775f0 msgr2=0x7fc1c4077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 -- 172.21.15.87:0/3125573133 shutdown_connections 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.251+0000 7fc1c8af5640 1 -- 172.21.15.87:0/3125573133 wait complete. 2024-06-06T07:55:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.252+0000 7fc1c8af5640 1 Processor -- start 2024-06-06T07:55:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.252+0000 7fc1c8af5640 1 -- start start 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.252+0000 7fc1c8af5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 0x7fc1c4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 0x7fc1c4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 0x7fc1c419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4110c90 con 0x7fc1c41069c0 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4110b10 con 0x7fc1c4101540 2024-06-06T07:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4110e10 con 0x7fc1c4100b80 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 0x7fc1c419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 0x7fc1c4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 0x7fc1c4197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58376/0 (socket says 172.21.15.87:58376) 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c259c640 1 -- 172.21.15.87:0/1965736065 learned_addr learned my addr 172.21.15.87:0/1965736065 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 0x7fc1c4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/1965736065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 msgr2=0x7fc1c4197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 0x7fc1c4197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/1965736065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 msgr2=0x7fc1c4197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 0x7fc1c4197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/1965736065 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c419c040 con 0x7fc1c41069c0 2024-06-06T07:55:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 0x7fc1c419b910 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c710 tx=0x7fc1bc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1965736065 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004220 con 0x7fc1c41069c0 2024-06-06T07:55:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c419c330 con 0x7fc1c41069c0 2024-06-06T07:55:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1965736065 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc00ce20 con 0x7fc1c41069c0 2024-06-06T07:55:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1965736065 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc003e20 con 0x7fc1c41069c0 2024-06-06T07:55:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.253+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41a3bc0 con 0x7fc1c41069c0 2024-06-06T07:55:46.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:45 smithi195 ceph-mon[37040]: pgmap v237: 129 pgs: 129 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 289 op/s 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.256+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1965736065 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc1bc0079b0 con 0x7fc1c41069c0 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.256+0000 7fc1cb801740 1 Processor -- start 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 -- start start 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1b37fe640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 0x7fc1ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012deca0 0x5588012e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5588012404f0 con 0x5588012e9560 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588011de870 con 0x5588012de380 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1cb801740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55880123c4b0 con 0x5588012deca0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 0x7fc1ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1b37fe640 1 -- 172.21.15.87:0/1965736065 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc1bc0cf3f0 con 0x7fc1c41069c0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45208/0 (socket says 172.21.15.87:45208) 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/140337836 learned_addr learned my addr 172.21.15.87:0/140337836 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c259c640 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012deca0 0x5588012e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/140337836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012deca0 msgr2=0x5588012e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012deca0 0x5588012e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/140337836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 msgr2=0x5588012de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 -- 172.21.15.87:0/140337836 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588011fa9b0 con 0x5588012e9560 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 0x7fc1ac0a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009900 tx=0x7fc1b80043e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.257+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a3c30 tx=0x7fc1bc09af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a482eaf68c1996e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1b0ff9640 1 -- 172.21.15.87:0/140337836 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01d030 con 0x5588012e9560 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1b0ff9640 1 -- 172.21.15.87:0/140337836 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc09f8a0 con 0x5588012e9560 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1b0ff9640 1 -- 172.21.15.87:0/140337836 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0cc9a0 con 0x5588012e9560 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 -- 172.21.15.87:0/140337836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 msgr2=0x5588012eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a3c30 tx=0x7fc1bc09af00 comp rx=0 tx=0).stop 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 -- 172.21.15.87:0/140337836 shutdown_connections 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012e9560 0x5588012eb950 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012deca0 0x5588012e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 --2- 172.21.15.87:0/140337836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.258+0000 7fc1cb801740 1 -- 172.21.15.87:0/140337836 >> 172.21.15.87:0/140337836 conn(0x5588012da890 msgr2=0x5588012dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- 172.21.15.87:0/140337836 shutdown_connections 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- 172.21.15.87:0/140337836 wait complete. 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- start start 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 Processor -- start 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5588012ec190 con 0x5588012deca0 2024-06-06T07:55:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588011c9ae0 con 0x5588012de380 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588012ec310 con 0x5588012e9560 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1cb801740 10 client.0 _ll_register_callbacks cb 0x5588012029b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45214/0 (socket says 172.21.15.87:45214) 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/2739594486 learned_addr learned my addr 172.21.15.87:0/2739594486 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.259+0000 7fc1c259c640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/2739594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 msgr2=0x5588012e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 msgr2=0x5588012e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/2739594486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558800f54210 con 0x5588012deca0 2024-06-06T07:55:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c259c640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8033000 tx=0x7fc1b802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9ef7ff7cdbd071 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8035050 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc1b8031950 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc09f8a0 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1b805b050 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1cb801740 10 client.16803 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558801020620 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.260+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc09f8a0 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc1b803d3f0 con 0x5588012deca0 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 10 client.16803 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 20 client.16803 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588012f9710 0x5588012fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 4 client.16803 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558801253f10 con 0x5588012f9710 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1c259c640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588012f9710 0x5588012fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1c259c640 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588012f9710 0x5588012fbb00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2570b587ff79bb04 server_cookie=148da2b26b35466 in_seq=0 out_seq=0 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.261+0000 7fc1a9ffb640 10 client.16803 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/2739594486 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558801253f10 con 0x5588012f9710 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1a9ffb640 10 client.16803 handle_command_reply: tid=0 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 client.16803 shutdown 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 msgr2=0x5588012e6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8033000 tx=0x7fc1b802a420 comp rx=0 tx=0).stop 2024-06-06T07:55:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 shutdown_connections 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588012f9710 0x5588012fbb00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e6a60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1a9ffb640 0 client.16803 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 --2- 172.21.15.87:0/2739594486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 >> 172.21.15.87:0/2739594486 conn(0x5588012da890 msgr2=0x5588012d77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.262+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 shutdown_connections 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.263+0000 7fc1cb801740 1 -- 172.21.15.87:0/2739594486 wait complete. 2024-06-06T07:55:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.263+0000 7fc1cb801740 20 client.16803 trim_cache size 0 max 16384 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 Processor -- start 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 -- start start 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41af6c0 con 0x5588012deca0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41acf90 con 0x5588012de380 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1cb801740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41aae00 con 0x5588012e9560 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45230/0 (socket says 172.21.15.87:45230) 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3160071889 learned_addr learned my addr 172.21.15.87:0/3160071889 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c259c640 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3160071889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 msgr2=0x5588012f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3160071889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 msgr2=0x5588012e0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3160071889 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588012e54f0 con 0x5588012deca0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.278+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc1b803d810 tx=0x7fc1b8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325500c68bf883ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1abfff640 1 -- 172.21.15.87:0/3160071889 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8035070 con 0x5588012deca0 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1abfff640 1 -- 172.21.15.87:0/3160071889 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b803da00 con 0x5588012deca0 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1abfff640 1 -- 172.21.15.87:0/3160071889 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80528e0 con 0x5588012deca0 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- 172.21.15.87:0/3160071889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 msgr2=0x5588012d97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc1b803d810 tx=0x7fc1b8005a40 comp rx=0 tx=0).stop 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- 172.21.15.87:0/3160071889 shutdown_connections 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012e9560 0x5588012f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012d97f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3160071889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012de380 0x5588012e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- 172.21.15.87:0/3160071889 >> 172.21.15.87:0/3160071889 conn(0x5588012da890 msgr2=0x5588012dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- 172.21.15.87:0/3160071889 shutdown_connections 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- 172.21.15.87:0/3160071889 wait complete. 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.279+0000 7fc1cb801740 1 -- start start 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 Processor -- start 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012de380 0x5588012e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012e9560 0x5588012e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ab450 con 0x5588012deca0 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ac8c0 con 0x5588012e9560 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41abb20 con 0x5588012de380 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1cb801740 10 client.0 _ll_register_callbacks cb 0x7fc1b8003740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45238/0 (socket says 172.21.15.87:45238) 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3870774214 learned_addr learned my addr 172.21.15.87:0/3870774214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c259c640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012de380 0x5588012e29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c2d9d640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012e9560 0x5588012e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3870774214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012de380 msgr2=0x5588012e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012de380 0x5588012e29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012e9560 msgr2=0x5588012e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012e9560 0x5588012e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 -- 172.21.15.87:0/3870774214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588012fbe70 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1c1d9b640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8004470 tx=0x7fc1b8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c4c90847c2d58f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.280+0000 7fc1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8035040 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc1b80527e0 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b803da00 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1b803dbb0 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 10 client.16809 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588013669e0 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b803da00 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc1b805e030 con 0x5588012deca0 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 10 client.16809 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 20 client.16809 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558801366f90 0x558801369380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 4 client.16809 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55880124dc20 con 0x558801366f90 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.281+0000 7fc1c259c640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558801366f90 0x558801369380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.282+0000 7fc1c259c640 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558801366f90 0x558801369380 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2604be418c81 server_cookie=4f672461bb6fc0ba in_seq=0 out_seq=0 2024-06-06T07:55:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.282+0000 7fc1a9ffb640 10 client.16809 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.447+0000 7fc1a9ffb640 1 -- 172.21.15.87:0/3870774214 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55880124dc20 con 0x558801366f90 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.447+0000 7fc1a9ffb640 10 client.16809 handle_command_reply: tid=0 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.447+0000 7fc1cb801740 1 client.16809 shutdown 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.447+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 msgr2=0x5588012e2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.447+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8004470 tx=0x7fc1b8005bd0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 shutdown_connections 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558801366f90 0x558801369380 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588012e9560 0x5588012e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1a9ffb640 0 client.16809 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588012deca0 0x5588012e2ee0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 --2- 172.21.15.87:0/3870774214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588012de380 0x5588012e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 >> 172.21.15.87:0/3870774214 conn(0x5588012da890 msgr2=0x5588012e6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 shutdown_connections 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 1 -- 172.21.15.87:0/3870774214 wait complete. 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1cb801740 20 client.16809 trim_cache size 0 max 16384 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:55:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 msgr2=0x7fc1ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 0x7fc1ac0a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009900 tx=0x7fc1b80043e0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 msgr2=0x7fc1c419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 0x7fc1c419b910 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c710 tx=0x7fc1bc00cbd0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 shutdown_connections 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc1ac0a2da0 0x7fc1ac0a5260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c41069c0 0x7fc1c419b910 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4101540 0x7fc1c4197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 --2- 172.21.15.87:0/1965736065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1c4100b80 0x7fc1c4197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.448+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 >> 172.21.15.87:0/1965736065 conn(0x7fc1c40775f0 msgr2=0x7fc1c40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.449+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 shutdown_connections 2024-06-06T07:55:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:46.449+0000 7fc1c8af5640 1 -- 172.21.15.87:0/1965736065 wait complete. 2024-06-06T07:55:46.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.05 MB/sec execute 230 sec latency 5606.052 ms 2024-06-06T07:55:47.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 7.01 MB/sec execute 231 sec latency 6606.131 ms 2024-06-06T07:55:48.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:47 smithi181 ceph-mon[36790]: pgmap v238: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-06-06T07:55:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:47 smithi087 ceph-mon[32276]: pgmap v238: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-06-06T07:55:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:47 smithi195 ceph-mon[37040]: pgmap v238: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-06-06T07:55:48.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.98 MB/sec execute 232 sec latency 7606.234 ms 2024-06-06T07:55:49.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.95 MB/sec execute 233 sec latency 8606.336 ms 2024-06-06T07:55:50.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:49 smithi181 ceph-mon[36790]: pgmap v239: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-06T07:55:50.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:49 smithi087 ceph-mon[32276]: pgmap v239: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-06T07:55:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:49 smithi195 ceph-mon[37040]: pgmap v239: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-06-06T07:55:50.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.92 MB/sec execute 234 sec latency 9606.382 ms 2024-06-06T07:55:51.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.90 MB/sec execute 235 sec latency 10606.471 ms 2024-06-06T07:55:52.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:51 smithi181 ceph-mon[36790]: pgmap v240: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1022 KiB/s wr, 148 op/s 2024-06-06T07:55:52.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:51 smithi087 ceph-mon[32276]: pgmap v240: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1022 KiB/s wr, 148 op/s 2024-06-06T07:55:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:51 smithi195 ceph-mon[37040]: pgmap v240: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1022 KiB/s wr, 148 op/s 2024-06-06T07:55:52.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.87 MB/sec execute 236 sec latency 11606.539 ms 2024-06-06T07:55:53.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.84 MB/sec execute 237 sec latency 12606.580 ms 2024-06-06T07:55:54.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:53 smithi181 ceph-mon[36790]: pgmap v241: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-06T07:55:54.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:53 smithi087 ceph-mon[32276]: pgmap v241: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-06T07:55:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:53 smithi195 ceph-mon[37040]: pgmap v241: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-06-06T07:55:54.670 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.81 MB/sec execute 238 sec latency 13606.620 ms 2024-06-06T07:55:55.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.78 MB/sec execute 239 sec latency 14606.712 ms 2024-06-06T07:55:56.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:55 smithi087 ceph-mon[32276]: pgmap v242: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 543 KiB/s wr, 93 op/s 2024-06-06T07:55:56.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:55 smithi195 ceph-mon[37040]: pgmap v242: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 543 KiB/s wr, 93 op/s 2024-06-06T07:55:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:55 smithi181 ceph-mon[36790]: pgmap v242: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 543 KiB/s wr, 93 op/s 2024-06-06T07:55:56.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.75 MB/sec execute 240 sec latency 15606.825 ms 2024-06-06T07:55:57.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.72 MB/sec execute 241 sec latency 16606.875 ms 2024-06-06T07:55:58.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:57 smithi087 ceph-mon[32276]: pgmap v243: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-06T07:55:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:57 smithi195 ceph-mon[37040]: pgmap v243: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-06T07:55:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:57 smithi181 ceph-mon[36790]: pgmap v243: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-06T07:55:58.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.70 MB/sec execute 242 sec latency 17606.928 ms 2024-06-06T07:55:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 -- 172.21.15.87:0/156932010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc1015b0 msgr2=0x7f8ecc10d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 --2- 172.21.15.87:0/156932010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc10d6e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ca0 tx=0x7f8ec402a8c0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 -- 172.21.15.87:0/156932010 shutdown_connections 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 --2- 172.21.15.87:0/156932010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc10d6e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 --2- 172.21.15.87:0/156932010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 --2- 172.21.15.87:0/156932010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 0x7f8ecc105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 -- 172.21.15.87:0/156932010 >> 172.21.15.87:0/156932010 conn(0x7f8ecc0766a0 msgr2=0x7f8ecc076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 -- 172.21.15.87:0/156932010 shutdown_connections 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.547+0000 7f8ed1163640 1 -- 172.21.15.87:0/156932010 wait complete. 2024-06-06T07:55:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 Processor -- start 2024-06-06T07:55:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 -- start start 2024-06-06T07:55:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 0x7f8ecc1a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc110c90 con 0x7f8ecc100bf0 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc110b10 con 0x7f8ecc105850 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.548+0000 7f8ed1163640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc110e10 con 0x7f8ecc1015b0 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36222/0 (socket says 172.21.15.87:36222) 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 -- 172.21.15.87:0/2626713648 learned_addr learned my addr 172.21.15.87:0/2626713648 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 0x7f8ecc1a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8eca59c640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 -- 172.21.15.87:0/2626713648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 msgr2=0x7f8ecc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 -- 172.21.15.87:0/2626713648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 msgr2=0x7f8ecc1a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 0x7f8ecc1a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 -- 172.21.15.87:0/2626713648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1a4390 con 0x7f8ecc100bf0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8eca59c640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c920 tx=0x7f8ebc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ec3fff640 1 -- 172.21.15.87:0/2626713648 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc007c70 con 0x7f8ecc100bf0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ec3fff640 1 -- 172.21.15.87:0/2626713648 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc007e10 con 0x7f8ecc100bf0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1a4560 con 0x7f8ecc100bf0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ec3fff640 1 -- 172.21.15.87:0/2626713648 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc018890 con 0x7f8ecc100bf0 2024-06-06T07:55:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.549+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1a4ab0 con 0x7f8ecc100bf0 2024-06-06T07:55:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ec3fff640 1 -- 172.21.15.87:0/2626713648 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8ebc018a70 con 0x7f8ecc100bf0 2024-06-06T07:55:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 Processor -- start 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 -- start start 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10170 0x555cfbd10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd1ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555cfbc744f0 con 0x555cfbd1b350 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555cfbc12870 con 0x555cfbd10a90 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.552+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555cfbc704b0 con 0x555cfbd10170 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ec3fff640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 0x7f8eb80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36236/0 (socket says 172.21.15.87:36236) 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 -- 172.21.15.87:0/4151607034 learned_addr learned my addr 172.21.15.87:0/4151607034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10170 0x555cfbd10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd1ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 0x7f8eb80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 -- 172.21.15.87:0/4151607034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10170 msgr2=0x555cfbd10550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10170 0x555cfbd10550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 -- 172.21.15.87:0/4151607034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 msgr2=0x555cfbd1ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ec3fff640 1 -- 172.21.15.87:0/2626713648 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f8ebc0cf970 con 0x7f8ecc100bf0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd1ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 -- 172.21.15.87:0/4151607034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfbc2e9b0 con 0x555cfbd1b350 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd1ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8eca59c640 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4008c10 tx=0x7f8eb400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9897771f1dbff5ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 0x7f8eb80a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005fe0 tx=0x7f8ec40095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.553+0000 7f8ec17fa640 1 -- 172.21.15.87:0/4151607034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4014070 con 0x555cfbd1b350 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ec17fa640 1 -- 172.21.15.87:0/4151607034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb40047a0 con 0x555cfbd1b350 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ec17fa640 1 -- 172.21.15.87:0/4151607034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4010690 con 0x555cfbd1b350 2024-06-06T07:55:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 -- 172.21.15.87:0/4151607034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 msgr2=0x555cfbd1d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4008c10 tx=0x7f8eb400e720 comp rx=0 tx=0).stop 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 -- 172.21.15.87:0/4151607034 shutdown_connections 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1d740 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd1ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/4151607034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10170 0x555cfbd10550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 -- 172.21.15.87:0/4151607034 >> 172.21.15.87:0/4151607034 conn(0x555cfbd0c680 msgr2=0x555cfbd0ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 -- 172.21.15.87:0/4151607034 shutdown_connections 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.554+0000 7f8ed3e73740 1 -- 172.21.15.87:0/4151607034 wait complete. 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 -- start start 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 Processor -- start 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 0x555cfbd18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555cfbd1df80 con 0x555cfbd10170 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555cfbbfdae0 con 0x555cfbd10a90 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555cfbd1e100 con 0x555cfbd1b350 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ed3e73740 10 client.0 _ll_register_callbacks cb 0x555cfbc369b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36252/0 (socket says 172.21.15.87:36252) 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecad9d640 1 -- 172.21.15.87:0/517710471 learned_addr learned my addr 172.21.15.87:0/517710471 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 0x555cfbd18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8eca59c640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecad9d640 1 -- 172.21.15.87:0/517710471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 msgr2=0x555cfbd18cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.555+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 0x555cfbd18cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ecad9d640 1 -- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 msgr2=0x555cfbd18790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd18790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ecad9d640 1 -- 172.21.15.87:0/517710471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfb988210 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 0x555cfbd18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eca59c640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd18790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc09ba00 tx=0x7f8ebc06b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84add2dbcd279996 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc09ba70 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f8ebc00fb30 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb40047a0 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ebc09fe20 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 10 client.16821 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cfba54620 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb40047a0 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8ebc0960f0 con 0x555cfbd10170 2024-06-06T07:55:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 20 client.16821 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd2b4e0 0x555cfbd2d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 4 client.16821 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.556+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cfbc87f10 con 0x555cfbd2b4e0 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.557+0000 7f8eca59c640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd2b4e0 0x555cfbd2d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.557+0000 7f8eb27fc640 10 client.16821 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.557+0000 7f8eca59c640 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd2b4e0 0x555cfbd2d8d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c1109dbd510a56 server_cookie=2a007e5064c86327 in_seq=0 out_seq=0 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8eb27fc640 1 -- 172.21.15.87:0/517710471 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555cfbc87f10 con 0x555cfbd2b4e0 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8eb27fc640 10 client.16821 handle_command_reply: tid=0 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 client.16821 shutdown 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 msgr2=0x555cfbd18250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc09ba00 tx=0x7f8ebc06b8d0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 shutdown_connections 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd2b4e0 0x555cfbd2d8d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd1b350 0x555cfbd18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10a90 0x555cfbd18790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8eb27fc640 0 client.16821 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/517710471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd10170 0x555cfbd18250 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 >> 172.21.15.87:0/517710471 conn(0x555cfbd0c680 msgr2=0x555cfbd123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 shutdown_connections 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 1 -- 172.21.15.87:0/517710471 wait complete. 2024-06-06T07:55:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.558+0000 7f8ed3e73740 20 client.16821 trim_cache size 0 max 16384 2024-06-06T07:55:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.573+0000 7f8ed3e73740 1 Processor -- start 2024-06-06T07:55:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.573+0000 7f8ed3e73740 1 -- start start 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b1700 con 0x555cfbd1b350 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1aefd0 con 0x555cfbd10170 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1ace40 con 0x555cfbd10a90 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36262/0 (socket says 172.21.15.87:36262) 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 -- 172.21.15.87:0/2614388529 learned_addr learned my addr 172.21.15.87:0/2614388529 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8eca59c640 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 -- 172.21.15.87:0/2614388529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 msgr2=0x555cfbd331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 -- 172.21.15.87:0/2614388529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 msgr2=0x555cfbd12230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd12230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 -- 172.21.15.87:0/2614388529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfbd2a080 con 0x555cfbd1b350 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.574+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4035000 tx=0x7f8ec403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb81ffcbb97c227f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ec0ff9640 1 -- 172.21.15.87:0/2614388529 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40305c0 con 0x555cfbd1b350 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ec0ff9640 1 -- 172.21.15.87:0/2614388529 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4036070 con 0x555cfbd1b350 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ec0ff9640 1 -- 172.21.15.87:0/2614388529 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec402fcf0 con 0x555cfbd1b350 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 -- 172.21.15.87:0/2614388529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 msgr2=0x555cfbd336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4035000 tx=0x7f8ec403b000 comp rx=0 tx=0).stop 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 -- 172.21.15.87:0/2614388529 shutdown_connections 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd336e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/2614388529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd12230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 -- 172.21.15.87:0/2614388529 >> 172.21.15.87:0/2614388529 conn(0x555cfbd0c680 msgr2=0x555cfbd11b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 -- 172.21.15.87:0/2614388529 shutdown_connections 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.575+0000 7f8ed3e73740 1 -- 172.21.15.87:0/2614388529 wait complete. 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 -- start start 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 Processor -- start 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd23be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1ad490 con 0x555cfbd1b350 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1ae900 con 0x555cfbd10170 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1adb60 con 0x555cfbd10a90 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ed3e73740 10 client.0 _ll_register_callbacks cb 0x7f8ebc09b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36266/0 (socket says 172.21.15.87:36266) 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 -- 172.21.15.87:0/1099326462 learned_addr learned my addr 172.21.15.87:0/1099326462 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 -- 172.21.15.87:0/1099326462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 msgr2=0x555cfbd1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 -- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 msgr2=0x555cfbd23be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd23be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.576+0000 7f8ecb59e640 1 -- 172.21.15.87:0/1099326462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfbd1fd90 con 0x555cfbd1b350 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ecb59e640 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec402f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1955750b8d991376 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:55:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec402fe40 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f8ec40305c0 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4036070 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ec4067050 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 10 client.16827 fetch_fsmap learned FSMap version 32 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cfbd962b0 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4036070 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec404c860 con 0x555cfbd1b350 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 10 client.16827 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 20 client.16827 populate_metadata read hostname 'smithi087' 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd96970 0x555cfbd98df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:55:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 4 client.16827 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:55:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.577+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cfbc81c20 con 0x555cfbd96970 2024-06-06T07:55:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.578+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd96970 0x555cfbd98df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:55:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.578+0000 7f8eb27fc640 10 client.16827 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.578+0000 7f8ecad9d640 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd96970 0x555cfbd98df0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42af54da7f1ba1a9 server_cookie=ab48f939daa375a7 in_seq=0 out_seq=0 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8eb27fc640 1 -- 172.21.15.87:0/1099326462 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555cfbc81c20 con 0x555cfbd96970 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8eb27fc640 10 client.16827 handle_command_reply: tid=0 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 client.16827 shutdown 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 msgr2=0x555cfbd1f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4006010 tx=0x7f8ec402f610 comp rx=0 tx=0).stop 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 shutdown_connections 2024-06-06T07:55:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cfbd96970 0x555cfbd98df0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cfbd1b350 0x555cfbd1f560 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cfbd10a90 0x555cfbd1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 --2- 172.21.15.87:0/1099326462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cfbd10170 0x555cfbd23be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8eb27fc640 0 client.16827 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 >> 172.21.15.87:0/1099326462 conn(0x555cfbd0c680 msgr2=0x555cfbd123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 shutdown_connections 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 1 -- 172.21.15.87:0/1099326462 wait complete. 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed3e73740 20 client.16827 trim_cache size 0 max 16384 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:55:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:55:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:55:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 msgr2=0x7f8eb80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 0x7f8eb80a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005fe0 tx=0x7f8ec40095c0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 msgr2=0x7f8ecc198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.615+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c920 tx=0x7f8ebc00cde0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 shutdown_connections 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8eb80a2da0 0x7f8eb80a5260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc105850 0x7f8ecc1a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ecc1015b0 0x7f8ecc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 --2- 172.21.15.87:0/2626713648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc100bf0 0x7f8ecc198dd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 >> 172.21.15.87:0/2626713648 conn(0x7f8ecc0766a0 msgr2=0x7f8ecc0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 shutdown_connections 2024-06-06T07:55:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:55:59.616+0000 7f8ed1163640 1 -- 172.21.15.87:0/2626713648 wait complete. 2024-06-06T07:55:59.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.67 MB/sec execute 243 sec latency 18607.017 ms 2024-06-06T07:56:00.062 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:55:59 smithi195 ceph-mon[37040]: pgmap v244: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T07:56:00.108 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:55:59 smithi181 ceph-mon[36790]: pgmap v244: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T07:56:00.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:55:59 smithi087 ceph-mon[32276]: pgmap v244: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T07:56:00.540 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:56:00.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.64 MB/sec execute 244 sec latency 19607.082 ms 2024-06-06T07:56:00.804 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:01.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.61 MB/sec execute 245 sec latency 20607.159 ms 2024-06-06T07:56:02.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:01 smithi087 ceph-mon[32276]: pgmap v245: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.0 MiB/s wr, 43 op/s 2024-06-06T07:56:02.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:01 smithi195 ceph-mon[37040]: pgmap v245: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.0 MiB/s wr, 43 op/s 2024-06-06T07:56:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:01 smithi181 ceph-mon[36790]: pgmap v245: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.0 MiB/s wr, 43 op/s 2024-06-06T07:56:02.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.59 MB/sec execute 246 sec latency 21607.211 ms 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 -- 172.21.15.87:0/2605822012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101380 msgr2=0x7f2bcc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 --2- 172.21.15.87:0/2605822012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101380 0x7f2bcc101800 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009990 tx=0x7f2bc0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 -- 172.21.15.87:0/2605822012 shutdown_connections 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 --2- 172.21.15.87:0/2605822012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101d40 0x7f2bcc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 --2- 172.21.15.87:0/2605822012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 0x7f2bcc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 --2- 172.21.15.87:0/2605822012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101380 0x7f2bcc101800 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 -- 172.21.15.87:0/2605822012 >> 172.21.15.87:0/2605822012 conn(0x7f2bcc078300 msgr2=0x7f2bcc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.760+0000 7f2bd250c700 1 -- 172.21.15.87:0/2605822012 shutdown_connections 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 -- 172.21.15.87:0/2605822012 wait complete. 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 Processor -- start 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 -- start start 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101380 0x7f2bcc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 0x7f2bcc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.761+0000 7f2bd250c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1116e0 con 0x7f2bcc101d40 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bd250c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc111560 con 0x7f2bcc101380 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bd250c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc111860 con 0x7f2bcc1073f0 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36280/0 (socket says 172.21.15.87:36280) 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 -- 172.21.15.87:0/819684270 learned_addr learned my addr 172.21.15.87:0/819684270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:02.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bd0aa9700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 0x7f2bcc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcbfff700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101380 0x7f2bcc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 -- 172.21.15.87:0/819684270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 msgr2=0x7f2bcc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 0x7f2bcc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 -- 172.21.15.87:0/819684270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101380 msgr2=0x7f2bcc19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101380 0x7f2bcc19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 -- 172.21.15.87:0/819684270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc199180 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bcb7fe700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0027070 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc199410 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc0004190 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000f460 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.762+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc1a5730 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.763+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc069180 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.765+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2bc002a070 con 0x7f2bcc101d40 2024-06-06T07:56:02.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.765+0000 7f2bc97fa700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 0x7f2bb40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:02.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.765+0000 7f2bcbfff700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 0x7f2bb40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:02.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.765+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2bc00d0c00 con 0x7f2bcc101d40 2024-06-06T07:56:02.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.766+0000 7f2bcbfff700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 0x7f2bb40a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005950 tx=0x7f2bbc005ed0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:02.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:02.769+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2bc0013070 con 0x7f2bcc101d40 2024-06-06T07:56:03.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.011+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bcc199f50 con 0x7f2bcc101d40 2024-06-06T07:56:03.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.012+0000 7f2bc97fa700 1 -- 172.21.15.87:0/819684270 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7f2bcc199f50 con 0x7f2bcc101d40 2024-06-06T07:56:03.011 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:56:03.011 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:56:03.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 msgr2=0x7f2bb40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 0x7f2bb40a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005950 tx=0x7f2bbc005ed0 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 msgr2=0x7f2bcc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0003730 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 shutdown_connections 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bcc101380 0x7f2bcc19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2bb40a2d90 0x7f2bb40a5250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bcc1073f0 0x7f2bcc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 --2- 172.21.15.87:0/819684270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bcc101d40 0x7f2bcc19eda0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 >> 172.21.15.87:0/819684270 conn(0x7f2bcc078300 msgr2=0x7f2bcc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 shutdown_connections 2024-06-06T07:56:03.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:03.014+0000 7f2bd250c700 1 -- 172.21.15.87:0/819684270 wait complete. 2024-06-06T07:56:03.015 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:56:03.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.56 MB/sec execute 247 sec latency 22607.514 ms 2024-06-06T07:56:03.762 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 1c04fd38 --release --await' 2024-06-06T07:56:03.775 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 1c04fd38 --release --await 2024-06-06T07:56:04.015 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:03 smithi087 ceph-mon[32276]: pgmap v246: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-06-06T07:56:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/819684270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:03 smithi195 ceph-mon[37040]: pgmap v246: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-06-06T07:56:04.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/819684270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:03 smithi181 ceph-mon[36790]: pgmap v246: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-06-06T07:56:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/819684270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:04.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.53 MB/sec execute 248 sec latency 23607.555 ms 2024-06-06T07:56:05.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 425136 6.51 MB/sec execute 249 sec latency 24607.595 ms 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 -- 172.21.15.87:0/2785441092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 msgr2=0x7fd470107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 --2- 172.21.15.87:0/2785441092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd470107110 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd4640099c0 tx=0x7fd464009cd0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 -- 172.21.15.87:0/2785441092 shutdown_connections 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 --2- 172.21.15.87:0/2785441092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470107840 0x7fd47010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 --2- 172.21.15.87:0/2785441092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470102740 0x7fd470102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 --2- 172.21.15.87:0/2785441092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd470107110 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 -- 172.21.15.87:0/2785441092 >> 172.21.15.87:0/2785441092 conn(0x7fd4700795f0 msgr2=0x7fd470079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 -- 172.21.15.87:0/2785441092 shutdown_connections 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.928+0000 7fd476099700 1 -- 172.21.15.87:0/2785441092 wait complete. 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 Processor -- start 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 -- start start 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470102740 0x7fd47019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 0x7fd470198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701127d0 con 0x7fd4701030f0 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112650 con 0x7fd470102740 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.929+0000 7fd476099700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112950 con 0x7fd470107840 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36302/0 (socket says 172.21.15.87:36302) 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 -- 172.21.15.87:0/3717514557 learned_addr learned my addr 172.21.15.87:0/3717514557 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470102740 0x7fd47019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46ffff700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 0x7fd470198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 -- 172.21.15.87:0/3717514557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 msgr2=0x7fd470198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 0x7fd470198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 -- 172.21.15.87:0/3717514557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470102740 msgr2=0x7fd47019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470102740 0x7fd47019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 -- 172.21.15.87:0/3717514557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd470199190 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46ffff700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 0x7fd470198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46effd700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd464009fd0 tx=0x7fd464003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46cff9700 1 -- 172.21.15.87:0/3717514557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464027070 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46cff9700 1 -- 172.21.15.87:0/3717514557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4640040d0 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd470199420 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd46cff9700 1 -- 172.21.15.87:0/3717514557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46400f460 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.930+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a5740 con 0x7fd4701030f0 2024-06-06T07:56:05.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.933+0000 7fd46cff9700 1 -- 172.21.15.87:0/3717514557 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd46402a070 con 0x7fd4701030f0 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.934+0000 7fd46cff9700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 0x7fd4580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.934+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 0x7fd4580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.934+0000 7fd46cff9700 1 -- 172.21.15.87:0/3717514557 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd4640d0860 con 0x7fd4701030f0 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.934+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 0x7fd4580a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd470199b70 tx=0x7fd460009190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 Processor -- start 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 -- start start 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d62020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d5c3d0 0x558576d5c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558576cad910 con 0x558576d62560 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558576c55820 con 0x558576d5c3d0 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd48091cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558576ca98d0 con 0x558576d61c40 2024-06-06T07:56:05.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36314/0 (socket says 172.21.15.87:36314) 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 -- 172.21.15.87:0/4166972262 learned_addr learned my addr 172.21.15.87:0/4166972262 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d62020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 -- 172.21.15.87:0/4166972262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 msgr2=0x558576d62020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d62020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 -- 172.21.15.87:0/4166972262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d5c3d0 msgr2=0x558576d5c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d5c3d0 0x558576d5c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 -- 172.21.15.87:0/4166972262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558576c4c050 con 0x558576d62560 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.935+0000 7fd46effd700 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd4640993c0 tx=0x7fd4640228f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc39b97932ad6a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd455ffb700 1 -- 172.21.15.87:0/4166972262 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4640a55d0 con 0x558576d62560 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd455ffb700 1 -- 172.21.15.87:0/4166972262 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd464099060 con 0x558576d62560 2024-06-06T07:56:05.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd455ffb700 1 -- 172.21.15.87:0/4166972262 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4640a5140 con 0x558576d62560 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 -- 172.21.15.87:0/4166972262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 msgr2=0x558576d5be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd4640993c0 tx=0x7fd4640228f0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 -- 172.21.15.87:0/4166972262 shutdown_connections 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d5c3d0 0x558576d5c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d62020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 --2- 172.21.15.87:0/4166972262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d5be90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 -- 172.21.15.87:0/4166972262 >> 172.21.15.87:0/4166972262 conn(0x558576c86dc0 msgr2=0x558576d57120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.936+0000 7fd48091cb80 1 -- 172.21.15.87:0/4166972262 shutdown_connections 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 -- 172.21.15.87:0/4166972262 wait complete. 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 -- start start 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 Processor -- start 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d5c3d0 0x558576d68330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d65630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558576d68cf0 con 0x558576d62560 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558576c4dcf0 con 0x558576d61c40 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558576d68e70 con 0x558576d5c3d0 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd48091cb80 10 client.0 _ll_register_callbacks cb 0x558576c47a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36326/0 (socket says 172.21.15.87:36326) 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.937+0000 7fd46ffff700 1 -- 172.21.15.87:0/2876469412 learned_addr learned my addr 172.21.15.87:0/2876469412 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d5c3d0 0x558576d68330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46effd700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d65630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 -- 172.21.15.87:0/2876469412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d5c3d0 msgr2=0x558576d68330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d5c3d0 0x558576d68330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 -- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 msgr2=0x558576d65630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d65630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 -- 172.21.15.87:0/2876469412 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558576bfc9d0 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd46ffff700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd468008b60 tx=0x7fd468008e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1568dc3edcf7e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd44effd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46800d9e0 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd46800db80 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd464099060 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd468021050 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd48091cb80 10 client.16845 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558576d73650 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.938+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd464099060 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd46801c3b0 con 0x558576d62560 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 10 client.16845 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 10 client.16845 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:05.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 20 client.16845 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576d75c90 0x558576d780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 4 client.16845 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558576cbae10 con 0x558576d75c90 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576d75c90 0x558576d780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576d75c90 0x558576d780b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac2c5e587935a98 server_cookie=650f721b1ddc7cb6 in_seq=0 out_seq=0 2024-06-06T07:56:05.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.939+0000 7fd44effd700 10 client.16845 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd44effd700 1 -- 172.21.15.87:0/2876469412 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558576cbae10 con 0x558576d75c90 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd44effd700 10 client.16845 handle_command_reply: tid=0 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd48091cb80 1 client.16845 shutdown 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 msgr2=0x558576d65b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd468008b60 tx=0x7fd468008e70 comp rx=0 tx=0).stop 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 shutdown_connections 2024-06-06T07:56:05.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.940+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576d75c90 0x558576d780b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d62560 0x558576d65b70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd44effd700 0 client.16845 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d5c3d0 0x558576d68330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2876469412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 >> 172.21.15.87:0/2876469412 conn(0x558576c86dc0 msgr2=0x558576d5ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 shutdown_connections 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 1 -- 172.21.15.87:0/2876469412 wait complete. 2024-06-06T07:56:05.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.941+0000 7fd48091cb80 20 client.16845 trim_cache size 0 max 16384 2024-06-06T07:56:05.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 Processor -- start 2024-06-06T07:56:05.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 -- start start 2024-06-06T07:56:05.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d62560 0x558576d62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b1000 con 0x558576d5c3d0 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701ae890 con 0x558576d61c40 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd48091cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701ac700 con 0x558576d62560 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36336/0 (socket says 172.21.15.87:36336) 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 -- 172.21.15.87:0/606373277 learned_addr learned my addr 172.21.15.87:0/606373277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46ffff700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d62560 0x558576d62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46effd700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 -- 172.21.15.87:0/606373277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d62560 msgr2=0x558576d62f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d62560 0x558576d62f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 -- 172.21.15.87:0/606373277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 msgr2=0x558576d62a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d62a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46f7fe700 1 -- 172.21.15.87:0/606373277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558576d68930 con 0x558576d5c3d0 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.961+0000 7fd46effd700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e4b0 tx=0x7fd460013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df82801fe34cdf01 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd4557fa700 1 -- 172.21.15.87:0/606373277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46001b040 con 0x558576d5c3d0 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd4557fa700 1 -- 172.21.15.87:0/606373277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd460015960 con 0x558576d5c3d0 2024-06-06T07:56:05.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd4557fa700 1 -- 172.21.15.87:0/606373277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600299c0 con 0x558576d5c3d0 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 -- 172.21.15.87:0/606373277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 msgr2=0x558576d5fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e4b0 tx=0x7fd460013000 comp rx=0 tx=0).stop 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 -- 172.21.15.87:0/606373277 shutdown_connections 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d62560 0x558576d62f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d5fdd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 --2- 172.21.15.87:0/606373277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d61c40 0x558576d62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.962+0000 7fd48091cb80 1 -- 172.21.15.87:0/606373277 >> 172.21.15.87:0/606373277 conn(0x558576c86dc0 msgr2=0x558576d5f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- 172.21.15.87:0/606373277 shutdown_connections 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- 172.21.15.87:0/606373277 wait complete. 2024-06-06T07:56:05.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- start start 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 Processor -- start 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d62560 0x558576d6bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701acd50 con 0x558576d5c3d0 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701ae1c0 con 0x558576d62560 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701ad420 con 0x558576d61c40 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd48091cb80 10 client.0 _ll_register_callbacks cb 0x558576c47a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.963+0000 7fd46f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36346/0 (socket says 172.21.15.87:36346) 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 -- 172.21.15.87:0/2510650471 learned_addr learned my addr 172.21.15.87:0/2510650471 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46ffff700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d62560 0x558576d6bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46effd700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 -- 172.21.15.87:0/2510650471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 msgr2=0x558576d6b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d6b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 -- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d62560 msgr2=0x558576d6bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d62560 0x558576d6bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 -- 172.21.15.87:0/2510650471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558576de4e30 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46effd700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd46f7fe700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd460029450 tx=0x7fd4600292c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce2231051526335 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd44effd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460037070 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd460015960 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd460015d20 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd460044050 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd48091cb80 10 client.16851 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.964+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558576de60e0 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd460015960 con 0x558576d5c3d0 2024-06-06T07:56:05.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd46003b780 con 0x558576d5c3d0 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 10 client.16851 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 10 client.16851 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 20 client.16851 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576de6730 0x558576de8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 4 client.16851 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1c04fd38", "release": true, "await": true}] 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1c04fd38", "release": true, "await": true}) -- 0x558576cb5390 con 0x558576de6730 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd46effd700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576de6730 0x558576de8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd44effd700 10 client.16851 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:05.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.965+0000 7fd46effd700 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576de6730 0x558576de8b50 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea6da5d7df9bb17 server_cookie=fa36a71dcb547b77 in_seq=0 out_seq=0 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd44effd700 1 -- 172.21.15.87:0/2510650471 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558576cb5390 con 0x558576de6730 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd44effd700 10 client.16851 handle_command_reply: tid=0 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 client.16851 shutdown 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 msgr2=0x558576d6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd460029450 tx=0x7fd4600292c0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 shutdown_connections 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558576de6730 0x558576de8b50 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558576d62560 0x558576d6bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558576d5c3d0 0x558576d6b090 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd44effd700 0 client.16851 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 --2- 172.21.15.87:0/2510650471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558576d61c40 0x558576d6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 >> 172.21.15.87:0/2510650471 conn(0x558576c86dc0 msgr2=0x558576d56830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 shutdown_connections 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 1 -- 172.21.15.87:0/2510650471 wait complete. 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.978+0000 7fd48091cb80 20 client.16851 trim_cache size 0 max 16384 2024-06-06T07:56:05.979 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 24, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "1c04fd38": { 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 24, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:56:05.980 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.6, 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:05.981 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 msgr2=0x7fd4580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 0x7fd4580a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd470199b70 tx=0x7fd460009190 comp rx=0 tx=0).stop 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 msgr2=0x7fd47019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd464009fd0 tx=0x7fd464003680 comp rx=0 tx=0).stop 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 shutdown_connections 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd470102740 0x7fd47019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd4580a2d90 0x7fd4580a5250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd470107840 0x7fd470198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 --2- 172.21.15.87:0/3717514557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4701030f0 0x7fd47019edb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 >> 172.21.15.87:0/3717514557 conn(0x7fd4700795f0 msgr2=0x7fd4701006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 shutdown_connections 2024-06-06T07:56:05.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:05.979+0000 7fd476099700 1 -- 172.21.15.87:0/3717514557 wait complete. 2024-06-06T07:56:06.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:05 smithi087 ceph-mon[32276]: pgmap v247: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-06-06T07:56:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:05 smithi195 ceph-mon[37040]: pgmap v247: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-06-06T07:56:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:05 smithi181 ceph-mon[36790]: pgmap v247: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-06-06T07:56:06.671 INFO:tasks.workunit.client.0.smithi087.stdout: 1 426075 6.51 MB/sec execute 250 sec latency 24917.232 ms 2024-06-06T07:56:06.793 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1c04fd38', total seconds elapsed: 31.9 2024-06-06T07:56:06.793 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-06-06T07:56:06.794 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:56:07.064 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:07.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 427655 6.50 MB/sec execute 251 sec latency 50.652 ms 2024-06-06T07:56:08.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:07 smithi087 ceph-mon[32276]: pgmap v248: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-06-06T07:56:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:07 smithi195 ceph-mon[37040]: pgmap v248: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-06-06T07:56:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:07 smithi181 ceph-mon[36790]: pgmap v248: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-06-06T07:56:08.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 429777 6.53 MB/sec execute 252 sec latency 58.551 ms 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 -- 172.21.15.87:0/1607354365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410101d50 msgr2=0x7f841010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 --2- 172.21.15.87:0/1607354365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410101d50 0x7f841010e240 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8400009990 tx=0x7f8400009ca0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 -- 172.21.15.87:0/1607354365 shutdown_connections 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 --2- 172.21.15.87:0/1607354365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410107400 0x7f84101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 --2- 172.21.15.87:0/1607354365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410101d50 0x7f841010e240 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 --2- 172.21.15.87:0/1607354365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 0x7f8410101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.291+0000 7f8417743700 1 -- 172.21.15.87:0/1607354365 >> 172.21.15.87:0/1607354365 conn(0x7f8410078340 msgr2=0x7f8410078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:09.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.292+0000 7f8417743700 1 -- 172.21.15.87:0/1607354365 shutdown_connections 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.292+0000 7f8417743700 1 -- 172.21.15.87:0/1607354365 wait complete. 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.292+0000 7f8417743700 1 Processor -- start 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.292+0000 7f8417743700 1 -- start start 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.292+0000 7f8417743700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 0x7f841019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8417743700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410101d50 0x7f841019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8417743700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8417743700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f84101116f0 con 0x7f8410107400 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8417743700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8410111570 con 0x7f8410101d50 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8417743700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8410111870 con 0x7f8410101390 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8414cde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410101d50 0x7f841019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:09.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f84154df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 0x7f841019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60362/0 (socket says 172.21.15.87:60362) 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 -- 172.21.15.87:0/2248351264 learned_addr learned my addr 172.21.15.87:0/2248351264 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 -- 172.21.15.87:0/2248351264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 msgr2=0x7f841019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 0x7f841019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 -- 172.21.15.87:0/2248351264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410101d50 msgr2=0x7f841019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410101d50 0x7f841019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 -- 172.21.15.87:0/2248351264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8410198050 con 0x7f8410107400 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f8415ce0700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8400005910 tx=0x7f84000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.293+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400027070 con 0x7f8410107400 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.294+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84101982e0 con 0x7f8410107400 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.294+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8400004050 con 0x7f8410107400 2024-06-06T07:56:09.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.294+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400022dd0 con 0x7f8410107400 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.294+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841006a170 con 0x7f8410107400 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.295+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410102ab0 con 0x7f8410107400 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.296+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f840002a070 con 0x7f8410107400 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.297+0000 7f840e7fc700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 0x7f83f80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.297+0000 7f84154df700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 0x7f83f80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.297+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f84000d05e0 con 0x7f8410107400 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.297+0000 7f84154df700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 0x7f83f80a52d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8410198a30 tx=0x7f84080079e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:09.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.301+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f840005e850 con 0x7f8410107400 2024-06-06T07:56:09.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.564+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8410198e20 con 0x7f8410107400 2024-06-06T07:56:09.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.565+0000 7f840e7fc700 1 -- 172.21.15.87:0/2248351264 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11874 (secure 0 0 0) 0x7f8400013070 con 0x7f8410107400 2024-06-06T07:56:09.565 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:56:09.565 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:52:10.586578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/494555841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":494555841},{"type":"v1","addr":"172.21.15.195:6837","nonce":494555841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 msgr2=0x7f83f80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 0x7f83f80a52d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8410198a30 tx=0x7f84080079e0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 msgr2=0x7f84101978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8400005910 tx=0x7f84000047b0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 shutdown_connections 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83f80a2e10 0x7f83f80a52d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8410107400 0x7f84101978f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8410101390 0x7f841019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 --2- 172.21.15.87:0/2248351264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8410101d50 0x7f841019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 >> 172.21.15.87:0/2248351264 conn(0x7f8410078340 msgr2=0x7f84100ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 shutdown_connections 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:09.567+0000 7f8417743700 1 -- 172.21.15.87:0/2248351264 wait complete. 2024-06-06T07:56:09.568 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 32 2024-06-06T07:56:09.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 431825 6.54 MB/sec execute 253 sec latency 24.800 ms 2024-06-06T07:56:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:09 smithi087 ceph-mon[32276]: pgmap v249: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-06-06T07:56:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:09 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2248351264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0' 2024-06-06T07:56:10.252 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0 2024-06-06T07:56:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:09 smithi195 ceph-mon[37040]: pgmap v249: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-06-06T07:56:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:09 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2248351264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:09 smithi181 ceph-mon[36790]: pgmap v249: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-06-06T07:56:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:09 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2248351264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:10.509 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:10.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 433935 6.56 MB/sec execute 254 sec latency 16.566 ms 2024-06-06T07:56:11.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 434660 6.55 MB/sec execute 255 sec latency 173.566 ms 2024-06-06T07:56:11.998 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:11 smithi087 ceph-mon[32276]: pgmap v250: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-06-06T07:56:11.998 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:11 smithi087 ceph-mon[32276]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:56:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:11 smithi195 ceph-mon[37040]: pgmap v250: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-06-06T07:56:12.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:11 smithi195 ceph-mon[37040]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:56:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:11 smithi181 ceph-mon[36790]: pgmap v250: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-06-06T07:56:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:11 smithi181 ceph-mon[36790]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:56:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.456+0000 7fc7b2406700 1 -- 172.21.15.87:0/548345774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 msgr2=0x7fc7ac109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.456+0000 7fc7b2406700 1 --2- 172.21.15.87:0/548345774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac109e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b0d0 tx=0x7fc7a400b3e0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 -- 172.21.15.87:0/548345774 shutdown_connections 2024-06-06T07:56:12.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 --2- 172.21.15.87:0/548345774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7ac11bc70 0x7fc7ac11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 --2- 172.21.15.87:0/548345774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 --2- 172.21.15.87:0/548345774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac109e50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 -- 172.21.15.87:0/548345774 >> 172.21.15.87:0/548345774 conn(0x7fc7ac06d600 msgr2=0x7fc7ac06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.457+0000 7fc7b2406700 1 -- 172.21.15.87:0/548345774 shutdown_connections 2024-06-06T07:56:12.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.458+0000 7fc7b2406700 1 -- 172.21.15.87:0/548345774 wait complete. 2024-06-06T07:56:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 Processor -- start 2024-06-06T07:56:12.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 -- start start 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac11b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7ac11bc70 0x7fc7ac1158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac111ea0 con 0x7fc7ac110ce0 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac111d20 con 0x7fc7ac11bc70 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.459+0000 7fc7b2406700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac112020 con 0x7fc7ac1099d0 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.460+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.460+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60384/0 (socket says 172.21.15.87:60384) 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.460+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/938680085 learned_addr learned my addr 172.21.15.87:0/938680085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.460+0000 7fc7abfff700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac11b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/938680085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 msgr2=0x7fc7ac11b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac11b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/938680085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7ac11bc70 msgr2=0x7fc7ac1158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7ac11bc70 0x7fc7ac1158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/938680085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac115f90 con 0x7fc7ac110ce0 2024-06-06T07:56:12.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.461+0000 7fc7abfff700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac11b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:12.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.462+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007a20 tx=0x7fc7a4007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.463+0000 7fc7a97fa700 1 -- 172.21.15.87:0/938680085 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4021020 con 0x7fc7ac110ce0 2024-06-06T07:56:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.463+0000 7fc7a97fa700 1 -- 172.21.15.87:0/938680085 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4007e10 con 0x7fc7ac110ce0 2024-06-06T07:56:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.463+0000 7fc7a97fa700 1 -- 172.21.15.87:0/938680085 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40053b0 con 0x7fc7ac110ce0 2024-06-06T07:56:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.463+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1bc8f0 con 0x7fc7ac110ce0 2024-06-06T07:56:12.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.463+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1bcc90 con 0x7fc7ac110ce0 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.470+0000 7fc7bcc89b80 1 Processor -- start 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.470+0000 7fc7a97fa700 1 -- 172.21.15.87:0/938680085 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc7a4024020 con 0x7fc7ac110ce0 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.470+0000 7fc7bcc89b80 1 -- start start 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1dcd0 0x555d07d1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1e5f0 0x555d07d17f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555d07c699a0 con 0x555d07d1e5f0 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555d07c118b0 con 0x555d07d18460 2024-06-06T07:56:12.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.471+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d07c65960 con 0x555d07d1dcd0 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7a97fa700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 0x7fc7940a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:59334/0 (socket says 172.21.15.87:59334) 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/1147692513 learned_addr learned my addr 172.21.15.87:0/1147692513 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7a97fa700 1 -- 172.21.15.87:0/938680085 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc7a400f030 con 0x7fc7ac110ce0 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/1147692513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1dcd0 msgr2=0x555d07d1e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:56:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1dcd0 0x555d07d1e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/1147692513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1e5f0 msgr2=0x555d07d17f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:56:12.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1e5f0 0x555d07d17f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.472+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/1147692513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d07c080e0 con 0x555d07d18460 2024-06-06T07:56:12.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.474+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a402a000 tx=0x7fc7a4091b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664a5add9dff4dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.475+0000 7fc7927fc700 1 -- 172.21.15.87:0/1147692513 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a409c040 con 0x555d07d18460 2024-06-06T07:56:12.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.475+0000 7fc7927fc700 1 -- 172.21.15.87:0/1147692513 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4011e50 con 0x555d07d18460 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.475+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 0x7fc7940a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7927fc700 1 -- 172.21.15.87:0/1147692513 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40a52c0 con 0x555d07d18460 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/1147692513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 msgr2=0x555d07d188a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a402a000 tx=0x7fc7a4091b70 comp rx=0 tx=0).stop 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 0x7fc7940a54d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009b30 tx=0x7fc79c009620 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/1147692513 shutdown_connections 2024-06-06T07:56:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d188a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1dcd0 0x555d07d1e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/1147692513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1e5f0 0x555d07d17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/1147692513 >> 172.21.15.87:0/1147692513 conn(0x555d07c42e50 msgr2=0x555d07d131b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/1147692513 shutdown_connections 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.476+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/1147692513 wait complete. 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.477+0000 7fc7bcc89b80 1 -- start start 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.477+0000 7fc7bcc89b80 1 Processor -- start 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d21ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555d07d24d80 con 0x555d07d1dcd0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555d07c09d80 con 0x555d07d18460 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d07d24f00 con 0x555d07d1e5f0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7bcc89b80 10 client.0 _ll_register_callbacks cb 0x555d07c03ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d21760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60400/0 (socket says 172.21.15.87:60400) 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.478+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/3187092597 learned_addr learned my addr 172.21.15.87:0/3187092597 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d21ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 -- 172.21.15.87:0/3187092597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 msgr2=0x555d07d21ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d21ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 -- 172.21.15.87:0/3187092597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 msgr2=0x555d07d21760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d21760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.479+0000 7fc7abfff700 1 -- 172.21.15.87:0/3187092597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d07bb8a60 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.480+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d243f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0010620 tx=0x7fc7a0010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba25256dd0e6e7ed server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.480+0000 7fc78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004230 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc7a00043d0 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4011e50 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7a0003680 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc7bcc89b80 10 client.26042 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.481+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d07d2f8a0 con 0x555d07d18460 2024-06-06T07:56:12.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4011e50 con 0x555d07d18460 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc7a000f040 con 0x555d07d18460 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 10 client.26042 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 20 client.26042 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d31e30 0x555d07d34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 4 client.26042 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.482+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d07c76ea0 con 0x555d07d31e30 2024-06-06T07:56:12.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.483+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d31e30 0x555d07d34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.484+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d31e30 0x555d07d34220 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f31f7ce1642bc08 server_cookie=1254f8f647f66f36 in_seq=0 out_seq=0 2024-06-06T07:56:12.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.484+0000 7fc78b7fe700 10 client.26042 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.486+0000 7fc78b7fe700 1 -- 172.21.15.87:0/3187092597 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d07c76ea0 con 0x555d07d31e30 2024-06-06T07:56:12.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.486+0000 7fc78b7fe700 10 client.26042 handle_command_reply: tid=0 2024-06-06T07:56:12.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.487+0000 7fc7bcc89b80 1 client.26042 shutdown 2024-06-06T07:56:12.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.488+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 msgr2=0x555d07d243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.488+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d243f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0010620 tx=0x7fc7a0010930 comp rx=0 tx=0).stop 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 shutdown_connections 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d31e30 0x555d07d34220 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d243f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d21ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3187092597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d21760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.489+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 >> 172.21.15.87:0/3187092597 conn(0x555d07c42e50 msgr2=0x555d07d12da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.490+0000 7fc78b7fe700 0 client.26042 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.491+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 shutdown_connections 2024-06-06T07:56:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.491+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3187092597 wait complete. 2024-06-06T07:56:12.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.491+0000 7fc7bcc89b80 20 client.26042 trim_cache size 0 max 16384 2024-06-06T07:56:12.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7bcc89b80 1 Processor -- start 2024-06-06T07:56:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7bcc89b80 1 -- start start 2024-06-06T07:56:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d17f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d1f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60404/0 (socket says 172.21.15.87:60404) 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.533+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/3289741020 learned_addr learned my addr 172.21.15.87:0/3289741020 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c87d0 con 0x555d07d1dcd0 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c6060 con 0x555d07d18460 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d17f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c3ed0 con 0x555d07d1e5f0 2024-06-06T07:56:12.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d1f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 -- 172.21.15.87:0/3289741020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 msgr2=0x555d07d17f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d17f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 -- 172.21.15.87:0/3289741020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 msgr2=0x555d07d2e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.534+0000 7fc7b09a3700 1 -- 172.21.15.87:0/3289741020 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d07d1bef0 con 0x555d07d1e5f0 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d1f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ead0 tx=0x7fc79c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32e912effc3b929 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc7abfff700 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc791ffb700 1 -- 172.21.15.87:0/3289741020 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c01d630 con 0x555d07d1e5f0 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc791ffb700 1 -- 172.21.15.87:0/3289741020 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c01dcb0 con 0x555d07d1e5f0 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.535+0000 7fc791ffb700 1 -- 172.21.15.87:0/3289741020 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c010e30 con 0x555d07d1e5f0 2024-06-06T07:56:12.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 msgr2=0x555d07d1f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d1f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ead0 tx=0x7fc79c011000 comp rx=0 tx=0).stop 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 shutdown_connections 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d18460 0x555d07d17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d1f0e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/3289741020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d1dcd0 0x555d07d2e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 >> 172.21.15.87:0/3289741020 conn(0x555d07c42e50 msgr2=0x555d07d1e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.536+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 shutdown_connections 2024-06-06T07:56:12.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.537+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/3289741020 wait complete. 2024-06-06T07:56:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.537+0000 7fc7bcc89b80 1 -- start start 2024-06-06T07:56:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.538+0000 7fc7bcc89b80 1 Processor -- start 2024-06-06T07:56:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.538+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 0x555d07d2a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 0x555d07d2aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7bcc89b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7bcc89b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c4520 con 0x555d07d18460 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 0x555d07d2aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 0x555d07d2a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 0x555d07d2a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60418/0 (socket says 172.21.15.87:60418) 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.539+0000 7fc7abfff700 1 -- 172.21.15.87:0/618589932 learned_addr learned my addr 172.21.15.87:0/618589932 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.540+0000 7fc7b09a3700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.540+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c5990 con 0x555d07d1dcd0 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.540+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1c4bf0 con 0x555d07d1e5f0 2024-06-06T07:56:12.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.540+0000 7fc7bcc89b80 10 client.0 _ll_register_callbacks cb 0x555d07c03ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:12.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.541+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/618589932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 msgr2=0x555d07d272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.541+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.541+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/618589932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 msgr2=0x555d07d2a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.541+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 0x555d07d2a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.541+0000 7fc7ab7fe700 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d07d27ae0 con 0x555d07d1dcd0 2024-06-06T07:56:12.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.542+0000 7fc7ab7fe700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 0x555d07d2aa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x555d07d22220 tx=0x7fc7a40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d9e4b2d10619ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.542+0000 7fc78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:56:12.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.542+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40b9070 con 0x555d07d1dcd0 2024-06-06T07:56:12.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.542+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc7a40074e0 con 0x555d07d1dcd0 2024-06-06T07:56:12.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.543+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c01dcb0 con 0x555d07d1dcd0 2024-06-06T07:56:12.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.544+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7a409cde0 con 0x555d07d1dcd0 2024-06-06T07:56:12.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.544+0000 7fc7bcc89b80 10 client.26045 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:12.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.544+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d07d9d580 con 0x555d07d1dcd0 2024-06-06T07:56:12.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4011e50 con 0x555d07d1dcd0 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc7a4011070 con 0x555d07d1dcd0 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 10 client.26045 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 20 client.26045 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d9da70 0x555d07d9fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 4 client.26045 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}] 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}) -- 0x7fc79c01dcb0 con 0x555d07d9da70 2024-06-06T07:56:12.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.545+0000 7fc7abfff700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d9da70 0x555d07d9fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.547+0000 7fc7abfff700 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d9da70 0x555d07d9fe60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0566d2272d799aa server_cookie=47b80652e66f9d28 in_seq=0 out_seq=0 2024-06-06T07:56:12.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.547+0000 7fc78b7fe700 10 client.26045 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435738 6.55 MB/sec execute 256 sec latency 103.020 ms 2024-06-06T07:56:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.709+0000 7fc78b7fe700 1 -- 172.21.15.87:0/618589932 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fc79c01dcb0 con 0x555d07d9da70 2024-06-06T07:56:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.709+0000 7fc78b7fe700 10 client.26045 handle_command_reply: tid=0 2024-06-06T07:56:12.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.709+0000 7fc7bcc89b80 1 client.26045 shutdown 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.709+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 msgr2=0x555d07d2aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.709+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 0x555d07d2aa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x555d07d22220 tx=0x7fc7a40099e0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 shutdown_connections 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555d07d9da70 0x555d07d9fe60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d07d1e5f0 0x555d07d272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc78b7fe700 0 client.26045 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555d07d18460 0x555d07d2a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 --2- 172.21.15.87:0/618589932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d07d1dcd0 0x555d07d2aa30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 >> 172.21.15.87:0/618589932 conn(0x555d07c42e50 msgr2=0x555d07d23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 shutdown_connections 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 1 -- 172.21.15.87:0/618589932 wait complete. 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.710+0000 7fc7bcc89b80 20 client.26045 trim_cache size 0 max 16384 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 32, 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:56:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 26, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "2fd621c9": { 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 26, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.6, 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:56:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 msgr2=0x7fc7940a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 0x7fc7940a54d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009b30 tx=0x7fc79c009620 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 msgr2=0x7fc7ac1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007a20 tx=0x7fc7a4007b00 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 shutdown_connections 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7ac11bc70 0x7fc7ac1158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7940a3010 0x7fc7940a54d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ac1099d0 0x7fc7ac11b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 --2- 172.21.15.87:0/938680085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac110ce0 0x7fc7ac1153b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 >> 172.21.15.87:0/938680085 conn(0x7fc7ac06d600 msgr2=0x7fc7ac11c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 shutdown_connections 2024-06-06T07:56:12.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:12.711+0000 7fc7b2406700 1 -- 172.21.15.87:0/938680085 wait complete. 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 -- 172.21.15.87:0/735460777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d01008d0 msgr2=0x7fc2d0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/735460777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d01008d0 0x7fc2d0100d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009900 tx=0x7fc2bc02a400 comp rx=0 tx=0).stop 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 -- 172.21.15.87:0/735460777 shutdown_connections 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/735460777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 0x7fc2d010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/735460777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d01008d0 0x7fc2d0100d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/735460777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d0106890 0x7fc2d0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.758+0000 7fc2d6aee640 1 -- 172.21.15.87:0/735460777 >> 172.21.15.87:0/735460777 conn(0x7fc2d0077a20 msgr2=0x7fc2d0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.759+0000 7fc2d6aee640 1 -- 172.21.15.87:0/735460777 shutdown_connections 2024-06-06T07:56:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.760+0000 7fc2d6aee640 1 -- 172.21.15.87:0/735460777 wait complete. 2024-06-06T07:56:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.760+0000 7fc2d6aee640 1 Processor -- start 2024-06-06T07:56:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 -- start start 2024-06-06T07:56:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d01008d0 0x7fc2d01a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 0x7fc2d01a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0110cf0 con 0x7fc2d01008d0 2024-06-06T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0110b70 con 0x7fc2d0106890 2024-06-06T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d6aee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0110e70 con 0x7fc2d0101290 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d508b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d508b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:59388/0 (socket says 172.21.15.87:59388) 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d508b640 1 -- 172.21.15.87:0/1410763763 learned_addr learned my addr 172.21.15.87:0/1410763763 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2cffff640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 0x7fc2d01a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.761+0000 7fc2d488a640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d01008d0 0x7fc2d01a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 -- 172.21.15.87:0/1410763763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 msgr2=0x7fc2d01a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 0x7fc2d01a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 -- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d01008d0 msgr2=0x7fc2d01a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d01008d0 0x7fc2d01a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 -- 172.21.15.87:0/1410763763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d019f120 con 0x7fc2d0106890 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.762+0000 7fc2d508b640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800eb10 tx=0x7fc2c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.763+0000 7fc2cdffb640 1 -- 172.21.15.87:0/1410763763 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800ce20 con 0x7fc2d0106890 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.763+0000 7fc2cdffb640 1 -- 172.21.15.87:0/1410763763 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c8005020 con 0x7fc2d0106890 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.763+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d019f410 con 0x7fc2d0106890 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.763+0000 7fc2cdffb640 1 -- 172.21.15.87:0/1410763763 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8010690 con 0x7fc2d0106890 2024-06-06T07:56:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.763+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d006b3c0 con 0x7fc2d0106890 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.767+0000 7fc2cdffb640 1 -- 172.21.15.87:0/1410763763 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc2c8005560 con 0x7fc2d0106890 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 Processor -- start 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 -- start start 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 0x55b3356d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b33562e4f0 con 0x55b3356caa90 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b3355cc870 con 0x55b3356d5350 2024-06-06T07:56:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.768+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b33562a4b0 con 0x55b3356ca170 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2cdffb640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 0x7fc2c00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2d508b640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 0x7fc2c00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2d488a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2cdffb640 1 -- 172.21.15.87:0/1410763763 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc2c80cff20 con 0x7fc2d0106890 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2d488a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60436/0 (socket says 172.21.15.87:60436) 2024-06-06T07:56:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.769+0000 7fc2d488a640 1 -- 172.21.15.87:0/3549304489 learned_addr learned my addr 172.21.15.87:0/3549304489 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2d488a640 1 -- 172.21.15.87:0/3549304489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 msgr2=0x55b3356ca550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2d488a640 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356ca550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2d488a640 1 -- 172.21.15.87:0/3549304489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 msgr2=0x55b3356d7740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2cffff640 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 0x55b3356d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2d488a640 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 0x55b3356d7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2d488a640 1 -- 172.21.15.87:0/3549304489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3355e89b0 con 0x55b3356caa90 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.770+0000 7fc2cffff640 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 0x55b3356d7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.771+0000 7fc2d508b640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 0x7fc2c00a53c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2c809bc60 tx=0x7fc2c809bd30 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.772+0000 7fc2d488a640 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009a70 tx=0x7fc2c4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc18d51d9458a95 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.773+0000 7fc2baffd640 1 -- 172.21.15.87:0/3549304489 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017070 con 0x55b3356caa90 2024-06-06T07:56:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.773+0000 7fc2baffd640 1 -- 172.21.15.87:0/3549304489 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4013470 con 0x55b3356caa90 2024-06-06T07:56:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.773+0000 7fc2baffd640 1 -- 172.21.15.87:0/3549304489 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017070 con 0x55b3356caa90 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 -- 172.21.15.87:0/3549304489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 msgr2=0x55b3356d4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009a70 tx=0x7fc2c4009f30 comp rx=0 tx=0).stop 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 -- 172.21.15.87:0/3549304489 shutdown_connections 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356d5350 0x55b3356d7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356caa90 0x55b3356d4e10 secure :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009a70 tx=0x7fc2c4009f30 comp rx=0 tx=0).stop 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/3549304489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.774+0000 7fc2d97f6740 1 -- 172.21.15.87:0/3549304489 >> 172.21.15.87:0/3549304489 conn(0x55b3356c6680 msgr2=0x55b3356c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.775+0000 7fc2d97f6740 1 -- 172.21.15.87:0/3549304489 shutdown_connections 2024-06-06T07:56:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.775+0000 7fc2d97f6740 1 -- 172.21.15.87:0/3549304489 wait complete. 2024-06-06T07:56:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.775+0000 7fc2d97f6740 1 -- start start 2024-06-06T07:56:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.776+0000 7fc2d97f6740 1 Processor -- start 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.776+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 0x55b3356d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356d2f90 0x55b3356e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b3356d7f80 con 0x55b3356d5350 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:59406/0 (socket says 172.21.15.87:59406) 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2cffff640 1 -- 172.21.15.87:0/1529667211 learned_addr learned my addr 172.21.15.87:0/1529667211 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b3355b7ae0 con 0x55b3356ca170 2024-06-06T07:56:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d488a640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 0x55b3356d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b3356d8100 con 0x55b3356d2f90 2024-06-06T07:56:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2cffff640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356d2f90 0x55b3356e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.777+0000 7fc2d97f6740 10 client.0 _ll_register_callbacks cb 0x55b3355f09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 -- 172.21.15.87:0/1529667211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356d2f90 msgr2=0x55b3356e13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356d2f90 0x55b3356e13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 -- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 msgr2=0x55b3356d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335342210 con 0x55b3356d5350 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2cffff640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2d488a640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 0x55b3356d2a50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x55b3356ccc90 tx=0x7fc2c40138d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4803b203352a4e55 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.778+0000 7fc2affff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017070 con 0x55b3356d5350 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc2c400f040 con 0x55b3356d5350 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c4013470 con 0x55b3356d5350 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c401c660 con 0x55b3356d5350 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2d97f6740 10 client.16878 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.779+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b33540e620 con 0x55b3356d5350 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4013470 con 0x55b3356d5350 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc2c4030c70 con 0x55b3356d5350 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 10 client.16878 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 20 client.16878 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356e5d80 0x55b3356e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 4 client.16878 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b335641f10 con 0x55b3356e5d80 2024-06-06T07:56:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.780+0000 7fc2cffff640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356e5d80 0x55b3356e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.781+0000 7fc2affff640 10 client.16878 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.781+0000 7fc2cffff640 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356e5d80 0x55b3356e8190 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c71994cc248cc3af server_cookie=8c3268ba522c10c3 in_seq=0 out_seq=0 2024-06-06T07:56:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.782+0000 7fc2affff640 1 -- 172.21.15.87:0/1529667211 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b335641f10 con 0x55b3356e5d80 2024-06-06T07:56:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.782+0000 7fc2affff640 10 client.16878 handle_command_reply: tid=0 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 client.16878 shutdown 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 msgr2=0x55b3356d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 0x55b3356d2a50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x55b3356ccc90 tx=0x7fc2c40138d0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 shutdown_connections 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356e5d80 0x55b3356e8190 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356d2f90 0x55b3356e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d5350 0x55b3356d2a50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2affff640 0 client.16878 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.783+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/1529667211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.784+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 >> 172.21.15.87:0/1529667211 conn(0x55b3356c6680 msgr2=0x55b3356cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.784+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 shutdown_connections 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.784+0000 7fc2d97f6740 1 -- 172.21.15.87:0/1529667211 wait complete. 2024-06-06T07:56:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.784+0000 7fc2d97f6740 20 client.16878 trim_cache size 0 max 16384 2024-06-06T07:56:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.810+0000 7fc2d97f6740 1 Processor -- start 2024-06-06T07:56:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 -- start start 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356c6680 0x55b3356cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01c07c0 con 0x55b3356d2f90 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01be090 con 0x55b3356c6680 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0073880 con 0x55b3356ca170 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.811+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60454/0 (socket says 172.21.15.87:60454) 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 -- 172.21.15.87:0/4225913510 learned_addr learned my addr 172.21.15.87:0/4225913510 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2d488a640 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356c6680 0x55b3356cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 -- 172.21.15.87:0/4225913510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 msgr2=0x55b3356c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 -- 172.21.15.87:0/4225913510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356c6680 msgr2=0x55b3356cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356c6680 0x55b3356cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.812+0000 7fc2cffff640 1 -- 172.21.15.87:0/4225913510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3356c64a0 con 0x55b3356d2f90 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.813+0000 7fc2cffff640 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006010 tx=0x7fc2bc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da6c5c1c968c654 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.813+0000 7fc2ba7fc640 1 -- 172.21.15.87:0/4225913510 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035040 con 0x55b3356d2f90 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.813+0000 7fc2ba7fc640 1 -- 172.21.15.87:0/4225913510 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc005570 con 0x55b3356d2f90 2024-06-06T07:56:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.813+0000 7fc2ba7fc640 1 -- 172.21.15.87:0/4225913510 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc003ab0 con 0x55b3356d2f90 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 -- 172.21.15.87:0/4225913510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 msgr2=0x55b3356e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006010 tx=0x7fc2bc005dd0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 -- 172.21.15.87:0/4225913510 shutdown_connections 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356e11e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356ca170 0x55b3356c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.814+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/4225913510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356c6680 0x55b3356cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.815+0000 7fc2d97f6740 1 -- 172.21.15.87:0/4225913510 >> 172.21.15.87:0/4225913510 conn(0x55b3356d2bb0 msgr2=0x55b3356d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.815+0000 7fc2d97f6740 1 -- 172.21.15.87:0/4225913510 shutdown_connections 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.815+0000 7fc2d97f6740 1 -- 172.21.15.87:0/4225913510 wait complete. 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.815+0000 7fc2d97f6740 1 -- start start 2024-06-06T07:56:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.815+0000 7fc2d97f6740 1 Processor -- start 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.816+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 0x55b3356dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.816+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.816+0000 7fc2d97f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.816+0000 7fc2d97f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01bc550 con 0x55b3356d2f90 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 0x55b3356dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 0x55b3356dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:51080/0 (socket says 172.21.15.87:51080) 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2cffff640 1 -- 172.21.15.87:0/2201337830 learned_addr learned my addr 172.21.15.87:0/2201337830 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01bd9c0 con 0x55b3356ca170 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d01bcc20 con 0x55b3356c6680 2024-06-06T07:56:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d97f6740 10 client.0 _ll_register_callbacks cb 0x7fc2c4013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2cffff640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 -- 172.21.15.87:0/2201337830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 msgr2=0x55b3356dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 0x55b3356dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 -- 172.21.15.87:0/2201337830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 msgr2=0x55b3356ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.817+0000 7fc2d488a640 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3356ddf00 con 0x55b3356ca170 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.818+0000 7fc2cffff640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.818+0000 7fc2d488a640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356dd890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4026600 tx=0x7fc2c4009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305e001b153a8db7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.819+0000 7fc2affff640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.819+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017040 con 0x55b3356ca170 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.819+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc005570 con 0x55b3356ca170 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.819+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fc2c4013b50 con 0x55b3356ca170 2024-06-06T07:56:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.819+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c4013470 con 0x55b3356ca170 2024-06-06T07:56:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.820+0000 7fc2d97f6740 10 client.26060 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.820+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b33574ffd0 con 0x55b3356ca170 2024-06-06T07:56:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.820+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc005570 con 0x55b3356ca170 2024-06-06T07:56:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc2c401c800 con 0x55b3356ca170 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 10 client.26060 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 20 client.26060 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356d5350 0x55b335752790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 4 client.26060 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b33563bc20 con 0x55b3356d5350 2024-06-06T07:56:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.821+0000 7fc2cffff640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356d5350 0x55b335752790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.822+0000 7fc2cffff640 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356d5350 0x55b335752790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc208d2649db84e server_cookie=fedb6cf8f434c288 in_seq=0 out_seq=0 2024-06-06T07:56:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.822+0000 7fc2affff640 10 client.26060 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2affff640 1 -- 172.21.15.87:0/2201337830 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b33563bc20 con 0x55b3356d5350 2024-06-06T07:56:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2affff640 10 client.26060 handle_command_reply: tid=0 2024-06-06T07:56:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 client.26060 shutdown 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 msgr2=0x55b3356dd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356dd890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4026600 tx=0x7fc2c4009110 comp rx=0 tx=0).stop 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 shutdown_connections 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b3356d5350 0x55b335752790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b3356d2f90 0x55b3356ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b3356ca170 0x55b3356dd890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 --2- 172.21.15.87:0/2201337830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b3356c6680 0x55b3356dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 >> 172.21.15.87:0/2201337830 conn(0x55b3356d2bb0 msgr2=0x55b3356cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.946+0000 7fc2affff640 0 client.26060 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.947+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 shutdown_connections 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.947+0000 7fc2d97f6740 1 -- 172.21.15.87:0/2201337830 wait complete. 2024-06-06T07:56:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.947+0000 7fc2d97f6740 20 client.26060 trim_cache size 0 max 16384 2024-06-06T07:56:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:56:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:56:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 msgr2=0x7fc2c00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 0x7fc2c00a53c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2c809bc60 tx=0x7fc2c809bd30 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 msgr2=0x7fc2d019e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800eb10 tx=0x7fc2c800ef10 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 shutdown_connections 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc2c00a2f00 0x7fc2c00a53c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0106890 0x7fc2d019e9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2d0101290 0x7fc2d01a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 --2- 172.21.15.87:0/1410763763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2d01008d0 0x7fc2d01a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 >> 172.21.15.87:0/1410763763 conn(0x7fc2d0077a20 msgr2=0x7fc2d0103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 shutdown_connections 2024-06-06T07:56:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:12.948+0000 7fc2d6aee640 1 -- 172.21.15.87:0/1410763763 wait complete. 2024-06-06T07:56:13.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.53 MB/sec execute 257 sec latency 966.801 ms 2024-06-06T07:56:13.815 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2fd621c9', quiesce took 7.0 seconds. Will release after: 18.6 2024-06-06T07:56:14.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:13 smithi087 ceph-mon[32276]: pgmap v252: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:13 smithi195 ceph-mon[37040]: pgmap v252: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:13 smithi181 ceph-mon[36790]: pgmap v252: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:14.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.50 MB/sec execute 258 sec latency 1966.898 ms 2024-06-06T07:56:15.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.48 MB/sec execute 259 sec latency 2966.997 ms 2024-06-06T07:56:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:15 smithi087 ceph-mon[32276]: pgmap v253: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:15 smithi195 ceph-mon[37040]: pgmap v253: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:15 smithi181 ceph-mon[36790]: pgmap v253: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 68 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-06-06T07:56:16.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.45 MB/sec execute 260 sec latency 3967.090 ms 2024-06-06T07:56:17.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.43 MB/sec execute 261 sec latency 4967.210 ms 2024-06-06T07:56:18.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:17 smithi087 ceph-mon[32276]: pgmap v254: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:18.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:17 smithi195 ceph-mon[37040]: pgmap v254: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:17 smithi181 ceph-mon[36790]: pgmap v254: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:18.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.40 MB/sec execute 262 sec latency 5967.274 ms 2024-06-06T07:56:19.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.38 MB/sec execute 263 sec latency 6967.376 ms 2024-06-06T07:56:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:19 smithi087 ceph-mon[32276]: pgmap v255: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:19 smithi087 ceph-mon[32276]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:56:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:19 smithi195 ceph-mon[37040]: pgmap v255: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:19 smithi195 ceph-mon[37040]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:56:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:19 smithi181 ceph-mon[36790]: pgmap v255: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-06-06T07:56:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:19 smithi181 ceph-mon[36790]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:56:20.672 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.36 MB/sec execute 264 sec latency 7967.471 ms 2024-06-06T07:56:21.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.33 MB/sec execute 265 sec latency 8967.525 ms 2024-06-06T07:56:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:21 smithi087 ceph-mon[32276]: pgmap v257: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 233 op/s 2024-06-06T07:56:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:21 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:56:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:21 smithi195 ceph-mon[37040]: pgmap v257: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 233 op/s 2024-06-06T07:56:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:21 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:56:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:21 smithi181 ceph-mon[36790]: pgmap v257: 129 pgs: 129 active+clean; 51 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 233 op/s 2024-06-06T07:56:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:21 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:56:22.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.31 MB/sec execute 266 sec latency 9967.620 ms 2024-06-06T07:56:23.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:22 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:56:23.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:22 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:56:23.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:22 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:56:23.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:22 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:56:23.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:22 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:56:23.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:22 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:56:23.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:22 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:56:23.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:22 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:56:23.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:22 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:56:23.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.28 MB/sec execute 267 sec latency 10967.695 ms 2024-06-06T07:56:24.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:24 smithi195 ceph-mon[37040]: pgmap v258: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:24 smithi181 ceph-mon[36790]: pgmap v258: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:24.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:24 smithi087 ceph-mon[32276]: pgmap v258: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:24.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.26 MB/sec execute 268 sec latency 11967.793 ms 2024-06-06T07:56:25.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.24 MB/sec execute 269 sec latency 12967.907 ms 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 -- 172.21.15.87:0/4128623067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441079c0 msgr2=0x7f1844107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 --2- 172.21.15.87:0/4128623067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441079c0 0x7f1844107da0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1838009930 tx=0x7f183802a420 comp rx=0 tx=0).stop 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 -- 172.21.15.87:0/4128623067 shutdown_connections 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 --2- 172.21.15.87:0/4128623067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f184410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 --2- 172.21.15.87:0/4128623067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441019b0 0x7f1844101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 --2- 172.21.15.87:0/4128623067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441079c0 0x7f1844107da0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 -- 172.21.15.87:0/4128623067 >> 172.21.15.87:0/4128623067 conn(0x7f1844078e30 msgr2=0x7f1844079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.049+0000 7f184ac5d640 1 -- 172.21.15.87:0/4128623067 shutdown_connections 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 -- 172.21.15.87:0/4128623067 wait complete. 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 Processor -- start 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 -- start start 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f1844198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441079c0 0x7f184419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111dd0 con 0x7f18441019b0 2024-06-06T07:56:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111c50 con 0x7f1844102370 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f184ac5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111f50 con 0x7f18441079c0 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.050+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54726/0 (socket says 172.21.15.87:54726) 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 -- 172.21.15.87:0/1016069008 learned_addr learned my addr 172.21.15.87:0/1016069008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18491fa640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441079c0 0x7f184419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f1843fff640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f1844198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 -- 172.21.15.87:0/1016069008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441079c0 msgr2=0x7f184419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441079c0 0x7f184419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 -- 172.21.15.87:0/1016069008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 msgr2=0x7f1844198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f1844198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 -- 172.21.15.87:0/1016069008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184419d1c0 con 0x7f18441019b0 2024-06-06T07:56:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f1843fff640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f1844198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f18489f9640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1838009230 tx=0x7f1838004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f1841ffb640 1 -- 172.21.15.87:0/1016069008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183803f030 con 0x7f18441019b0 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f1841ffb640 1 -- 172.21.15.87:0/1016069008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1838003ad0 con 0x7f18441019b0 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184419d450 con 0x7f18441019b0 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f1841ffb640 1 -- 172.21.15.87:0/1016069008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838035070 con 0x7f18441019b0 2024-06-06T07:56:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.051+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a4d40 con 0x7f18441019b0 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f1841ffb640 1 -- 172.21.15.87:0/1016069008 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1838042030 con 0x7f18441019b0 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 Processor -- start 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 -- start start 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 0x56073fac3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56073fa274f0 con 0x56073fac3990 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56073f9c5870 con 0x56073fad2760 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f184d96f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56073fa234b0 con 0x56073fac3070 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f1841ffb640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 0x7f18340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54738/0 (socket says 172.21.15.87:54738) 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.054+0000 7f18489f9640 1 -- 172.21.15.87:0/3144594002 learned_addr learned my addr 172.21.15.87:0/3144594002 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18491fa640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 0x56073fac3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18491fa640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 0x7f18340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f1843fff640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f1841ffb640 1 -- 172.21.15.87:0/1016069008 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f18380efcc0 con 0x7f18441019b0 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 -- 172.21.15.87:0/3144594002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 msgr2=0x56073fac3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 0x56073fac3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 -- 172.21.15.87:0/3144594002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 msgr2=0x56073fad4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 -- 172.21.15.87:0/3144594002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073f9e19b0 con 0x56073fac3990 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18491fa640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 0x56073fac3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18489f9640 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f18380bbd50 tx=0x7f183804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9500d018800d70 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f18491fa640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 0x7f18340a52e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f183c005c40 tx=0x7f183c009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f182effd640 1 -- 172.21.15.87:0/3144594002 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838049070 con 0x56073fac3990 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f182effd640 1 -- 172.21.15.87:0/3144594002 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18380bbdc0 con 0x56073fac3990 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.055+0000 7f182effd640 1 -- 172.21.15.87:0/3144594002 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380b8430 con 0x56073fac3990 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- 172.21.15.87:0/3144594002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 msgr2=0x56073fad2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f18380bbd50 tx=0x7f183804b000 comp rx=0 tx=0).stop 2024-06-06T07:56:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- 172.21.15.87:0/3144594002 shutdown_connections 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3990 0x56073fad2220 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 --2- 172.21.15.87:0/3144594002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3070 0x56073fac3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- 172.21.15.87:0/3144594002 >> 172.21.15.87:0/3144594002 conn(0x56073fabf680 msgr2=0x56073fabfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- 172.21.15.87:0/3144594002 shutdown_connections 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- 172.21.15.87:0/3144594002 wait complete. 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.056+0000 7f184d96f740 1 -- start start 2024-06-06T07:56:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 Processor -- start 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3990 0x56073facfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fad2760 0x56073fad0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56073fad5150 con 0x56073fac3070 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56073f9b0ae0 con 0x56073fac3990 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56073fad52d0 con 0x56073fad2760 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f184d96f740 10 client.0 _ll_register_callbacks cb 0x56073f9e99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54744/0 (socket says 172.21.15.87:54744) 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 -- 172.21.15.87:0/1020708539 learned_addr learned my addr 172.21.15.87:0/1020708539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18491fa640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fad2760 0x56073fad0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 -- 172.21.15.87:0/1020708539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fad2760 msgr2=0x56073fad0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fad2760 0x56073fad0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 -- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3990 msgr2=0x56073facfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3990 0x56073facfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.057+0000 7f18489f9640 1 -- 172.21.15.87:0/1020708539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073f73b210 con 0x56073fac3070 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f18489f9640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18380bbac0 tx=0x7f18380c4a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6caf8787bd193b65 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838049040 con 0x56073fac3070 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1838003c80 con 0x56073fac3070 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18380bbdc0 con 0x56073fac3070 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18380b8430 con 0x56073fac3070 2024-06-06T07:56:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 10 client.16896 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56073f807620 con 0x56073fac3070 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18380bbdc0 con 0x56073fac3070 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f18380c45a0 con 0x56073fac3070 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 20 client.16896 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fae2580 0x56073fae4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 4 client.16896 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.058+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56073fa3af10 con 0x56073fae2580 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.059+0000 7f1843fff640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fae2580 0x56073fae4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.059+0000 7f1843fff640 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fae2580 0x56073fae4a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6667943c9e7167 server_cookie=fba2956cef7c9ef in_seq=0 out_seq=0 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.059+0000 7f1823fff640 10 client.16896 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f1823fff640 1 -- 172.21.15.87:0/1020708539 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56073fa3af10 con 0x56073fae2580 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f1823fff640 10 client.16896 handle_command_reply: tid=0 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 client.16896 shutdown 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 msgr2=0x56073facf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18380bbac0 tx=0x7f18380c4a30 comp rx=0 tx=0).stop 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 shutdown_connections 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fae2580 0x56073fae4a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fad2760 0x56073fad0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3990 0x56073facfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f1823fff640 0 client.16896 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 --2- 172.21.15.87:0/1020708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073facf790 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 >> 172.21.15.87:0/1020708539 conn(0x56073fabf680 msgr2=0x56073fac54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 shutdown_connections 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 1 -- 172.21.15.87:0/1020708539 wait complete. 2024-06-06T07:56:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.060+0000 7f184d96f740 20 client.16896 trim_cache size 0 max 16384 2024-06-06T07:56:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.075+0000 7f184d96f740 1 Processor -- start 2024-06-06T07:56:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.075+0000 7f184d96f740 1 -- start start 2024-06-06T07:56:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.075+0000 7f184d96f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 0x56073fabe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.075+0000 7f184d96f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073faea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.075+0000 7f184d96f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f184d96f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18441af720 con 0x56073fad2760 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f184d96f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18441acff0 con 0x56073fac3070 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f184d96f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18441aae60 con 0x56073fac3990 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54752/0 (socket says 172.21.15.87:54752) 2024-06-06T07:56:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 -- 172.21.15.87:0/4176178615 learned_addr learned my addr 172.21.15.87:0/4176178615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f1843fff640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073faea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18489f9640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 0x56073fabe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 -- 172.21.15.87:0/4176178615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 msgr2=0x56073faea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073faea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 -- 172.21.15.87:0/4176178615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 msgr2=0x56073fabe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 0x56073fabe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 -- 172.21.15.87:0/4176178615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073fac3ed0 con 0x56073fad2760 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18489f9640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 0x56073fabe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f18491fa640 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f183c009ed0 tx=0x7f183c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608282dd2eb2961f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f182e7fc640 1 -- 172.21.15.87:0/4176178615 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00dde0 con 0x56073fad2760 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f182e7fc640 1 -- 172.21.15.87:0/4176178615 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c026030 con 0x56073fad2760 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.076+0000 7f182e7fc640 1 -- 172.21.15.87:0/4176178615 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00f6b0 con 0x56073fad2760 2024-06-06T07:56:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- 172.21.15.87:0/4176178615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 msgr2=0x56073faea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f183c009ed0 tx=0x7f183c009a20 comp rx=0 tx=0).stop 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- 172.21.15.87:0/4176178615 shutdown_connections 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fad2760 0x56073faea830 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073faea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 --2- 172.21.15.87:0/4176178615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fac3070 0x56073fabe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- 172.21.15.87:0/4176178615 >> 172.21.15.87:0/4176178615 conn(0x56073fabf680 msgr2=0x56073fabdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- 172.21.15.87:0/4176178615 shutdown_connections 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- 172.21.15.87:0/4176178615 wait complete. 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 -- start start 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.077+0000 7f184d96f740 1 Processor -- start 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073fad6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18441ab4b0 con 0x56073fac3070 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18441ac920 con 0x56073fad2760 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18441abb80 con 0x56073fac3990 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 10 client.0 _ll_register_callbacks cb 0x7f1830009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54764/0 (socket says 172.21.15.87:54764) 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 -- 172.21.15.87:0/4002520316 learned_addr learned my addr 172.21.15.87:0/4002520316 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f1843fff640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073fad6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 -- 172.21.15.87:0/4002520316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 msgr2=0x56073fad6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073fad6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 -- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 msgr2=0x56073fad6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 -- 172.21.15.87:0/4002520316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073fad6e30 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f18489f9640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f18380bc260 tx=0x7f18380ba2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e2a0dd8bd49319 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f1823fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838049050 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f18380c45a0 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.078+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183c026030 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18380b85d0 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 10 client.16902 fetch_fsmap learned FSMap version 32 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56073fad6e30 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c026030 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f183809abe0 con 0x56073fac3070 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 10 client.16902 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:56:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 20 client.16902 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fb4d690 0x56073fb4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 4 client.16902 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56073fa34c20 con 0x56073fb4d690 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.079+0000 7f1843fff640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fb4d690 0x56073fb4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.080+0000 7f1823fff640 10 client.16902 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.080+0000 7f1843fff640 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fb4d690 0x56073fb4faf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c37a720a2b0502 server_cookie=120285907f13e165 in_seq=0 out_seq=0 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.103+0000 7f1823fff640 1 -- 172.21.15.87:0/4002520316 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56073fa34c20 con 0x56073fb4d690 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.103+0000 7f1823fff640 10 client.16902 handle_command_reply: tid=0 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.103+0000 7f184d96f740 1 client.16902 shutdown 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 msgr2=0x56073fadad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f18380bc260 tx=0x7f18380ba2d0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 shutdown_connections 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56073fb4d690 0x56073fb4faf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073fad2760 0x56073fad6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56073fac3990 0x56073fad6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f1823fff640 0 client.16902 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 --2- 172.21.15.87:0/4002520316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56073fac3070 0x56073fadad10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 >> 172.21.15.87:0/4002520316 conn(0x56073fabf680 msgr2=0x56073faddc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 shutdown_connections 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 1 -- 172.21.15.87:0/4002520316 wait complete. 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184d96f740 20 client.16902 trim_cache size 0 max 16384 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:56:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:56:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 msgr2=0x7f18340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 0x7f18340a52e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f183c005c40 tx=0x7f183c009500 comp rx=0 tx=0).stop 2024-06-06T07:56:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 msgr2=0x7f18441981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1838009230 tx=0x7f1838004270 comp rx=0 tx=0).stop 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 shutdown_connections 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.104+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18340a2e20 0x7f18340a52e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18441079c0 0x7f184419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1844102370 0x7f1844198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 --2- 172.21.15.87:0/1016069008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18441019b0 0x7f18441981d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 >> 172.21.15.87:0/1016069008 conn(0x7f1844078e30 msgr2=0x7f18440ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 shutdown_connections 2024-06-06T07:56:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:26.105+0000 7f184ac5d640 1 -- 172.21.15.87:0/1016069008 wait complete. 2024-06-06T07:56:26.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:26 smithi195 ceph-mon[37040]: pgmap v259: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:26 smithi181 ceph-mon[36790]: pgmap v259: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:26 smithi087 ceph-mon[32276]: pgmap v259: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 270 KiB/s wr, 111 op/s 2024-06-06T07:56:26.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.21 MB/sec execute 270 sec latency 13968.022 ms 2024-06-06T07:56:27.167 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:56:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T07:56:27.168 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:56:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T07:56:27.168 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:56:26 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/872265488 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7eb6ffa40 con 0x55f7eb6d2c00 2024-06-06T07:56:27.168 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:56:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T07:56:27.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:27 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T07:56:27.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:27 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T07:56:27.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:27 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T07:56:27.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.19 MB/sec execute 271 sec latency 14968.122 ms 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: pgmap v260: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/1119640818,v1:172.21.15.195:6837/1119640818] up:boot 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:56:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:28 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: pgmap v260: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/1119640818,v1:172.21.15.195:6837/1119640818] up:boot 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:56:28.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:28 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: pgmap v260: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/1119640818,v1:172.21.15.195:6837/1119640818] up:boot 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:56:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:28 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:56:28.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.17 MB/sec execute 272 sec latency 15968.229 ms 2024-06-06T07:56:29.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.15 MB/sec execute 273 sec latency 16968.287 ms 2024-06-06T07:56:30.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:30 smithi195 ceph-mon[37040]: pgmap v262: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-06-06T07:56:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:30 smithi181 ceph-mon[36790]: pgmap v262: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-06-06T07:56:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:30 smithi087 ceph-mon[32276]: pgmap v262: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-06-06T07:56:30.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.12 MB/sec execute 274 sec latency 17968.407 ms 2024-06-06T07:56:31.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.10 MB/sec execute 275 sec latency 18968.509 ms 2024-06-06T07:56:32.416 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:56:32.444 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:32 smithi087 ceph-mon[32276]: pgmap v263: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:32 smithi195 ceph-mon[37040]: pgmap v263: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:32 smithi181 ceph-mon[36790]: pgmap v263: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 143 KiB/s wr, 18 op/s 2024-06-06T07:56:32.668 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:32.673 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.08 MB/sec execute 276 sec latency 19968.543 ms 2024-06-06T07:56:33.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.06 MB/sec execute 277 sec latency 20968.626 ms 2024-06-06T07:56:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:34 smithi087 ceph-mon[32276]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:34 smithi195 ceph-mon[37040]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:34 smithi181 ceph-mon[36790]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:34.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.04 MB/sec execute 278 sec latency 21968.667 ms 2024-06-06T07:56:34.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 -- 172.21.15.87:0/2169212823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024101390 msgr2=0x7fc024101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:34.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 --2- 172.21.15.87:0/2169212823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024101390 0x7fc024101810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009990 tx=0x7fc00c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 -- 172.21.15.87:0/2169212823 shutdown_connections 2024-06-06T07:56:34.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 --2- 172.21.15.87:0/2169212823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101d50 0x7fc02410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 --2- 172.21.15.87:0/2169212823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024107400 0x7fc0241077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 --2- 172.21.15.87:0/2169212823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024101390 0x7fc024101810 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 -- 172.21.15.87:0/2169212823 >> 172.21.15.87:0/2169212823 conn(0x7fc024078340 msgr2=0x7fc024078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 -- 172.21.15.87:0/2169212823 shutdown_connections 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.676+0000 7fc02898d700 1 -- 172.21.15.87:0/2169212823 wait complete. 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.677+0000 7fc02898d700 1 Processor -- start 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.677+0000 7fc02898d700 1 -- start start 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.677+0000 7fc02898d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 0x7fc02419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.677+0000 7fc02898d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024101d50 0x7fc02419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111690 con 0x7fc024107400 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111510 con 0x7fc024101390 2024-06-06T07:56:34.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111810 con 0x7fc024101d50 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55546/0 (socket says 172.21.15.87:55546) 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 -- 172.21.15.87:0/3791804614 learned_addr learned my addr 172.21.15.87:0/3791804614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02259c700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 0x7fc02419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc021d9b700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024101d50 0x7fc02419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 -- 172.21.15.87:0/3791804614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024101d50 msgr2=0x7fc02419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024101d50 0x7fc02419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 -- 172.21.15.87:0/3791804614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 msgr2=0x7fc02419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 0x7fc02419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 -- 172.21.15.87:0/3791804614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc024199180 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02259c700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 0x7fc02419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc022d9d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc01400d980 tx=0x7fc01400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014017070 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc024199470 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc014009e20 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014005440 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.678+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a5730 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.679+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024069180 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.681+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc0140055e0 con 0x7fc024107400 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.681+0000 7fc01b7fe700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 0x7fc0100a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:34.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.681+0000 7fc02259c700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 0x7fc0100a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:34.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.681+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fc0140cfb90 con 0x7fc024107400 2024-06-06T07:56:34.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.682+0000 7fc02259c700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 0x7fc0100a5250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc024199b60 tx=0x7fc01c009040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:34.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.685+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc014020020 con 0x7fc024107400 2024-06-06T07:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.928+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0241a5960 con 0x7fc024107400 2024-06-06T07:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.928+0000 7fc01b7fe700 1 -- 172.21.15.87:0/3791804614 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11877 (secure 0 0 0) 0x7fc01409c3e0 con 0x7fc024107400 2024-06-06T07:56:34.929 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:56:34.929 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:56:27.171576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26164":{"gid":26164,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1119640818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1119640818},{"type":"v1","addr":"172.21.15.195:6837","nonce":1119640818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 msgr2=0x7fc0100a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 0x7fc0100a5250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc024199b60 tx=0x7fc01c009040 comp rx=0 tx=0).stop 2024-06-06T07:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 msgr2=0x7fc024198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc01400d980 tx=0x7fc01400dd40 comp rx=0 tx=0).stop 2024-06-06T07:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 shutdown_connections 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024101390 0x7fc02419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc0100a2d90 0x7fc0100a5250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024107400 0x7fc024198a20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 --2- 172.21.15.87:0/3791804614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc024101d50 0x7fc02419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 >> 172.21.15.87:0/3791804614 conn(0x7fc024078340 msgr2=0x7fc0241005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 shutdown_connections 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:34.931+0000 7fc02898d700 1 -- 172.21.15.87:0/3791804614 wait complete. 2024-06-06T07:56:34.932 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 34 2024-06-06T07:56:35.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:35 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3791804614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:35.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:35 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3791804614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:35.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:35 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3791804614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:56:35.636 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 2fd621c9 --release --await' 2024-06-06T07:56:35.636 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 2fd621c9 --release --await 2024-06-06T07:56:35.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 6.01 MB/sec execute 279 sec latency 22968.765 ms 2024-06-06T07:56:35.915 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:56:36.305 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:36 smithi087 ceph-mon[32276]: pgmap v265: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:36 smithi195 ceph-mon[37040]: pgmap v265: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:36 smithi181 ceph-mon[36790]: pgmap v265: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-06-06T07:56:36.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 5.99 MB/sec execute 280 sec latency 23968.884 ms 2024-06-06T07:56:37.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 -- 172.21.15.87:0/2098755367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc8406a170 msgr2=0x7efc8406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 --2- 172.21.15.87:0/2098755367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc8406a170 0x7efc8406a5f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009990 tx=0x7efc6c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 -- 172.21.15.87:0/2098755367 shutdown_connections 2024-06-06T07:56:37.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 --2- 172.21.15.87:0/2098755367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 0x7efc8410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 --2- 172.21.15.87:0/2098755367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc84108500 0x7efc841088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 --2- 172.21.15.87:0/2098755367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc8406a170 0x7efc8406a5f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 -- 172.21.15.87:0/2098755367 >> 172.21.15.87:0/2098755367 conn(0x7efc84078340 msgr2=0x7efc84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 -- 172.21.15.87:0/2098755367 shutdown_connections 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.638+0000 7efc8a835700 1 -- 172.21.15.87:0/2098755367 wait complete. 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 Processor -- start 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 -- start start 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc8406a170 0x7efc8419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 0x7efc8419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc84112780 con 0x7efc84108500 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc84112600 con 0x7efc8406ab30 2024-06-06T07:56:37.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc8a835700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc84112900 con 0x7efc8406a170 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59566/0 (socket says 172.21.15.87:59566) 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc88dd2700 1 -- 172.21.15.87:0/2198538453 learned_addr learned my addr 172.21.15.87:0/2198538453 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc7bfff700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 0x7efc8419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.639+0000 7efc83fff700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc8406a170 0x7efc8419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 -- 172.21.15.87:0/2198538453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc8406a170 msgr2=0x7efc8419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc8406a170 0x7efc8419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 -- 172.21.15.87:0/2198538453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 msgr2=0x7efc8419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 0x7efc8419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 -- 172.21.15.87:0/2198538453 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84199180 con 0x7efc84108500 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efc7c007ee0 tx=0x7efc7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc81ffb700 1 -- 172.21.15.87:0/2198538453 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c004bb0 con 0x7efc84108500 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc84199470 con 0x7efc84108500 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc81ffb700 1 -- 172.21.15.87:0/2198538453 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c004d50 con 0x7efc84108500 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc81ffb700 1 -- 172.21.15.87:0/2198538453 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005960 con 0x7efc84108500 2024-06-06T07:56:37.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.640+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841a5730 con 0x7efc84108500 2024-06-06T07:56:37.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.643+0000 7efc81ffb700 1 -- 172.21.15.87:0/2198538453 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efc7c01a020 con 0x7efc84108500 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.644+0000 7efc81ffb700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 0x7efc700a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.644+0000 7efc83fff700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 0x7efc700a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.644+0000 7efc81ffb700 1 -- 172.21.15.87:0/2198538453 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7efc7c0cea00 con 0x7efc84108500 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.644+0000 7efc83fff700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 0x7efc700a5250 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc84199b60 tx=0x7efc7400b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 Processor -- start 2024-06-06T07:56:37.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 -- start start 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 0x55e986738020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986738560 0x55e986731e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e986683910 con 0x55e9867323d0 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e98662b820 con 0x55e986738560 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc950b8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e98667f8d0 con 0x55e986737c40 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59572/0 (socket says 172.21.15.87:59572) 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc88dd2700 1 -- 172.21.15.87:0/1626320294 learned_addr learned my addr 172.21.15.87:0/1626320294 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc7bfff700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986738560 0x55e986731e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.645+0000 7efc83fff700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 0x55e986738020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 -- 172.21.15.87:0/1626320294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 msgr2=0x55e986738020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 0x55e986738020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 -- 172.21.15.87:0/1626320294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986738560 msgr2=0x55e986731e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986738560 0x55e986731e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 -- 172.21.15.87:0/1626320294 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e986622050 con 0x55e9867323d0 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc83fff700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 0x55e986738020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005b90 tx=0x7efc7c010250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316a9269d1bb46a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc7a7fc700 1 -- 172.21.15.87:0/1626320294 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c09eea0 con 0x55e9867323d0 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc7a7fc700 1 -- 172.21.15.87:0/1626320294 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c0a3760 con 0x55e9867323d0 2024-06-06T07:56:37.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.646+0000 7efc7a7fc700 1 -- 172.21.15.87:0/1626320294 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c09ad50 con 0x55e9867323d0 2024-06-06T07:56:37.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- 172.21.15.87:0/1626320294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 msgr2=0x55e986732810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005b90 tx=0x7efc7c010250 comp rx=0 tx=0).stop 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- 172.21.15.87:0/1626320294 shutdown_connections 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e9867323d0 0x55e986732810 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e986737c40 0x55e986738020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1626320294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986738560 0x55e986731e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- 172.21.15.87:0/1626320294 >> 172.21.15.87:0/1626320294 conn(0x55e98665cdc0 msgr2=0x55e98672d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- 172.21.15.87:0/1626320294 shutdown_connections 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- 172.21.15.87:0/1626320294 wait complete. 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.647+0000 7efc950b8b80 1 -- start start 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 Processor -- start 2024-06-06T07:56:37.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e98673e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e98673efd0 con 0x55e986738560 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e986623cf0 con 0x55e986737c40 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e98673f150 con 0x55e9867323d0 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc950b8b80 10 client.0 _ll_register_callbacks cb 0x55e98661da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59576/0 (socket says 172.21.15.87:59576) 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 -- 172.21.15.87:0/2639215569 learned_addr learned my addr 172.21.15.87:0/2639215569 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc7bfff700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc83fff700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e98673e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 -- 172.21.15.87:0/2639215569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 msgr2=0x55e98673e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e98673e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 -- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 msgr2=0x55e98673b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.648+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc88dd2700 1 -- 172.21.15.87:0/2639215569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9865d29d0 con 0x55e986738560 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc7bfff700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00de20 tx=0x7efc7c097390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49457712ba04349a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0b9500 con 0x55e986738560 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7efc7c004f00 con 0x55e986738560 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7c0a3760 con 0x55e986738560 2024-06-06T07:56:37.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc7c005100 con 0x55e986738560 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc950b8b80 10 client.16920 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e98673efd0 con 0x55e986738560 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c0a3760 con 0x55e986738560 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.649+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7efc7c0a3360 con 0x55e986738560 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 10 client.16920 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 10 client.16920 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 20 client.16920 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e98674bc00 0x55e98674dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 4 client.16920 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e986690e10 con 0x55e98674bc00 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc83fff700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e98674bc00 0x55e98674dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc83fff700 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e98674bc00 0x55e98674dff0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29565d2f24cda4e server_cookie=1e236001d05b2725 in_seq=0 out_seq=0 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.650+0000 7efc637fe700 10 client.16920 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc637fe700 1 -- 172.21.15.87:0/2639215569 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e986690e10 con 0x55e98674bc00 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc637fe700 10 client.16920 handle_command_reply: tid=0 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 client.16920 shutdown 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 msgr2=0x55e98673bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00de20 tx=0x7efc7c097390 comp rx=0 tx=0).stop 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 shutdown_connections 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e98674bc00 0x55e98674dff0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e98673bb50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e98673e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2639215569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 >> 172.21.15.87:0/2639215569 conn(0x55e98665cdc0 msgr2=0x55e98672cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc637fe700 0 client.16920 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:37.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.651+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 shutdown_connections 2024-06-06T07:56:37.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.652+0000 7efc950b8b80 1 -- 172.21.15.87:0/2639215569 wait complete. 2024-06-06T07:56:37.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.652+0000 7efc950b8b80 20 client.16920 trim_cache size 0 max 16384 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.671+0000 7efc950b8b80 1 Processor -- start 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.671+0000 7efc950b8b80 1 -- start start 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e986731e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b0ed0 con 0x55e986738560 2024-06-06T07:56:37.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ae760 con 0x55e986737c40 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc950b8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ac5b0 con 0x55e9867323d0 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59584/0 (socket says 172.21.15.87:59584) 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 -- 172.21.15.87:0/1815015189 learned_addr learned my addr 172.21.15.87:0/1815015189 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc83fff700 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e986731e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc7bfff700 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 -- 172.21.15.87:0/1815015189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 msgr2=0x55e986731e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e986731e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 -- 172.21.15.87:0/1815015189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 msgr2=0x55e98673d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 -- 172.21.15.87:0/1815015189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98673dee0 con 0x55e986738560 2024-06-06T07:56:37.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc88dd2700 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efc7c096f50 tx=0x7efc7c09eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e329b8d603b2280 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc79ffb700 1 -- 172.21.15.87:0/1815015189 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c010250 con 0x55e986738560 2024-06-06T07:56:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc79ffb700 1 -- 172.21.15.87:0/1815015189 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c0a3760 con 0x55e986738560 2024-06-06T07:56:37.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.672+0000 7efc79ffb700 1 -- 172.21.15.87:0/1815015189 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0a32d0 con 0x55e986738560 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- 172.21.15.87:0/1815015189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 msgr2=0x55e986750c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efc7c096f50 tx=0x7efc7c09eea0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- 172.21.15.87:0/1815015189 shutdown_connections 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e986750c40 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e986731e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 --2- 172.21.15.87:0/1815015189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e98673d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- 172.21.15.87:0/1815015189 >> 172.21.15.87:0/1815015189 conn(0x55e98665cdc0 msgr2=0x55e98673ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- 172.21.15.87:0/1815015189 shutdown_connections 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- 172.21.15.87:0/1815015189 wait complete. 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.673+0000 7efc950b8b80 1 -- start start 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 Processor -- start 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e9867aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e9867af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc841acc20 con 0x55e986738560 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ae090 con 0x55e986737c40 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ad2f0 con 0x55e9867323d0 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc950b8b80 10 client.0 _ll_register_callbacks cb 0x55e98661da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59596/0 (socket says 172.21.15.87:59596) 2024-06-06T07:56:37.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 -- 172.21.15.87:0/2350956911 learned_addr learned my addr 172.21.15.87:0/2350956911 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:37.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 435740 5.97 MB/sec execute 281 sec latency 24968.934 ms 2024-06-06T07:56:37.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc7bfff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e9867af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc83fff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e9867aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 -- 172.21.15.87:0/2350956911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 msgr2=0x55e9867aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e9867aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 -- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 msgr2=0x55e9867af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e9867af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc88dd2700 1 -- 172.21.15.87:0/2350956911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9867b3150 con 0x55e986738560 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc7bfff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e9867af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.674+0000 7efc83fff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e9867aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc88dd2700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efc7c09a7f0 tx=0x7efc7c099000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1004b5b48a5835c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c099fe0 con 0x55e986738560 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7efc7c004f00 con 0x55e986738560 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7c0a3760 con 0x55e986738560 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc7c0ab050 con 0x55e986738560 2024-06-06T07:56:37.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 10 client.16926 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e9867b3150 con 0x55e986738560 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c0a3760 con 0x55e986738560 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7efc7c099aa0 con 0x55e986738560 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 10 client.16926 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 10 client.16926 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 20 client.16926 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e9867bb940 0x55e9867bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 4 client.16926 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2fd621c9", "release": true, "await": true}] 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.675+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2fd621c9", "release": true, "await": true}) -- 0x55e98668b390 con 0x55e9867bb940 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.676+0000 7efc83fff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e9867bb940 0x55e9867bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.676+0000 7efc83fff700 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e9867bb940 0x55e9867bdd30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283e4a57020caeba server_cookie=790d5762d194b008 in_seq=0 out_seq=0 2024-06-06T07:56:37.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.676+0000 7efc637fe700 10 client.16926 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:37.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc637fe700 1 -- 172.21.15.87:0/2350956911 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e98668b390 con 0x55e9867bb940 2024-06-06T07:56:37.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc637fe700 10 client.16926 handle_command_reply: tid=0 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 client.16926 shutdown 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 msgr2=0x55e9867b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efc7c09a7f0 tx=0x7efc7c099000 comp rx=0 tx=0).stop 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 shutdown_connections 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e9867bb940 0x55e9867bdd30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e986738560 0x55e9867b2a70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9867323d0 0x55e9867aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc637fe700 0 client.16926 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 --2- 172.21.15.87:0/2350956911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e986737c40 0x55e9867af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.685+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 >> 172.21.15.87:0/2350956911 conn(0x55e98665cdc0 msgr2=0x55e98673d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 shutdown_connections 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc950b8b80 1 -- 172.21.15.87:0/2350956911 wait complete. 2024-06-06T07:56:37.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc950b8b80 20 client.16926 trim_cache size 0 max 16384 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 34, 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 28, 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "2fd621c9": { 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 28, 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:37.687 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.6, 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:37.688 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:56:37.689 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:56:37.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 msgr2=0x7efc700a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 0x7efc700a5250 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc84199b60 tx=0x7efc7400b3c0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 msgr2=0x7efc84198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efc7c007ee0 tx=0x7efc7c00d3b0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 shutdown_connections 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7efc700a2d90 0x7efc700a5250 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84108500 0x7efc84198a20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc8406a170 0x7efc8419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 --2- 172.21.15.87:0/2198538453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8406ab30 0x7efc8419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 >> 172.21.15.87:0/2198538453 conn(0x7efc84078340 msgr2=0x7efc840fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 shutdown_connections 2024-06-06T07:56:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:56:37.686+0000 7efc8a835700 1 -- 172.21.15.87:0/2198538453 wait complete. 2024-06-06T07:56:38.414 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2fd621c9', total seconds elapsed: 31.6 2024-06-06T07:56:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:38 smithi195 ceph-mon[37040]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 897 KiB/s wr, 51 op/s 2024-06-06T07:56:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:38 smithi181 ceph-mon[36790]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 897 KiB/s wr, 51 op/s 2024-06-06T07:56:38.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:38 smithi087 ceph-mon[32276]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 897 KiB/s wr, 51 op/s 2024-06-06T07:56:38.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 437497 5.98 MB/sec execute 282 sec latency 24983.132 ms 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- 172.21.15.87:0/110461963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 msgr2=0x7fc37010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 --2- 172.21.15.87:0/110461963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc37010d780 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc368009cd0 tx=0x7fc36802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- 172.21.15.87:0/110461963 shutdown_connections 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 --2- 172.21.15.87:0/110461963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc37010d780 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 --2- 172.21.15.87:0/110461963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 0x7fc370100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 --2- 172.21.15.87:0/110461963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 0x7fc370106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- 172.21.15.87:0/110461963 >> 172.21.15.87:0/110461963 conn(0x7fc370077ab0 msgr2=0x7fc370077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- 172.21.15.87:0/110461963 shutdown_connections 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- 172.21.15.87:0/110461963 wait complete. 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 Processor -- start 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.198+0000 7fc3762cd640 1 -- start start 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 0x7fc370198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 0x7fc37019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc370110c90 con 0x7fc370101270 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc370110b10 con 0x7fc3701008b0 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc370110e10 con 0x7fc3701068c0 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59604/0 (socket says 172.21.15.87:59604) 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 -- 172.21.15.87:0/2882917216 learned_addr learned my addr 172.21.15.87:0/2882917216 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc37486a640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 0x7fc37019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36ffff640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 0x7fc370198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 -- 172.21.15.87:0/2882917216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 msgr2=0x7fc37019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 0x7fc37019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 -- 172.21.15.87:0/2882917216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 msgr2=0x7fc370198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 0x7fc370198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 -- 172.21.15.87:0/2882917216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37019d150 con 0x7fc370101270 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e9f0 tx=0x7fc36400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36d7fa640 1 -- 172.21.15.87:0/2882917216 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36400cd80 con 0x7fc370101270 2024-06-06T07:56:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36d7fa640 1 -- 172.21.15.87:0/2882917216 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc364010910 con 0x7fc370101270 2024-06-06T07:56:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc36d7fa640 1 -- 172.21.15.87:0/2882917216 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364018990 con 0x7fc370101270 2024-06-06T07:56:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.199+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37019d440 con 0x7fc370101270 2024-06-06T07:56:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.200+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3701a4cd0 con 0x7fc370101270 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.202+0000 7fc36d7fa640 1 -- 172.21.15.87:0/2882917216 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc364010ac0 con 0x7fc370101270 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.202+0000 7fc378fda740 1 Processor -- start 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.202+0000 7fc378fda740 1 -- start start 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55de6c7134f0 con 0x55de6c7ba350 2024-06-06T07:56:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55de6c6b1870 con 0x55de6c7afa90 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc378fda740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de6c70f4b0 con 0x55de6c7af170 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59618/0 (socket says 172.21.15.87:59618) 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1914307524 learned_addr learned my addr 172.21.15.87:0/1914307524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc37486a640 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1914307524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 msgr2=0x55de6c7af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1914307524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 msgr2=0x55de6c7b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1914307524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6c6cd9b0 con 0x55de6c7ba350 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36d7fa640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 0x7fc3580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc37486a640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 0x7fc3580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ccd80 tx=0x7fc3640d85d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276d64f56049398f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.203+0000 7fc36d7fa640 1 -- 172.21.15.87:0/2882917216 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fc3640cfa90 con 0x7fc370101270 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc3567fc640 1 -- 172.21.15.87:0/1914307524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364026020 con 0x55de6c7ba350 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc3567fc640 1 -- 172.21.15.87:0/1914307524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640d7040 con 0x55de6c7ba350 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc3567fc640 1 -- 172.21.15.87:0/1914307524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640a9e60 con 0x55de6c7ba350 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 -- 172.21.15.87:0/1914307524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 msgr2=0x55de6c7bc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc37486a640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 0x7fc3580a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc368004ab0 tx=0x7fc368004a40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ccd80 tx=0x7fc3640d85d0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 -- 172.21.15.87:0/1914307524 shutdown_connections 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bc740 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 --2- 172.21.15.87:0/1914307524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 -- 172.21.15.87:0/1914307524 >> 172.21.15.87:0/1914307524 conn(0x55de6c7ab680 msgr2=0x55de6c7aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 -- 172.21.15.87:0/1914307524 shutdown_connections 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.204+0000 7fc378fda740 1 -- 172.21.15.87:0/1914307524 wait complete. 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 -- start start 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 Processor -- start 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 0x55de6c7b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55de6c7bcf80 con 0x55de6c7ba350 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55de6c69cae0 con 0x55de6c7af170 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de6c7bd100 con 0x55de6c7afa90 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc378fda740 10 client.0 _ll_register_callbacks cb 0x55de6c6d59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc37486a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc37486a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59624/0 (socket says 172.21.15.87:59624) 2024-06-06T07:56:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 0x55de6c7b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.205+0000 7fc37486a640 1 -- 172.21.15.87:0/350381399 learned_addr learned my addr 172.21.15.87:0/350381399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc36ffff640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 -- 172.21.15.87:0/350381399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 msgr2=0x55de6c7b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 0x55de6c7b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 -- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 msgr2=0x55de6c7b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 -- 172.21.15.87:0/350381399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6c427210 con 0x55de6c7ba350 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 0x55de6c7b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc37486a640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc368035000 tx=0x7fc36802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0e6cf85bf11807 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36805a060 con 0x55de6c7ba350 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fc368045030 con 0x55de6c7ba350 2024-06-06T07:56:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3640d7040 con 0x55de6c7ba350 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc368061050 con 0x55de6c7ba350 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc378fda740 10 client.16938 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55de6c4f3620 con 0x55de6c7ba350 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.206+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640d7040 con 0x55de6c7ba350 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc368030070 con 0x55de6c7ba350 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 10 client.16938 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 20 client.16938 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c7ca4d0 0x55de6c7cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 4 client.16938 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de6c726f10 con 0x55de6c7ca4d0 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc36ffff640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c7ca4d0 0x55de6c7cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc34f7fe640 10 client.16938 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.207+0000 7fc36ffff640 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c7ca4d0 0x55de6c7cc8c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1454a6393187cfa1 server_cookie=f80d7d191aedc4cd in_seq=0 out_seq=0 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.208+0000 7fc34f7fe640 1 -- 172.21.15.87:0/350381399 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de6c726f10 con 0x55de6c7ca4d0 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.208+0000 7fc34f7fe640 10 client.16938 handle_command_reply: tid=0 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.208+0000 7fc378fda740 1 client.16938 shutdown 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 msgr2=0x55de6c7b7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc368035000 tx=0x7fc36802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 shutdown_connections 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c7ca4d0 0x55de6c7cc8c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7b7d30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7afa90 0x55de6c7b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 --2- 172.21.15.87:0/350381399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 >> 172.21.15.87:0/350381399 conn(0x55de6c7ab680 msgr2=0x55de6c7b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc34f7fe640 0 client.16938 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 shutdown_connections 2024-06-06T07:56:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 1 -- 172.21.15.87:0/350381399 wait complete. 2024-06-06T07:56:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.209+0000 7fc378fda740 20 client.16938 trim_cache size 0 max 16384 2024-06-06T07:56:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 Processor -- start 2024-06-06T07:56:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 -- start start 2024-06-06T07:56:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 0x55de6c7b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701af5f0 con 0x55de6c7afa90 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701acec0 con 0x55de6c7af170 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc378fda740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701aad10 con 0x55de6c7ba350 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59626/0 (socket says 172.21.15.87:59626) 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.225+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1131937931 learned_addr learned my addr 172.21.15.87:0/1131937931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36ffff640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc37486a640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 0x55de6c7b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1131937931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 msgr2=0x55de6c7b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 0x55de6c7b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1131937931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 msgr2=0x55de6c7d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1131937931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6c7af8c0 con 0x55de6c7afa90 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc37486a640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 0x55de6c7b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c6a0 tx=0x7fc364009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0dc5507df5f76b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc355ffb640 1 -- 172.21.15.87:0/1131937931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36409b5f0 con 0x55de6c7afa90 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc355ffb640 1 -- 172.21.15.87:0/1131937931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640d33b0 con 0x55de6c7afa90 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.226+0000 7fc355ffb640 1 -- 172.21.15.87:0/1131937931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36409ba60 con 0x55de6c7afa90 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- 172.21.15.87:0/1131937931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 msgr2=0x55de6c7d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c6a0 tx=0x7fc364009310 comp rx=0 tx=0).stop 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- 172.21.15.87:0/1131937931 shutdown_connections 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7ba350 0x55de6c7b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7afa90 0x55de6c7d2710 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 --2- 172.21.15.87:0/1131937931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7af170 0x55de6c7d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- 172.21.15.87:0/1131937931 >> 172.21.15.87:0/1131937931 conn(0x55de6c7ab680 msgr2=0x55de6c7b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- 172.21.15.87:0/1131937931 shutdown_connections 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- 172.21.15.87:0/1131937931 wait complete. 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 -- start start 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.227+0000 7fc378fda740 1 Processor -- start 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701ab380 con 0x55de6c7ba350 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701ac7f0 con 0x55de6c7afa90 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701aba50 con 0x55de6c7af170 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc378fda740 10 client.0 _ll_register_callbacks cb 0x7fc368009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:33496/0 (socket says 172.21.15.87:33496) 2024-06-06T07:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1554680360 learned_addr learned my addr 172.21.15.87:0/1554680360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc37486a640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1554680360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 msgr2=0x55de6c7c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1554680360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 msgr2=0x55de6c7bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc36f7fe640 1 -- 172.21.15.87:0/1554680360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6c7be050 con 0x55de6c7afa90 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.228+0000 7fc37486a640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc36f7fe640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc364018460 tx=0x7fc3640a4910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4f96e7a73b6d5e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:56:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364097f30 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fc364098300 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3640d33b0 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc36409ba60 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 10 client.26117 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55de6c834bb0 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640d33b0 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc36409bef0 con 0x55de6c7afa90 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 10 client.26117 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 20 client.26117 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c834dd0 0x55de6c8371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 4 client.26117 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.229+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de6c720c20 con 0x55de6c834dd0 2024-06-06T07:56:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.230+0000 7fc36ffff640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c834dd0 0x55de6c8371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.230+0000 7fc36ffff640 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c834dd0 0x55de6c8371c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af89a70d2e48b201 server_cookie=efbb1ceb0be664f8 in_seq=0 out_seq=0 2024-06-06T07:56:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.230+0000 7fc34f7fe640 10 client.26117 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.316+0000 7fc34f7fe640 1 -- 172.21.15.87:0/1554680360 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de6c720c20 con 0x55de6c834dd0 2024-06-06T07:56:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc34f7fe640 10 client.26117 handle_command_reply: tid=0 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 client.26117 shutdown 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 msgr2=0x55de6c7c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc364018460 tx=0x7fc3640a4910 comp rx=0 tx=0).stop 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 shutdown_connections 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55de6c834dd0 0x55de6c8371c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55de6c7ba350 0x55de6c7bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55de6c7afa90 0x55de6c7c2890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 --2- 172.21.15.87:0/1554680360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de6c7af170 0x55de6c7c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc34f7fe640 0 client.26117 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 >> 172.21.15.87:0/1554680360 conn(0x55de6c7ab680 msgr2=0x55de6c7b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 shutdown_connections 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 1 -- 172.21.15.87:0/1554680360 wait complete. 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.317+0000 7fc378fda740 20 client.26117 trim_cache size 0 max 16384 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:56:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:56:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 msgr2=0x7fc3580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 0x7fc3580a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc368004ab0 tx=0x7fc368004a40 comp rx=0 tx=0).stop 2024-06-06T07:56:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 msgr2=0x7fc370198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e9f0 tx=0x7fc36400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 shutdown_connections 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc3580a2e20 0x7fc3580a52e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3701068c0 0x7fc37019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc370101270 0x7fc370198740 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 --2- 172.21.15.87:0/2882917216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3701008b0 0x7fc370198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 >> 172.21.15.87:0/2882917216 conn(0x7fc370077ab0 msgr2=0x7fc3700ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 shutdown_connections 2024-06-06T07:56:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:39.318+0000 7fc3762cd640 1 -- 172.21.15.87:0/2882917216 wait complete. 2024-06-06T07:56:39.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 439864 6.01 MB/sec execute 283 sec latency 47.897 ms 2024-06-06T07:56:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:40 smithi195 ceph-mon[37040]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 788 KiB/s wr, 45 op/s 2024-06-06T07:56:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:40 smithi181 ceph-mon[36790]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 788 KiB/s wr, 45 op/s 2024-06-06T07:56:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:40 smithi087 ceph-mon[32276]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 788 KiB/s wr, 45 op/s 2024-06-06T07:56:40.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 441374 6.02 MB/sec execute 284 sec latency 17.378 ms 2024-06-06T07:56:41.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 443759 6.04 MB/sec execute 285 sec latency 15.316 ms 2024-06-06T07:56:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:42 smithi195 ceph-mon[37040]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 747 KiB/s wr, 42 op/s 2024-06-06T07:56:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:42 smithi181 ceph-mon[36790]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 747 KiB/s wr, 42 op/s 2024-06-06T07:56:42.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:42 smithi087 ceph-mon[32276]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 747 KiB/s wr, 42 op/s 2024-06-06T07:56:42.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 445740 6.05 MB/sec execute 286 sec latency 32.443 ms 2024-06-06T07:56:43.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 447752 6.07 MB/sec execute 287 sec latency 16.590 ms 2024-06-06T07:56:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:44 smithi195 ceph-mon[37040]: pgmap v269: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-06T07:56:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:44 smithi181 ceph-mon[36790]: pgmap v269: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-06T07:56:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:44 smithi087 ceph-mon[32276]: pgmap v269: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-06T07:56:44.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 450241 6.10 MB/sec execute 288 sec latency 32.160 ms 2024-06-06T07:56:45.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 452014 6.11 MB/sec execute 289 sec latency 16.204 ms 2024-06-06T07:56:46.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:46 smithi195 ceph-mon[37040]: pgmap v270: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-06T07:56:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:46 smithi181 ceph-mon[36790]: pgmap v270: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-06T07:56:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:46 smithi087 ceph-mon[32276]: pgmap v270: 129 pgs: 129 active+clean; 54 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-06T07:56:46.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 454365 6.14 MB/sec execute 290 sec latency 16.671 ms 2024-06-06T07:56:47.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 455864 6.14 MB/sec execute 291 sec latency 42.034 ms 2024-06-06T07:56:48.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:48 smithi195 ceph-mon[37040]: pgmap v271: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-06-06T07:56:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:48 smithi181 ceph-mon[36790]: pgmap v271: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-06-06T07:56:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:48 smithi087 ceph-mon[32276]: pgmap v271: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-06-06T07:56:48.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 458082 6.16 MB/sec execute 292 sec latency 17.468 ms 2024-06-06T07:56:49.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 459324 6.22 MB/sec execute 293 sec latency 58.766 ms 2024-06-06T07:56:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:50 smithi195 ceph-mon[37040]: pgmap v272: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:50 smithi181 ceph-mon[36790]: pgmap v272: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:50 smithi087 ceph-mon[32276]: pgmap v272: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:50.674 INFO:tasks.workunit.client.0.smithi087.stdout: 1 461988 6.25 MB/sec execute 294 sec latency 27.537 ms 2024-06-06T07:56:51.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 463749 6.25 MB/sec execute 295 sec latency 16.893 ms 2024-06-06T07:56:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 -- 172.21.15.87:0/1685452977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 msgr2=0x7f481810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 --2- 172.21.15.87:0/1685452977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f481810d750 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4810009cd0 tx=0x7f481002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 -- 172.21.15.87:0/1685452977 shutdown_connections 2024-06-06T07:56:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 --2- 172.21.15.87:0/1685452977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f481810d750 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 --2- 172.21.15.87:0/1685452977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48181008d0 0x7f4818100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 --2- 172.21.15.87:0/1685452977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4818106890 0x7f4818106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 -- 172.21.15.87:0/1685452977 >> 172.21.15.87:0/1685452977 conn(0x7f4818077a20 msgr2=0x7f4818077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 -- 172.21.15.87:0/1685452977 shutdown_connections 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.422+0000 7f481fbf9640 1 -- 172.21.15.87:0/1685452977 wait complete. 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 Processor -- start 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 -- start start 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48181008d0 0x7f4818195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818106890 0x7f481819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f481810fb90 con 0x7f4818101290 2024-06-06T07:56:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f481810fa10 con 0x7f4818106890 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481fbf9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f481810fd10 con 0x7f48181008d0 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481d194640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481d194640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54418/0 (socket says 172.21.15.87:54418) 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481d194640 1 -- 172.21.15.87:0/899805062 learned_addr learned my addr 172.21.15.87:0/899805062 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.423+0000 7f481e196640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818106890 0x7f481819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d995640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48181008d0 0x7f4818195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 -- 172.21.15.87:0/899805062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48181008d0 msgr2=0x7f4818195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48181008d0 0x7f4818195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 -- 172.21.15.87:0/899805062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818106890 msgr2=0x7f481819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818106890 0x7f481819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 -- 172.21.15.87:0/899805062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481819af60 con 0x7f4818101290 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481d194640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9f0 tx=0x7f481400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f480affd640 1 -- 172.21.15.87:0/899805062 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481400cd80 con 0x7f4818101290 2024-06-06T07:56:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f480affd640 1 -- 172.21.15.87:0/899805062 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4814010910 con 0x7f4818101290 2024-06-06T07:56:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481819b250 con 0x7f4818101290 2024-06-06T07:56:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f480affd640 1 -- 172.21.15.87:0/899805062 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814018990 con 0x7f4818101290 2024-06-06T07:56:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.424+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181a2ae0 con 0x7f4818101290 2024-06-06T07:56:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.426+0000 7f480affd640 1 -- 172.21.15.87:0/899805062 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4814010ac0 con 0x7f4818101290 2024-06-06T07:56:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 Processor -- start 2024-06-06T07:56:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 -- start start 2024-06-06T07:56:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4a90 0x564ad31bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31c1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564ad31184f0 con 0x564ad31b4170 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ad30b6870 con 0x564ad31bf350 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f48229e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ad31144b0 con 0x564ad31b4a90 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54422/0 (socket says 172.21.15.87:54422) 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 -- 172.21.15.87:0/4070044496 learned_addr learned my addr 172.21.15.87:0/4070044496 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481d194640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481d995640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4a90 0x564ad31bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f480affd640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 0x7f48040a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 -- 172.21.15.87:0/4070044496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4a90 msgr2=0x564ad31bee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4a90 0x564ad31bee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 -- 172.21.15.87:0/4070044496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 msgr2=0x564ad31c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31c1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.427+0000 7f481e196640 1 -- 172.21.15.87:0/4070044496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad30d29b0 con 0x564ad31b4170 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f481d194640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f481e196640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 0x7f48040a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f480affd640 1 -- 172.21.15.87:0/899805062 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7f48140cfb40 con 0x7f4818101290 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f481e196640 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4810004ab0 tx=0x7f4810031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc0b64eab003698 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f4803fff640 1 -- 172.21.15.87:0/4070044496 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810041040 con 0x564ad31b4170 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f4803fff640 1 -- 172.21.15.87:0/4070044496 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4810043070 con 0x564ad31b4170 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f4803fff640 1 -- 172.21.15.87:0/4070044496 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810030b80 con 0x564ad31b4170 2024-06-06T07:56:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.428+0000 7f481e196640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 0x7f48040a74f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4810005fe0 tx=0x7f481004d000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 -- 172.21.15.87:0/4070044496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 msgr2=0x564ad31b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4810004ab0 tx=0x7f4810031430 comp rx=0 tx=0).stop 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 -- 172.21.15.87:0/4070044496 shutdown_connections 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4a90 0x564ad31bee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 --2- 172.21.15.87:0/4070044496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b4550 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 -- 172.21.15.87:0/4070044496 >> 172.21.15.87:0/4070044496 conn(0x564ad31b0680 msgr2=0x564ad31b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 -- 172.21.15.87:0/4070044496 shutdown_connections 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.429+0000 7f48229e9740 1 -- 172.21.15.87:0/4070044496 wait complete. 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 -- start start 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 Processor -- start 2024-06-06T07:56:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4170 0x564ad31bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564ad31c1f80 con 0x564ad31b4a90 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ad30a1ae0 con 0x564ad31bf350 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ad31c2100 con 0x564ad31b4170 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f48229e9740 10 client.0 _ll_register_callbacks cb 0x564ad30da9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f481d194640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f481d194640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54436/0 (socket says 172.21.15.87:54436) 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f481e196640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f481d194640 1 -- 172.21.15.87:0/3992479557 learned_addr learned my addr 172.21.15.87:0/3992479557 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.430+0000 7f481d995640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4170 0x564ad31bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 -- 172.21.15.87:0/3992479557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4170 msgr2=0x564ad31bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4170 0x564ad31bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 -- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 msgr2=0x564ad31bcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31bcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 -- 172.21.15.87:0/3992479557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad2e2c210 con 0x564ad31b4a90 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f481d194640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f481400c6a0 tx=0x7f4814009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5dd3e3d68fa9e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481409b5f0 con 0x564ad31b4a90 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7f4814018b30 con 0x564ad31b4a90 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4810043070 con 0x564ad31b4a90 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f481409c390 con 0x564ad31b4a90 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 10 client.16956 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564ad2ef8620 con 0x564ad31b4a90 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4810043070 con 0x564ad31b4a90 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f481409b0e0 con 0x564ad31b4a90 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 10 client.16956 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 20 client.16956 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad31cf530 0x564ad31d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.431+0000 7f48229e9740 4 client.16956 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.432+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad312bf10 con 0x564ad31cf530 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.432+0000 7f481d995640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad31cf530 0x564ad31d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.432+0000 7f48017fa640 10 client.16956 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.432+0000 7f481d995640 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad31cf530 0x564ad31d1920 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26416816872c448b server_cookie=c21cf4c96da4f4b3 in_seq=0 out_seq=0 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48017fa640 1 -- 172.21.15.87:0/3992479557 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ad312bf10 con 0x564ad31cf530 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48017fa640 10 client.16956 handle_command_reply: tid=0 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 client.16956 shutdown 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 msgr2=0x564ad31bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f481400c6a0 tx=0x7f4814009310 comp rx=0 tx=0).stop 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 shutdown_connections 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad31cf530 0x564ad31d1920 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31bf350 0x564ad31bcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4a90 0x564ad31bc7e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 --2- 172.21.15.87:0/3992479557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31b4170 0x564ad31bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48017fa640 0 client.16956 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 >> 172.21.15.87:0/3992479557 conn(0x564ad31b0680 msgr2=0x564ad31b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 shutdown_connections 2024-06-06T07:56:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.433+0000 7f48229e9740 1 -- 172.21.15.87:0/3992479557 wait complete. 2024-06-06T07:56:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.434+0000 7f48229e9740 20 client.16956 trim_cache size 0 max 16384 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 Processor -- start 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 -- start start 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48181af4e0 con 0x564ad31b4170 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48181acdb0 con 0x564ad31b4a90 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f48229e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48181aac00 con 0x564ad31bf350 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d995640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d995640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54446/0 (socket says 172.21.15.87:54446) 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d995640 1 -- 172.21.15.87:0/2054722367 learned_addr learned my addr 172.21.15.87:0/2054722367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481e196640 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 -- 172.21.15.87:0/2054722367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 msgr2=0x564ad31d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 -- 172.21.15.87:0/2054722367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 msgr2=0x564ad31b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d194640 1 -- 172.21.15.87:0/2054722367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad31d1c90 con 0x564ad31b4a90 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.450+0000 7f481d995640 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f481d194640 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31be1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4814009380 tx=0x7f4814098510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ceede9978f740fa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48037fe640 1 -- 172.21.15.87:0/2054722367 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481409ba60 con 0x564ad31b4a90 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48037fe640 1 -- 172.21.15.87:0/2054722367 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481409bc90 con 0x564ad31b4a90 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48037fe640 1 -- 172.21.15.87:0/2054722367 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814018b30 con 0x564ad31b4a90 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 -- 172.21.15.87:0/2054722367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 msgr2=0x564ad31be1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31be1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4814009380 tx=0x7f4814098510 comp rx=0 tx=0).stop 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 -- 172.21.15.87:0/2054722367 shutdown_connections 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31be1e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 --2- 172.21.15.87:0/2054722367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.451+0000 7f48229e9740 1 -- 172.21.15.87:0/2054722367 >> 172.21.15.87:0/2054722367 conn(0x564ad31b0680 msgr2=0x564ad31bdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.452+0000 7f48229e9740 1 -- 172.21.15.87:0/2054722367 shutdown_connections 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.452+0000 7f48229e9740 1 -- 172.21.15.87:0/2054722367 wait complete. 2024-06-06T07:56:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.452+0000 7f48229e9740 1 -- start start 2024-06-06T07:56:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.452+0000 7f48229e9740 1 Processor -- start 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48181ab270 con 0x564ad31b4170 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48181ac6e0 con 0x564ad31b4a90 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48181ab940 con 0x564ad31bf350 2024-06-06T07:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d995640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 10 client.0 _ll_register_callbacks cb 0x7f481400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d995640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54458/0 (socket says 172.21.15.87:54458) 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d995640 1 -- 172.21.15.87:0/3446162309 learned_addr learned my addr 172.21.15.87:0/3446162309 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481e196640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 -- 172.21.15.87:0/3446162309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 msgr2=0x564ad31c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 -- 172.21.15.87:0/3446162309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 msgr2=0x564ad31c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 -- 172.21.15.87:0/3446162309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad31c3170 con 0x564ad31b4a90 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481e196640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:56:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f481d194640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31c7a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4814009380 tx=0x7f48140ccf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b96969d28cd153 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48017fa640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140b8020 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7f4814010430 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.453+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481409bc90 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f481409ba60 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 10 client.26135 fetch_fsmap learned FSMap version 34 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564ad31c3170 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481409bc90 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f48140a11d0 con 0x564ad31b4a90 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 10 client.26135 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:56:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 20 client.26135 populate_metadata read hostname 'smithi087' 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad323a030 0x564ad323c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 4 client.26135 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ad3125c20 con 0x564ad323a030 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.454+0000 7f481d995640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad323a030 0x564ad323c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.455+0000 7f48017fa640 10 client.26135 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.455+0000 7f481d995640 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad323a030 0x564ad323c420 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9285aedd4322465e server_cookie=d0f42db8d9fe494d in_seq=0 out_seq=0 2024-06-06T07:56:52.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:52 smithi195 ceph-mon[37040]: pgmap v273: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:52 smithi181 ceph-mon[36790]: pgmap v273: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:52 smithi087 ceph-mon[32276]: pgmap v273: 129 pgs: 129 active+clean; 62 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-06T07:56:52.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 465733 6.28 MB/sec execute 296 sec latency 135.511 ms 2024-06-06T07:56:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48017fa640 1 -- 172.21.15.87:0/3446162309 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564ad3125c20 con 0x564ad323a030 2024-06-06T07:56:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48017fa640 10 client.26135 handle_command_reply: tid=0 2024-06-06T07:56:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 client.26135 shutdown 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 msgr2=0x564ad31c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31c7a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4814009380 tx=0x7f48140ccf90 comp rx=0 tx=0).stop 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 shutdown_connections 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564ad323a030 0x564ad323c420 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ad31bf350 0x564ad31c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ad31b4a90 0x564ad31c7a60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48017fa640 0 client.26135 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 --2- 172.21.15.87:0/3446162309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564ad31b4170 0x564ad31c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 >> 172.21.15.87:0/3446162309 conn(0x564ad31b0680 msgr2=0x564ad31bcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 shutdown_connections 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 1 -- 172.21.15.87:0/3446162309 wait complete. 2024-06-06T07:56:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.714+0000 7f48229e9740 20 client.26135 trim_cache size 0 max 16384 2024-06-06T07:56:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:56:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:56:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 msgr2=0x7f48040a74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 0x7f48040a74f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4810005fe0 tx=0x7f481004d000 comp rx=0 tx=0).stop 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 msgr2=0x7f48181964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9f0 tx=0x7f481400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 shutdown_connections 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f48040a5030 0x7f48040a74f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818106890 0x7f481819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4818101290 0x7f48181964b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 --2- 172.21.15.87:0/899805062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48181008d0 0x7f4818195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:56:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 >> 172.21.15.87:0/899805062 conn(0x7f4818077a20 msgr2=0x7f48181033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:56:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 shutdown_connections 2024-06-06T07:56:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:56:52.715+0000 7f481fbf9640 1 -- 172.21.15.87:0/899805062 wait complete. 2024-06-06T07:56:53.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 467743 6.28 MB/sec execute 297 sec latency 23.605 ms 2024-06-06T07:56:54.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:54 smithi195 ceph-mon[37040]: pgmap v274: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-06-06T07:56:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:54 smithi181 ceph-mon[36790]: pgmap v274: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-06-06T07:56:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:54 smithi087 ceph-mon[32276]: pgmap v274: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-06-06T07:56:54.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 469800 6.30 MB/sec execute 298 sec latency 22.320 ms 2024-06-06T07:56:55.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 472509 6.34 MB/sec execute 299 sec latency 19.489 ms 2024-06-06T07:56:56.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:56 smithi195 ceph-mon[37040]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-06-06T07:56:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:56 smithi181 ceph-mon[36790]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-06-06T07:56:56.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:56 smithi087 ceph-mon[32276]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-06-06T07:56:56.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 474062 6.34 MB/sec execute 300 sec latency 20.443 ms 2024-06-06T07:56:57.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 476347 6.36 MB/sec execute 301 sec latency 114.584 ms 2024-06-06T07:56:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:56:58 smithi195 ceph-mon[37040]: pgmap v276: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-06-06T07:56:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:56:58 smithi181 ceph-mon[36790]: pgmap v276: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-06-06T07:56:58.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:56:58 smithi087 ceph-mon[32276]: pgmap v276: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-06-06T07:56:58.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 478329 6.36 MB/sec execute 302 sec latency 16.323 ms 2024-06-06T07:56:59.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 480345 6.39 MB/sec execute 303 sec latency 22.524 ms 2024-06-06T07:57:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:00 smithi195 ceph-mon[37040]: pgmap v277: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:00 smithi181 ceph-mon[36790]: pgmap v277: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:00 smithi087 ceph-mon[32276]: pgmap v277: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:00.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 482864 6.41 MB/sec execute 304 sec latency 16.986 ms 2024-06-06T07:57:01.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483213 6.40 MB/sec execute 305 sec latency 896.733 ms 2024-06-06T07:57:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:02 smithi195 ceph-mon[37040]: pgmap v278: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:02 smithi181 ceph-mon[36790]: pgmap v278: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:02 smithi087 ceph-mon[32276]: pgmap v278: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-06T07:57:02.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483213 6.38 MB/sec execute 306 sec latency 1896.828 ms 2024-06-06T07:57:03.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483331 6.36 MB/sec execute 307 sec latency 2065.064 ms 2024-06-06T07:57:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:04 smithi195 ceph-mon[37040]: pgmap v279: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-06T07:57:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:04 smithi181 ceph-mon[36790]: pgmap v279: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-06T07:57:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:04 smithi087 ceph-mon[32276]: pgmap v279: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-06-06T07:57:04.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483331 6.34 MB/sec execute 308 sec latency 1764.528 ms 2024-06-06T07:57:05.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483331 6.32 MB/sec execute 309 sec latency 2764.626 ms 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 -- 172.21.15.87:0/599123164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 msgr2=0x7fe098101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/599123164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe098101e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009930 tx=0x7fe08c02a420 comp rx=0 tx=0).stop 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 -- 172.21.15.87:0/599123164 shutdown_connections 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/599123164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098102350 0x7fe09810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/599123164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe098101e10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/599123164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0981079a0 0x7fe098107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 -- 172.21.15.87:0/599123164 >> 172.21.15.87:0/599123164 conn(0x7fe0980fd440 msgr2=0x7fe0980ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 -- 172.21.15.87:0/599123164 shutdown_connections 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.813+0000 7fe09c9b0640 1 -- 172.21.15.87:0/599123164 wait complete. 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 Processor -- start 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 -- start start 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe098102350 0x7fe098198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 0x7fe09819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111db0 con 0x7fe098101990 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111c30 con 0x7fe0981079a0 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.814+0000 7fe09c9b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111f30 con 0x7fe098102350 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44668/0 (socket says 172.21.15.87:44668) 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 -- 172.21.15.87:0/1841503933 learned_addr learned my addr 172.21.15.87:0/1841503933 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe095d9b640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe098102350 0x7fe098198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 0x7fe09819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 -- 172.21.15.87:0/1841503933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe098102350 msgr2=0x7fe098198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe098102350 0x7fe098198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 -- 172.21.15.87:0/1841503933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 msgr2=0x7fe09819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 0x7fe09819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 -- 172.21.15.87:0/1841503933 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819d290 con 0x7fe098101990 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 0x7fe09819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09659c640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe08800d770 tx=0x7fe08800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe0877fe640 1 -- 172.21.15.87:0/1841503933 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088017070 con 0x7fe098101990 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe0877fe640 1 -- 172.21.15.87:0/1841503933 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe088005500 con 0x7fe098101990 2024-06-06T07:57:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09819d580 con 0x7fe098101990 2024-06-06T07:57:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe0877fe640 1 -- 172.21.15.87:0/1841503933 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004e00 con 0x7fe098101990 2024-06-06T07:57:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.815+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a4e10 con 0x7fe098101990 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.818+0000 7fe09f79f740 1 Processor -- start 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe0877fe640 1 -- 172.21.15.87:0/1841503933 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe088004fa0 con 0x7fe098101990 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 -- start start 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561970 0x555cef570220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef572b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555cef4c54f0 con 0x555cef561050 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555cef463870 con 0x555cef570760 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09f79f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555cef4c14b0 con 0x555cef561970 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44684/0 (socket says 172.21.15.87:44684) 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe096d9d640 1 -- 172.21.15.87:0/817415455 learned_addr learned my addr 172.21.15.87:0/817415455 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe09659c640 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561970 0x555cef570220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.819+0000 7fe095d9b640 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef572b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe0877fe640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 0x7fe0800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 -- 172.21.15.87:0/817415455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561970 msgr2=0x555cef570220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561970 0x555cef570220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 -- 172.21.15.87:0/817415455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 msgr2=0x555cef572b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef572b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 -- 172.21.15.87:0/817415455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cef47f9b0 con 0x555cef561050 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe0877fe640 1 -- 172.21.15.87:0/1841503933 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fe0880cf1f0 con 0x7fe098101990 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 0x7fe0800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe096d9d640 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e480 tx=0x7fe09000e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e75f25d895c46f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe084ff9640 1 -- 172.21.15.87:0/817415455 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090022070 con 0x555cef561050 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe084ff9640 1 -- 172.21.15.87:0/817415455 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe090018070 con 0x555cef561050 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.820+0000 7fe084ff9640 1 -- 172.21.15.87:0/817415455 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090005640 con 0x555cef561050 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- 172.21.15.87:0/817415455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 msgr2=0x555cef561430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 0x7fe0800a5260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe09001f000 tx=0x7fe090020000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e480 tx=0x7fe09000e880 comp rx=0 tx=0).stop 2024-06-06T07:57:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- 172.21.15.87:0/817415455 shutdown_connections 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef572b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561970 0x555cef570220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 --2- 172.21.15.87:0/817415455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561050 0x555cef561430 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- 172.21.15.87:0/817415455 >> 172.21.15.87:0/817415455 conn(0x555cef55d680 msgr2=0x555cef55da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- 172.21.15.87:0/817415455 shutdown_connections 2024-06-06T07:57:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- 172.21.15.87:0/817415455 wait complete. 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.821+0000 7fe09f79f740 1 -- start start 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 Processor -- start 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef56d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef56dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555cef573390 con 0x555cef570760 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555cef44eae0 con 0x555cef561970 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555cef573510 con 0x555cef561050 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09f79f740 10 client.0 _ll_register_callbacks cb 0x555cef4879b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44686/0 (socket says 172.21.15.87:44686) 2024-06-06T07:57:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 -- 172.21.15.87:0/901095947 learned_addr learned my addr 172.21.15.87:0/901095947 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe09659c640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef56d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe095d9b640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef56dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 -- 172.21.15.87:0/901095947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 msgr2=0x555cef56d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef56d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 -- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 msgr2=0x555cef56dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef56dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 -- 172.21.15.87:0/901095947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cef1d9210 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe096d9d640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe09000f010 tx=0x7fe09000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f525611cfd9d1a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.822+0000 7fe07dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090032020 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe090018070 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fe090011a30 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe09003a050 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 10 client.16974 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555cef2a5620 con 0x555cef570760 2024-06-06T07:57:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe090018070 con 0x555cef570760 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe090011c30 con 0x555cef570760 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 10 client.16974 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 20 client.16974 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5808f0 0x555cef582ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 4 client.16974 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cef4d8f10 con 0x555cef5808f0 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.823+0000 7fe09659c640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5808f0 0x555cef582ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.824+0000 7fe07dffb640 10 client.16974 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.824+0000 7fe09659c640 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5808f0 0x555cef582ce0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49549f0a7fd9aa5a server_cookie=3e6b67a001139793 in_seq=0 out_seq=0 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe07dffb640 1 -- 172.21.15.87:0/901095947 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555cef4d8f10 con 0x555cef5808f0 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe07dffb640 10 client.16974 handle_command_reply: tid=0 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 client.16974 shutdown 2024-06-06T07:57:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 msgr2=0x555cef56e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe09000f010 tx=0x7fe09000ef30 comp rx=0 tx=0).stop 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 shutdown_connections 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5808f0 0x555cef582ce0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef56e140 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef56dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe07dffb640 0 client.16974 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 --2- 172.21.15.87:0/901095947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef56d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 >> 172.21.15.87:0/901095947 conn(0x555cef55d680 msgr2=0x555cef563470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 shutdown_connections 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 1 -- 172.21.15.87:0/901095947 wait complete. 2024-06-06T07:57:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.825+0000 7fe09f79f740 20 client.16974 trim_cache size 0 max 16384 2024-06-06T07:57:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 Processor -- start 2024-06-06T07:57:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 -- start start 2024-06-06T07:57:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef588340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef55bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981af550 con 0x555cef561970 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ace20 con 0x555cef570760 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe09f79f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981aac70 con 0x555cef561050 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe095d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe095d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44694/0 (socket says 172.21.15.87:44694) 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.840+0000 7fe095d9b640 1 -- 172.21.15.87:0/2302824742 learned_addr learned my addr 172.21.15.87:0/2302824742 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe096d9d640 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef55bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe09659c640 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef588340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 -- 172.21.15.87:0/2302824742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 msgr2=0x555cef588340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef588340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 -- 172.21.15.87:0/2302824742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 msgr2=0x555cef55bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef55bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 -- 172.21.15.87:0/2302824742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cef5617a0 con 0x555cef561970 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe095d9b640 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006010 tx=0x7fe08c031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd61edfddb1b9b77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe07ffff640 1 -- 172.21.15.87:0/2302824742 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c035040 con 0x555cef561970 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe07ffff640 1 -- 172.21.15.87:0/2302824742 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0035f0 con 0x555cef561970 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe07ffff640 1 -- 172.21.15.87:0/2302824742 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0476e0 con 0x555cef561970 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe09f79f740 1 -- 172.21.15.87:0/2302824742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 msgr2=0x555cef588880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.841+0000 7fe09f79f740 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006010 tx=0x7fe08c031660 comp rx=0 tx=0).stop 2024-06-06T07:57:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- 172.21.15.87:0/2302824742 shutdown_connections 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef570760 0x555cef55bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef561970 0x555cef588880 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- 172.21.15.87:0/2302824742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef588340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- 172.21.15.87:0/2302824742 >> 172.21.15.87:0/2302824742 conn(0x555cef55d680 msgr2=0x555cef5689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- 172.21.15.87:0/2302824742 shutdown_connections 2024-06-06T07:57:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- 172.21.15.87:0/2302824742 wait complete. 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- start start 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 Processor -- start 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef578b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef579060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ab2e0 con 0x555cef570760 2024-06-06T07:57:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ac750 con 0x555cef561970 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ab9b0 con 0x555cef561050 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09f79f740 10 client.0 _ll_register_callbacks cb 0x7fe090020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44704/0 (socket says 172.21.15.87:44704) 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe096d9d640 1 -- 172.21.15.87:0/1773704940 learned_addr learned my addr 172.21.15.87:0/1773704940 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe09659c640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef578b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.842+0000 7fe095d9b640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef579060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 -- 172.21.15.87:0/1773704940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 msgr2=0x555cef578b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef578b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 -- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 msgr2=0x555cef579060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef579060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 -- 172.21.15.87:0/1773704940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cef574790 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe095d9b640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef579060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe096d9d640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe09000a440 tx=0x7fe09000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a124abaa58a58d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe07dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090042070 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9495+0+0 (secure 0 0 0) 0x7fe090004c10 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08c0035f0 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe090032040 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe09f79f740 10 client.16980 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555cef5eaf20 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.843+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0035f0 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe09001cad0 con 0x555cef570760 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 10 client.16980 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 20 client.16980 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5eb5e0 0x555cef5eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 4 client.16980 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cef4d2c20 con 0x555cef5eb5e0 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09659c640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5eb5e0 0x555cef5eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe09659c640 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5eb5e0 0x555cef5eda00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fcab8084cdb685 server_cookie=75ec2306ec30763a in_seq=0 out_seq=0 2024-06-06T07:57:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:05.844+0000 7fe07dffb640 10 client.16980 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe07dffb640 1 -- 172.21.15.87:0/1773704940 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555cef4d2c20 con 0x555cef5eb5e0 2024-06-06T07:57:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe07dffb640 10 client.16980 handle_command_reply: tid=0 2024-06-06T07:57:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 client.16980 shutdown 2024-06-06T07:57:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 msgr2=0x555cef573f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe09000a440 tx=0x7fe09000ed80 comp rx=0 tx=0).stop 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 shutdown_connections 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555cef5eb5e0 0x555cef5eda00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555cef570760 0x555cef573f60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555cef561970 0x555cef579060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe07dffb640 0 client.16980 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 --2- 172.21.15.87:0/1773704940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555cef561050 0x555cef578b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 >> 172.21.15.87:0/1773704940 conn(0x555cef55d680 msgr2=0x555cef56d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 shutdown_connections 2024-06-06T07:57:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 1 -- 172.21.15.87:0/1773704940 wait complete. 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.001+0000 7fe09f79f740 20 client.16980 trim_cache size 0 max 16384 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 msgr2=0x7fe0800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 0x7fe0800a5260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe09001f000 tx=0x7fe090020000 comp rx=0 tx=0).stop 2024-06-06T07:57:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 msgr2=0x7fe0981982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe08800d770 tx=0x7fe08800dc30 comp rx=0 tx=0).stop 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 shutdown_connections 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0800a2da0 0x7fe0800a5260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0981079a0 0x7fe09819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe098102350 0x7fe098198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 --2- 172.21.15.87:0/1841503933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe098101990 0x7fe0981982f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 >> 172.21.15.87:0/1841503933 conn(0x7fe0980fd440 msgr2=0x7fe0980fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 shutdown_connections 2024-06-06T07:57:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:06.002+0000 7fe09c9b0640 1 -- 172.21.15.87:0/1841503933 wait complete. 2024-06-06T07:57:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:06 smithi195 ceph-mon[37040]: pgmap v280: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:57:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:06 smithi181 ceph-mon[36790]: pgmap v280: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:57:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:06 smithi087 ceph-mon[32276]: pgmap v280: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:57:06.675 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483331 6.30 MB/sec execute 310 sec latency 3764.725 ms 2024-06-06T07:57:07.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483331 6.28 MB/sec execute 311 sec latency 4764.810 ms 2024-06-06T07:57:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:08 smithi195 ceph-mon[37040]: pgmap v281: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 341 op/s 2024-06-06T07:57:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:08 smithi181 ceph-mon[36790]: pgmap v281: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 341 op/s 2024-06-06T07:57:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:08 smithi087 ceph-mon[32276]: pgmap v281: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 341 op/s 2024-06-06T07:57:08.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 483443 6.26 MB/sec execute 312 sec latency 4925.707 ms 2024-06-06T07:57:09.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 484998 6.26 MB/sec execute 313 sec latency 20.015 ms 2024-06-06T07:57:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:10 smithi195 ceph-mon[37040]: pgmap v282: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:10 smithi181 ceph-mon[36790]: pgmap v282: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:10 smithi087 ceph-mon[32276]: pgmap v282: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:10.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 487354 6.28 MB/sec execute 314 sec latency 15.563 ms 2024-06-06T07:57:11.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 488741 6.28 MB/sec execute 315 sec latency 43.556 ms 2024-06-06T07:57:12.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:12 smithi195 ceph-mon[37040]: pgmap v283: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:12 smithi195 ceph-mon[37040]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:57:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:12 smithi181 ceph-mon[36790]: pgmap v283: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:12 smithi181 ceph-mon[36790]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:57:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:12 smithi087 ceph-mon[32276]: pgmap v283: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 729 KiB/s wr, 154 op/s 2024-06-06T07:57:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:12 smithi087 ceph-mon[32276]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:57:12.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 489748 6.28 MB/sec execute 316 sec latency 53.613 ms 2024-06-06T07:57:13.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 490891 6.29 MB/sec execute 317 sec latency 21.521 ms 2024-06-06T07:57:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:14 smithi195 ceph-mon[37040]: pgmap v285: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:14 smithi181 ceph-mon[36790]: pgmap v285: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:14 smithi087 ceph-mon[32276]: pgmap v285: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:14.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 491032 6.27 MB/sec execute 318 sec latency 292.469 ms 2024-06-06T07:57:15.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 492449 6.26 MB/sec execute 319 sec latency 43.375 ms 2024-06-06T07:57:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:16 smithi195 ceph-mon[37040]: pgmap v286: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:16 smithi181 ceph-mon[36790]: pgmap v286: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:16 smithi087 ceph-mon[32276]: pgmap v286: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 961 KiB/s wr, 191 op/s 2024-06-06T07:57:16.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 494412 6.29 MB/sec execute 320 sec latency 70.722 ms 2024-06-06T07:57:17.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 496260 6.29 MB/sec execute 321 sec latency 31.831 ms 2024-06-06T07:57:18.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:18 smithi195 ceph-mon[37040]: pgmap v287: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:18.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:18 smithi181 ceph-mon[36790]: pgmap v287: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:18 smithi087 ceph-mon[32276]: pgmap v287: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:18.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 498212 6.31 MB/sec execute 322 sec latency 17.768 ms 2024-06-06T07:57:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- 172.21.15.87:0/2026387611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c1079c0 msgr2=0x7f417c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 --2- 172.21.15.87:0/2026387611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c1079c0 0x7f417c107da0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f416c009930 tx=0x7f416c02a420 comp rx=0 tx=0).stop 2024-06-06T07:57:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- 172.21.15.87:0/2026387611 shutdown_connections 2024-06-06T07:57:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 --2- 172.21.15.87:0/2026387611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c102370 0x7f417c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 --2- 172.21.15.87:0/2026387611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 0x7f417c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 --2- 172.21.15.87:0/2026387611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c1079c0 0x7f417c107da0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- 172.21.15.87:0/2026387611 >> 172.21.15.87:0/2026387611 conn(0x7f417c078e30 msgr2=0x7f417c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- 172.21.15.87:0/2026387611 shutdown_connections 2024-06-06T07:57:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- 172.21.15.87:0/2026387611 wait complete. 2024-06-06T07:57:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 Processor -- start 2024-06-06T07:57:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.096+0000 7f4182348640 1 -- start start 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 0x7f417c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c1079c0 0x7f417c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f417c111dd0 con 0x7f417c102370 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f417c111c50 con 0x7f417c1019b0 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f4182348640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f417c111f50 con 0x7f417c1079c0 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45514/0 (socket says 172.21.15.87:45514) 2024-06-06T07:57:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 -- 172.21.15.87:0/933860520 learned_addr learned my addr 172.21.15.87:0/933860520 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417bfff640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 0x7f417c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f41808e5640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c1079c0 0x7f417c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 -- 172.21.15.87:0/933860520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c1079c0 msgr2=0x7f417c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c1079c0 0x7f417c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 -- 172.21.15.87:0/933860520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 msgr2=0x7f417c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 0x7f417c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 -- 172.21.15.87:0/933860520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c19d1c0 con 0x7f417c102370 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.097+0000 7f417b7fe640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9f0 tx=0x7f417000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.098+0000 7f41797fa640 1 -- 172.21.15.87:0/933860520 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000cd80 con 0x7f417c102370 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.098+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c19d4b0 con 0x7f417c102370 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.098+0000 7f41797fa640 1 -- 172.21.15.87:0/933860520 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4170010910 con 0x7f417c102370 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.098+0000 7f41797fa640 1 -- 172.21.15.87:0/933860520 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170018990 con 0x7f417c102370 2024-06-06T07:57:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.098+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a4d40 con 0x7f417c102370 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.100+0000 7f41797fa640 1 -- 172.21.15.87:0/933860520 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4170010ac0 con 0x7f417c102370 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 Processor -- start 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 -- start start 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a02a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41797fa640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 0x7f41640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a037740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x559549f8e4f0 con 0x55954a02aa90 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559549f2c870 con 0x55954a035350 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f418504b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559549f8a4b0 con 0x55954a02a170 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45526/0 (socket says 172.21.15.87:45526) 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f417b7fe640 1 -- 172.21.15.87:0/413218550 learned_addr learned my addr 172.21.15.87:0/413218550 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a037740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f417bfff640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 0x7f41640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41797fa640 1 -- 172.21.15.87:0/933860520 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9671+0+0 (secure 0 0 0) 0x7f41700cfbc0 con 0x7f417c102370 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a02a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 -- 172.21.15.87:0/413218550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 msgr2=0x55954a02a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a02a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 -- 172.21.15.87:0/413218550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 msgr2=0x55954a034e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f41808e5640 1 -- 172.21.15.87:0/413218550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559549f489b0 con 0x55954a035350 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.101+0000 7f417b7fe640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.102+0000 7f41808e5640 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a037740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41740116a0 tx=0x7f4174009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190b6429118c4a31 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.102+0000 7f417bfff640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 0x7f41640a5260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f416c009900 tx=0x7f416c0058d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.102+0000 7f41627fc640 1 -- 172.21.15.87:0/413218550 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004b20 con 0x55954a035350 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.102+0000 7f41627fc640 1 -- 172.21.15.87:0/413218550 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174004cc0 con 0x55954a035350 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.102+0000 7f41627fc640 1 -- 172.21.15.87:0/413218550 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417400f530 con 0x55954a035350 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- 172.21.15.87:0/413218550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 msgr2=0x55954a037740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a037740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41740116a0 tx=0x7f4174009c60 comp rx=0 tx=0).stop 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- 172.21.15.87:0/413218550 shutdown_connections 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a037740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a034e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- 172.21.15.87:0/413218550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a02a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- 172.21.15.87:0/413218550 >> 172.21.15.87:0/413218550 conn(0x55954a026680 msgr2=0x55954a026a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- 172.21.15.87:0/413218550 shutdown_connections 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- 172.21.15.87:0/413218550 wait complete. 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- start start 2024-06-06T07:57:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 Processor -- start 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a02aa90 0x55954a032830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a035350 0x55954a032d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55954a037f80 con 0x55954a02a170 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559549f17ae0 con 0x55954a02aa90 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55954a038100 con 0x55954a035350 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.103+0000 7f418504b740 10 client.0 _ll_register_callbacks cb 0x559549f509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45536/0 (socket says 172.21.15.87:45536) 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 -- 172.21.15.87:0/2262720265 learned_addr learned my addr 172.21.15.87:0/2262720265 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a02aa90 0x55954a032830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f41808e5640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a035350 0x55954a032d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 -- 172.21.15.87:0/2262720265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a035350 msgr2=0x55954a032d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a035350 0x55954a032d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 -- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a02aa90 msgr2=0x55954a032830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a02aa90 0x55954a032830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 -- 172.21.15.87:0/2262720265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559549ca2210 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f417bfff640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f416c033000 tx=0x7f416c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73eff0114b6f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f415b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c04d020 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9671+0+0 (secure 0 0 0) 0x7f416c0035f0 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4174004cc0 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f416c058050 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f418504b740 10 client.16992 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.104+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559549d6e620 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174004cc0 con 0x55954a02a170 2024-06-06T07:57:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f416c049a20 con 0x55954a02a170 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 10 client.16992 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 20 client.16992 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a045580 0x55954a0479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 4 client.16992 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559549fa1f10 con 0x55954a045580 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a045580 0x55954a0479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a045580 0x55954a0479b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f566e6ec99528b server_cookie=689a5562364f87de in_seq=0 out_seq=0 2024-06-06T07:57:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.105+0000 7f415b7fe640 10 client.16992 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.106+0000 7f415b7fe640 1 -- 172.21.15.87:0/2262720265 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559549fa1f10 con 0x55954a045580 2024-06-06T07:57:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.106+0000 7f415b7fe640 10 client.16992 handle_command_reply: tid=0 2024-06-06T07:57:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 client.16992 shutdown 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 msgr2=0x55954a0322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f416c033000 tx=0x7f416c009260 comp rx=0 tx=0).stop 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 shutdown_connections 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a045580 0x55954a0479b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a035350 0x55954a032d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f415b7fe640 0 client.16992 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a02aa90 0x55954a032830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 --2- 172.21.15.87:0/2262720265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a0322f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 >> 172.21.15.87:0/2262720265 conn(0x55954a026680 msgr2=0x55954a02c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 shutdown_connections 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 1 -- 172.21.15.87:0/2262720265 wait complete. 2024-06-06T07:57:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.107+0000 7f418504b740 20 client.16992 trim_cache size 0 max 16384 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 Processor -- start 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 -- start start 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02aa90 0x55954a034230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a04d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1b1a20 con 0x55954a02a170 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1af2f0 con 0x55954a035350 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.123+0000 7f418504b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1ad160 con 0x55954a02aa90 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45550/0 (socket says 172.21.15.87:45550) 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 -- 172.21.15.87:0/2779018777 learned_addr learned my addr 172.21.15.87:0/2779018777 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02aa90 0x55954a034230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f41808e5640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a04d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 -- 172.21.15.87:0/2779018777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02aa90 msgr2=0x55954a034230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02aa90 0x55954a034230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 -- 172.21.15.87:0/2779018777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 msgr2=0x55954a04d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a04d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 -- 172.21.15.87:0/2779018777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954a0278b0 con 0x55954a02a170 2024-06-06T07:57:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f41808e5640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a04d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:57:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f417bfff640 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f416c02a940 tx=0x7f416c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61d4f7a04282571 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f4161ffb640 1 -- 172.21.15.87:0/2779018777 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c04d040 con 0x55954a02a170 2024-06-06T07:57:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f4161ffb640 1 -- 172.21.15.87:0/2779018777 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416c060070 con 0x55954a02a170 2024-06-06T07:57:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.124+0000 7f4161ffb640 1 -- 172.21.15.87:0/2779018777 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c035040 con 0x55954a02a170 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- 172.21.15.87:0/2779018777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 msgr2=0x55954a02cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f416c02a940 tx=0x7f416c0045e0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- 172.21.15.87:0/2779018777 shutdown_connections 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a04d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02aa90 0x55954a034230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 --2- 172.21.15.87:0/2779018777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02a170 0x55954a02cb30 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- 172.21.15.87:0/2779018777 >> 172.21.15.87:0/2779018777 conn(0x55954a026680 msgr2=0x55954a033b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- 172.21.15.87:0/2779018777 shutdown_connections 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- 172.21.15.87:0/2779018777 wait complete. 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.125+0000 7f418504b740 1 -- start start 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 Processor -- start 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a039210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a039c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1ad7b0 con 0x55954a02aa90 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1aec20 con 0x55954a035350 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f417c1ade80 con 0x55954a02a170 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f418504b740 10 client.0 _ll_register_callbacks cb 0x7f416c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a039210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45562/0 (socket says 172.21.15.87:45562) 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 -- 172.21.15.87:0/2248527831 learned_addr learned my addr 172.21.15.87:0/2248527831 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f41808e5640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a039c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 -- 172.21.15.87:0/2248527831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 msgr2=0x55954a039210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a039210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 -- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 msgr2=0x55954a039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a039c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.126+0000 7f417b7fe640 1 -- 172.21.15.87:0/2248527831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954a021f70 con 0x55954a02aa90 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f417b7fe640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f41700986e0 tx=0x7f4170018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7836361a7d857b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41700a0f40 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9671+0+0 (secure 0 0 0) 0x7f417009fdb0 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416c060070 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f41700b4e60 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 10 client.16998 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55954a0b0530 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416c060070 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4170098380 con 0x55954a02aa90 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 10 client.16998 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 20 client.16998 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a0b0ce0 0x55954a0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 4 client.16998 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559549f9bc20 con 0x55954a0b0ce0 2024-06-06T07:57:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.127+0000 7f417bfff640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a0b0ce0 0x55954a0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.128+0000 7f417bfff640 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a0b0ce0 0x55954a0b3100 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf86aab8867a9c6 server_cookie=e613421c5c692e3a in_seq=0 out_seq=0 2024-06-06T07:57:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.128+0000 7f415b7fe640 10 client.16998 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f415b7fe640 1 -- 172.21.15.87:0/2248527831 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559549f9bc20 con 0x55954a0b0ce0 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f415b7fe640 10 client.16998 handle_command_reply: tid=0 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 client.16998 shutdown 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 msgr2=0x55954a039750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f41700986e0 tx=0x7f4170018420 comp rx=0 tx=0).stop 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 shutdown_connections 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55954a0b0ce0 0x55954a0b3100 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55954a035350 0x55954a039c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f415b7fe640 0 client.16998 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55954a02aa90 0x55954a039750 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 --2- 172.21.15.87:0/2248527831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55954a02a170 0x55954a039210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 >> 172.21.15.87:0/2248527831 conn(0x55954a026680 msgr2=0x55954a0343b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 shutdown_connections 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.266+0000 7f418504b740 1 -- 172.21.15.87:0/2248527831 wait complete. 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f418504b740 20 client.16998 trim_cache size 0 max 16384 2024-06-06T07:57:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:57:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:57:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:57:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 msgr2=0x7f41640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 0x7f41640a5260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f416c009900 tx=0x7f416c0058d0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 msgr2=0x7f417c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9f0 tx=0x7f417000eeb0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 shutdown_connections 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41640a2da0 0x7f41640a5260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f417c1079c0 0x7f417c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f417c102370 0x7f417c198710 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 --2- 172.21.15.87:0/933860520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417c1019b0 0x7f417c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 >> 172.21.15.87:0/933860520 conn(0x7f417c078e30 msgr2=0x7f417c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 shutdown_connections 2024-06-06T07:57:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:19.267+0000 7f4182348640 1 -- 172.21.15.87:0/933860520 wait complete. 2024-06-06T07:57:19.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 500380 6.33 MB/sec execute 323 sec latency 65.872 ms 2024-06-06T07:57:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:20 smithi195 ceph-mon[37040]: pgmap v288: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:20 smithi195 ceph-mon[37040]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:57:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:20 smithi181 ceph-mon[36790]: pgmap v288: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:20 smithi181 ceph-mon[36790]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:57:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:20 smithi087 ceph-mon[32276]: pgmap v288: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.7 MiB/s wr, 340 op/s 2024-06-06T07:57:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:20 smithi087 ceph-mon[32276]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:57:20.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 502000 6.33 MB/sec execute 324 sec latency 26.664 ms 2024-06-06T07:57:21.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 504456 6.36 MB/sec execute 325 sec latency 16.817 ms 2024-06-06T07:57:22.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:22 smithi195 ceph-mon[37040]: pgmap v290: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.2 MiB/s wr, 365 op/s 2024-06-06T07:57:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:22 smithi181 ceph-mon[36790]: pgmap v290: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.2 MiB/s wr, 365 op/s 2024-06-06T07:57:22.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:22 smithi087 ceph-mon[32276]: pgmap v290: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.2 MiB/s wr, 365 op/s 2024-06-06T07:57:22.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 505916 6.36 MB/sec execute 326 sec latency 19.125 ms 2024-06-06T07:57:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:57:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:57:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:57:23.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:57:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:57:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:57:23.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:57:23.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:57:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:57:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:57:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:57:23.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:57:23.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 508299 6.38 MB/sec execute 327 sec latency 20.028 ms 2024-06-06T07:57:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:24 smithi195 ceph-mon[37040]: pgmap v291: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:24 smithi181 ceph-mon[36790]: pgmap v291: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:24.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:24 smithi087 ceph-mon[32276]: pgmap v291: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:24.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 510179 6.38 MB/sec execute 328 sec latency 32.367 ms 2024-06-06T07:57:25.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 512192 6.40 MB/sec execute 329 sec latency 54.636 ms 2024-06-06T07:57:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:26 smithi195 ceph-mon[37040]: pgmap v292: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:26 smithi181 ceph-mon[36790]: pgmap v292: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:26 smithi087 ceph-mon[32276]: pgmap v292: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 435 op/s 2024-06-06T07:57:26.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 514364 6.42 MB/sec execute 330 sec latency 24.205 ms 2024-06-06T07:57:27.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 516105 6.43 MB/sec execute 331 sec latency 48.284 ms 2024-06-06T07:57:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:28 smithi195 ceph-mon[37040]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:28 smithi181 ceph-mon[36790]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:28.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:28 smithi087 ceph-mon[32276]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:28.676 INFO:tasks.workunit.client.0.smithi087.stdout: 1 518785 6.46 MB/sec execute 332 sec latency 17.336 ms 2024-06-06T07:57:29.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 520235 6.46 MB/sec execute 333 sec latency 24.652 ms 2024-06-06T07:57:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:30 smithi195 ceph-mon[37040]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:30 smithi181 ceph-mon[36790]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:30 smithi087 ceph-mon[32276]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-06-06T07:57:30.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 522640 6.48 MB/sec execute 334 sec latency 14.405 ms 2024-06-06T07:57:31.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 524561 6.48 MB/sec execute 335 sec latency 38.717 ms 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 -- 172.21.15.87:0/4021519289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 msgr2=0x7f003010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 --2- 172.21.15.87:0/4021519289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f003010e850 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0028009cd0 tx=0x7f002802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 -- 172.21.15.87:0/4021519289 shutdown_connections 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 --2- 172.21.15.87:0/4021519289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f003010e850 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 --2- 172.21.15.87:0/4021519289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030101980 0x7f0030101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 --2- 172.21.15.87:0/4021519289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030107990 0x7f0030107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 -- 172.21.15.87:0/4021519289 >> 172.21.15.87:0/4021519289 conn(0x7f00300fd450 msgr2=0x7f00300ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.357+0000 7f0036f90640 1 -- 172.21.15.87:0/4021519289 shutdown_connections 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 -- 172.21.15.87:0/4021519289 wait complete. 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 Processor -- start 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 -- start start 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030101980 0x7f00301981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030107990 0x7f003019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0030111da0 con 0x7f0030102340 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0030111c20 con 0x7f0030101980 2024-06-06T07:57:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0036f90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0030111f20 con 0x7f0030107990 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f002ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f002ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59162/0 (socket says 172.21.15.87:59162) 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f002ffff640 1 -- 172.21.15.87:0/3745868858 learned_addr learned my addr 172.21.15.87:0/3745868858 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f003552d640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030107990 0x7f003019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.358+0000 7f0034d2c640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030101980 0x7f00301981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 -- 172.21.15.87:0/3745868858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030107990 msgr2=0x7f003019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030107990 0x7f003019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 -- 172.21.15.87:0/3745868858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030101980 msgr2=0x7f00301981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030101980 0x7f00301981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 -- 172.21.15.87:0/3745868858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003019d190 con 0x7f0030102340 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002ffff640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e9f0 tx=0x7f001c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002dffb640 1 -- 172.21.15.87:0/3745868858 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c00cd80 con 0x7f0030102340 2024-06-06T07:57:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003019d480 con 0x7f0030102340 2024-06-06T07:57:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002dffb640 1 -- 172.21.15.87:0/3745868858 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f001c005500 con 0x7f0030102340 2024-06-06T07:57:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f002dffb640 1 -- 172.21.15.87:0/3745868858 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c010690 con 0x7f0030102340 2024-06-06T07:57:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.359+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a4d10 con 0x7f0030102340 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f002dffb640 1 -- 172.21.15.87:0/3745868858 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f001c0046c0 con 0x7f0030102340 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 Processor -- start 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 -- start start 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d5176364f0 con 0x55d5176d2a90 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5175d4870 con 0x55d5176d2170 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.362+0000 7f0039d7c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5176324b0 con 0x55d5176dd350 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f003552d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59166/0 (socket says 172.21.15.87:59166) 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f003552d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:55386/0 (socket says 172.21.15.87:55386) 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 -- 172.21.15.87:0/1878458116 learned_addr learned my addr 172.21.15.87:0/1878458116 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f002ffff640 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f002dffb640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 0x7f00200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f003552d640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 0x7f00200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 -- 172.21.15.87:0/1878458116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 msgr2=0x55d5176df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 -- 172.21.15.87:0/1878458116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 msgr2=0x55d5176d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 -- 172.21.15.87:0/1878458116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5175f09b0 con 0x55d5176d2a90 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f003552d640 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f002dffb640 1 -- 172.21.15.87:0/3745868858 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7f001c0cfc90 con 0x7f0030102340 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0024009c70 tx=0x7f0024011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4190a396f78696da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f003552d640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 0x7f00200a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0028004ab0 tx=0x7f002802a8e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f001affd640 1 -- 172.21.15.87:0/1878458116 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024004230 con 0x55d5176d2a90 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f001affd640 1 -- 172.21.15.87:0/1878458116 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00240043d0 con 0x55d5176d2a90 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.363+0000 7f001affd640 1 -- 172.21.15.87:0/1878458116 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024019740 con 0x55d5176d2a90 2024-06-06T07:57:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- 172.21.15.87:0/1878458116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 msgr2=0x55d5176dce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0024009c70 tx=0x7f0024011810 comp rx=0 tx=0).stop 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- 172.21.15.87:0/1878458116 shutdown_connections 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176dce10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1878458116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2170 0x55d5176d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- 172.21.15.87:0/1878458116 >> 172.21.15.87:0/1878458116 conn(0x55d5176ce680 msgr2=0x55d5176cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- 172.21.15.87:0/1878458116 shutdown_connections 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- 172.21.15.87:0/1878458116 wait complete. 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.364+0000 7f0039d7c740 1 -- start start 2024-06-06T07:57:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 Processor -- start 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2170 0x55d5176da290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d5176dff80 con 0x55d5176d2a90 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5175bfae0 con 0x55d5176dd350 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5176e0100 con 0x55d5176d2170 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0039d7c740 10 client.0 _ll_register_callbacks cb 0x55d5175f89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59174/0 (socket says 172.21.15.87:59174) 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 -- 172.21.15.87:0/840912919 learned_addr learned my addr 172.21.15.87:0/840912919 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f003552d640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f0034d2c640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2170 0x55d5176da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 -- 172.21.15.87:0/840912919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2170 msgr2=0x55d5176da290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2170 0x55d5176da290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 -- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 msgr2=0x55d5176dad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176dad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.365+0000 7f002ffff640 1 -- 172.21.15.87:0/840912919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d51734a210 con 0x55d5176d2a90 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f002ffff640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f001c098760 tx=0x7f001c004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a1d48ac68bee97 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c0cd440 con 0x55d5176d2a90 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7f001c09bff0 con 0x55d5176d2a90 2024-06-06T07:57:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00240043d0 con 0x55d5176d2a90 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f001c0a0140 con 0x55d5176d2a90 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 10 client.17010 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d517416620 con 0x55d5176d2a90 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00240043d0 con 0x55d5176d2a90 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f001c09b1e0 con 0x55d5176d2a90 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 10 client.17010 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 20 client.17010 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d5176ed480 0x55d5176ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 4 client.17010 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d517649f10 con 0x55d5176ed480 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.366+0000 7f0034d2c640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d5176ed480 0x55d5176ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.367+0000 7f0034d2c640 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d5176ed480 0x55d5176ef870 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63dd2c050ba3545 server_cookie=e30a9fd710e0f42c in_seq=0 out_seq=0 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.367+0000 7f000ffff640 10 client.17010 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f000ffff640 1 -- 172.21.15.87:0/840912919 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d517649f10 con 0x55d5176ed480 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f000ffff640 10 client.17010 handle_command_reply: tid=0 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 client.17010 shutdown 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 msgr2=0x55d5176da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f001c098760 tx=0x7f001c004ae0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 shutdown_connections 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d5176ed480 0x55d5176ef870 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2a90 0x55d5176da7d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 --2- 172.21.15.87:0/840912919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2170 0x55d5176da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f000ffff640 0 client.17010 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 >> 172.21.15.87:0/840912919 conn(0x55d5176ce680 msgr2=0x55d5176d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 shutdown_connections 2024-06-06T07:57:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.368+0000 7f0039d7c740 1 -- 172.21.15.87:0/840912919 wait complete. 2024-06-06T07:57:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.369+0000 7f0039d7c740 20 client.17010 trim_cache size 0 max 16384 2024-06-06T07:57:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 Processor -- start 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 -- start start 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2a90 0x55d5176f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00301b1940 con 0x55d5176d2170 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00301af210 con 0x55d5176dd350 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0039d7c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ad080 con 0x55d5176d2a90 2024-06-06T07:57:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59186/0 (socket says 172.21.15.87:59186) 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 -- 172.21.15.87:0/1214034671 learned_addr learned my addr 172.21.15.87:0/1214034671 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f002ffff640 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2a90 0x55d5176f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f003552d640 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 -- 172.21.15.87:0/1214034671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2a90 msgr2=0x55d5176f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2a90 0x55d5176f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 -- 172.21.15.87:0/1214034671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 msgr2=0x55d5176f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.385+0000 7f0034d2c640 1 -- 172.21.15.87:0/1214034671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5176d28c0 con 0x55d5176d2170 2024-06-06T07:57:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0034d2c640 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0024003f40 tx=0x7f00240098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4a46fc2fcf2cf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f001a7fc640 1 -- 172.21.15.87:0/1214034671 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024011d10 con 0x55d5176d2170 2024-06-06T07:57:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f001a7fc640 1 -- 172.21.15.87:0/1214034671 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024005520 con 0x55d5176d2170 2024-06-06T07:57:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f001a7fc640 1 -- 172.21.15.87:0/1214034671 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003cf0 con 0x55d5176d2170 2024-06-06T07:57:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 -- 172.21.15.87:0/1214034671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 msgr2=0x55d5176d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0024003f40 tx=0x7f00240098d0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 -- 172.21.15.87:0/1214034671 shutdown_connections 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176dd350 0x55d5176f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176d2a90 0x55d5176f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 --2- 172.21.15.87:0/1214034671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176d4b30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 -- 172.21.15.87:0/1214034671 >> 172.21.15.87:0/1214034671 conn(0x55d5176ce680 msgr2=0x55d5176d5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.386+0000 7f0039d7c740 1 -- 172.21.15.87:0/1214034671 shutdown_connections 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 -- 172.21.15.87:0/1214034671 wait complete. 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 -- start start 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 Processor -- start 2024-06-06T07:57:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2a90 0x55d5176e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176d6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ad6d0 con 0x55d5176d2170 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aeb40 con 0x55d5176d2a90 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00301adda0 con 0x55d5176dd350 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0039d7c740 10 client.0 _ll_register_callbacks cb 0x7f001c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0034d2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59198/0 (socket says 172.21.15.87:59198) 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f0034d2c640 1 -- 172.21.15.87:0/2583990257 learned_addr learned my addr 172.21.15.87:0/2583990257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f002ffff640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2a90 0x55d5176e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.387+0000 7f003552d640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176d6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 -- 172.21.15.87:0/2583990257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 msgr2=0x55d5176d6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176d6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 -- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2a90 msgr2=0x55d5176e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2a90 0x55d5176e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 -- 172.21.15.87:0/2583990257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5176d6c40 con 0x55d5176d2170 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0034d2c640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0024009970 tx=0x7f0024005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec3cbaf95d765de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003c70 con 0x55d5176d2170 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7f002401f040 con 0x55d5176d2170 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0024005520 con 0x55d5176d2170 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0024041050 con 0x55d5176d2170 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0039d7c740 10 client.17016 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d517757850 con 0x55d5176d2170 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024005520 con 0x55d5176d2170 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0024018070 con 0x55d5176d2170 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0039d7c740 10 client.17016 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.388+0000 7f0039d7c740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f0039d7c740 20 client.17016 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d517757d60 0x55d51775a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f0039d7c740 4 client.17016 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d517643c20 con 0x55d517757d60 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f002ffff640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d517757d60 0x55d51775a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f002ffff640 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d517757d60 0x55d51775a150 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37dccfc9a4292afe server_cookie=f996eb8f602e5822 in_seq=0 out_seq=0 2024-06-06T07:57:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.389+0000 7f000ffff640 10 client.17016 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:32 smithi195 ceph-mon[37040]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-06-06T07:57:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:32 smithi181 ceph-mon[36790]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-06-06T07:57:32.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:32 smithi087 ceph-mon[32276]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f000ffff640 1 -- 172.21.15.87:0/2583990257 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d517643c20 con 0x55d517757d60 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f000ffff640 10 client.17016 handle_command_reply: tid=0 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 client.17016 shutdown 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 msgr2=0x55d5176e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0024009970 tx=0x7f0024005f10 comp rx=0 tx=0).stop 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 shutdown_connections 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d517757d60 0x55d51775a150 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f000ffff640 0 client.17016 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5176dd350 0x55d5176d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5176d2a90 0x55d5176e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 --2- 172.21.15.87:0/2583990257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d5176d2170 0x55d5176e5350 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 >> 172.21.15.87:0/2583990257 conn(0x55d5176ce680 msgr2=0x55d5176dbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 shutdown_connections 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 1 -- 172.21.15.87:0/2583990257 wait complete. 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.674+0000 7f0039d7c740 20 client.17016 trim_cache size 0 max 16384 2024-06-06T07:57:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:57:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:57:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:57:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 msgr2=0x7f00200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 0x7f00200a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0028004ab0 tx=0x7f002802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 msgr2=0x7f00301986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e9f0 tx=0x7f001c00edf0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 shutdown_connections 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f00200a2da0 0x7f00200a5260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0030107990 0x7f003019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0030102340 0x7f00301986e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 --2- 172.21.15.87:0/3745868858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0030101980 0x7f00301981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 >> 172.21.15.87:0/3745868858 conn(0x7f00300fd450 msgr2=0x7f00300fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 shutdown_connections 2024-06-06T07:57:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:32.675+0000 7f0036f90640 1 -- 172.21.15.87:0/3745868858 wait complete. 2024-06-06T07:57:32.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 526212 6.50 MB/sec execute 336 sec latency 89.598 ms 2024-06-06T07:57:33.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 528153 6.50 MB/sec execute 337 sec latency 35.305 ms 2024-06-06T07:57:34.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:34 smithi195 ceph-mon[37040]: pgmap v296: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-06-06T07:57:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:34 smithi181 ceph-mon[36790]: pgmap v296: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-06-06T07:57:34.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:34 smithi087 ceph-mon[32276]: pgmap v296: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-06-06T07:57:34.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 530107 6.52 MB/sec execute 338 sec latency 18.455 ms 2024-06-06T07:57:35.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 532352 6.53 MB/sec execute 339 sec latency 22.955 ms 2024-06-06T07:57:36.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:36 smithi195 ceph-mon[37040]: pgmap v297: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-06-06T07:57:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:36 smithi181 ceph-mon[36790]: pgmap v297: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-06-06T07:57:36.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:36 smithi087 ceph-mon[32276]: pgmap v297: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-06-06T07:57:36.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 534272 6.55 MB/sec execute 340 sec latency 19.753 ms 2024-06-06T07:57:37.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 536668 6.57 MB/sec execute 341 sec latency 15.961 ms 2024-06-06T07:57:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:38 smithi181 ceph-mon[36790]: pgmap v298: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-06-06T07:57:38.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:38 smithi087 ceph-mon[32276]: pgmap v298: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-06-06T07:57:38.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 538346 6.57 MB/sec execute 342 sec latency 23.875 ms 2024-06-06T07:57:38.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:38 smithi195 ceph-mon[37040]: pgmap v298: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-06-06T07:57:39.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 540646 6.59 MB/sec execute 343 sec latency 16.344 ms 2024-06-06T07:57:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:40 smithi181 ceph-mon[36790]: pgmap v299: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:40 smithi087 ceph-mon[32276]: pgmap v299: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:40.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 542988 6.61 MB/sec execute 344 sec latency 16.602 ms 2024-06-06T07:57:40.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:40 smithi195 ceph-mon[37040]: pgmap v299: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:41.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 544704 6.61 MB/sec execute 345 sec latency 17.360 ms 2024-06-06T07:57:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:42 smithi181 ceph-mon[36790]: pgmap v300: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:42.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:42 smithi087 ceph-mon[32276]: pgmap v300: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:42.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 547290 6.64 MB/sec execute 346 sec latency 15.657 ms 2024-06-06T07:57:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:42 smithi195 ceph-mon[37040]: pgmap v300: 129 pgs: 129 active+clean; 85 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-06-06T07:57:43.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 548849 6.64 MB/sec execute 347 sec latency 35.205 ms 2024-06-06T07:57:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:44 smithi181 ceph-mon[36790]: pgmap v301: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-06-06T07:57:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:44 smithi087 ceph-mon[32276]: pgmap v301: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-06-06T07:57:44.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 551162 6.66 MB/sec execute 348 sec latency 16.358 ms 2024-06-06T07:57:44.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:44 smithi195 ceph-mon[37040]: pgmap v301: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-06-06T07:57:45.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 553082 6.66 MB/sec execute 349 sec latency 33.155 ms 2024-06-06T07:57:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 -- 172.21.15.87:0/2995053304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 msgr2=0x7fe6c0106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/2995053304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c0106c70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009930 tx=0x7fe6b402a420 comp rx=0 tx=0).stop 2024-06-06T07:57:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 -- 172.21.15.87:0/2995053304 shutdown_connections 2024-06-06T07:57:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/2995053304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c0101290 0x7fe6c010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/2995053304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c01008d0 0x7fe6c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/2995053304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c0106c70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 -- 172.21.15.87:0/2995053304 >> 172.21.15.87:0/2995053304 conn(0x7fe6c0077a20 msgr2=0x7fe6c0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 -- 172.21.15.87:0/2995053304 shutdown_connections 2024-06-06T07:57:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.772+0000 7fe6c6e09640 1 -- 172.21.15.87:0/2995053304 wait complete. 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 Processor -- start 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 -- start start 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 0x7fe6c01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c01a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110ca0 con 0x7fe6c0106890 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110b20 con 0x7fe6c01008d0 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c6e09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110e20 con 0x7fe6c0101290 2024-06-06T07:57:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c53a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c01a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53302/0 (socket says 172.21.15.87:53302) 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.773+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/3062610124 learned_addr learned my addr 172.21.15.87:0/3062610124 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 0x7fe6c01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/3062610124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 msgr2=0x7fe6c01995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 0x7fe6c01995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 msgr2=0x7fe6c01a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c01a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/3062610124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c01a4640 con 0x7fe6c01008d0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 0x7fe6c01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009230 tx=0x7fe6b4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6bdffb640 1 -- 172.21.15.87:0/3062610124 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b403f030 con 0x7fe6c01008d0 2024-06-06T07:57:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01a4810 con 0x7fe6c01008d0 2024-06-06T07:57:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6bdffb640 1 -- 172.21.15.87:0/3062610124 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4004440 con 0x7fe6c01008d0 2024-06-06T07:57:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.774+0000 7fe6bdffb640 1 -- 172.21.15.87:0/3062610124 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4035070 con 0x7fe6c01008d0 2024-06-06T07:57:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.775+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a4c70 con 0x7fe6c01008d0 2024-06-06T07:57:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 Processor -- start 2024-06-06T07:57:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 -- start start 2024-06-06T07:57:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6bdffb640 1 -- 172.21.15.87:0/3062610124 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe6b4042030 con 0x7fe6c01008d0 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fb6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557a73f0d4f0 con 0x557a73fa9a90 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a73eab870 con 0x557a73fb4350 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557a73f094b0 con 0x557a73fa9170 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6bffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fb6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fb3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55372/0 (socket says 172.21.15.87:55372) 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.777+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/3459465441 learned_addr learned my addr 172.21.15.87:0/3459465441 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 -- 172.21.15.87:0/3459465441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 msgr2=0x557a73fa9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bdffb640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 0x7fe6b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 -- 172.21.15.87:0/3459465441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 msgr2=0x557a73fb3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fb3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 -- 172.21.15.87:0/3459465441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a73ec79b0 con 0x557a73fb4350 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 0x7fe6b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bdffb640 1 -- 172.21.15.87:0/3062610124 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7fe6b40efbf0 con 0x7fe6c01008d0 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fb6740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fa0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5a24a266cf25da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6aaffd640 1 -- 172.21.15.87:0/3459465441 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004310 con 0x557a73fb4350 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6aaffd640 1 -- 172.21.15.87:0/3459465441 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0044b0 con 0x557a73fb4350 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.778+0000 7fe6aaffd640 1 -- 172.21.15.87:0/3459465441 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0035f0 con 0x557a73fb4350 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 0x7fe6b00a5260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005fd0 tx=0x7fe6b8014040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3459465441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 msgr2=0x557a73fb6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fb6740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fa0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3459465441 shutdown_connections 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fb6740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3459465441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3459465441 >> 172.21.15.87:0/3459465441 conn(0x557a73fa5680 msgr2=0x557a73fa5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3459465441 shutdown_connections 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3459465441 wait complete. 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.779+0000 7fe6c9bf0740 1 -- start start 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 Processor -- start 2024-06-06T07:57:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fa9a90 0x557a73fb1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fb4350 0x557a73fb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x557a73fb6f80 con 0x557a73fa9170 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a73e96ae0 con 0x557a73fa9a90 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557a73fb7100 con 0x557a73fb4350 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c9bf0740 10 client.0 _ll_register_callbacks cb 0x557a73ecf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55384/0 (socket says 172.21.15.87:55384) 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/4037592374 learned_addr learned my addr 172.21.15.87:0/4037592374 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6bffff640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fa9a90 0x557a73fb1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.780+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fb4350 0x557a73fb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/4037592374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fb4350 msgr2=0x557a73fb1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fb4350 0x557a73fb1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fa9a90 msgr2=0x557a73fb1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fa9a90 0x557a73fb1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/4037592374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a73c21210 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b8710 tx=0x7fe6b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30f771eac6fcf2d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe69ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4049070 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7fe6b40b83b0 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac0044b0 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6b40bb880 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c9bf0740 10 client.17028 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a73ced620 con 0x557a73fa9170 2024-06-06T07:57:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.781+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0044b0 con 0x557a73fa9170 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe6b4031bd0 con 0x557a73fa9170 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 10 client.17028 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 20 client.17028 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a73fc44a0 0x557a73fc68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 4 client.17028 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a73f20f10 con 0x557a73fc44a0 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6bffff640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a73fc44a0 0x557a73fc68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe69ffff640 10 client.17028 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.782+0000 7fe6bffff640 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a73fc44a0 0x557a73fc68b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ee08f247661673 server_cookie=3b5024cb27ff8577 in_seq=0 out_seq=0 2024-06-06T07:57:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.783+0000 7fe69ffff640 1 -- 172.21.15.87:0/4037592374 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a73f20f10 con 0x557a73fc44a0 2024-06-06T07:57:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.783+0000 7fe69ffff640 10 client.17028 handle_command_reply: tid=0 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.783+0000 7fe6c9bf0740 1 client.17028 shutdown 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 msgr2=0x557a73fb12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b8710 tx=0x7fe6b4005b50 comp rx=0 tx=0).stop 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 shutdown_connections 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a73fc44a0 0x557a73fc68b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fb4350 0x557a73fb1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fa9a90 0x557a73fb1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/4037592374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fb12d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe69ffff640 0 client.17028 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 >> 172.21.15.87:0/4037592374 conn(0x557a73fa5680 msgr2=0x557a73fab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 shutdown_connections 2024-06-06T07:57:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/4037592374 wait complete. 2024-06-06T07:57:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.784+0000 7fe6c9bf0740 20 client.17028 trim_cache size 0 max 16384 2024-06-06T07:57:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 Processor -- start 2024-06-06T07:57:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 -- start start 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fcc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01b17d0 con 0x557a73fa9a90 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01af0a0 con 0x557a73fb4350 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01acf10 con 0x557a73fa9170 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6bffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6bffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55400/0 (socket says 172.21.15.87:55400) 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6bffff640 1 -- 172.21.15.87:0/3347484511 learned_addr learned my addr 172.21.15.87:0/3347484511 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fcc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.801+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 -- 172.21.15.87:0/3347484511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 msgr2=0x557a73fa4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 -- 172.21.15.87:0/3347484511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 msgr2=0x557a73fcc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fcc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 -- 172.21.15.87:0/3347484511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a73fb6740 con 0x557a73fa9a90 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6bffff640 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac005fd0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39621fa3ccccde8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6aa7fc640 1 -- 172.21.15.87:0/3347484511 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac02f020 con 0x557a73fa9a90 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6aa7fc640 1 -- 172.21.15.87:0/3347484511 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0047c0 con 0x557a73fa9a90 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.802+0000 7fe6aa7fc640 1 -- 172.21.15.87:0/3347484511 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac01d040 con 0x557a73fa9a90 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3347484511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 msgr2=0x557a73fcc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac005fd0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3347484511 shutdown_connections 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fcc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9a90 0x557a73fcc160 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/3347484511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9170 0x557a73fa4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3347484511 >> 172.21.15.87:0/3347484511 conn(0x557a73fa5680 msgr2=0x557a73fa3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3347484511 shutdown_connections 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/3347484511 wait complete. 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 -- start start 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 Processor -- start 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.803+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 0x557a73fadfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01ad560 con 0x557a73fa9170 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01ae9d0 con 0x557a73fb4350 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01adc30 con 0x557a73fa9a90 2024-06-06T07:57:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 10 client.0 _ll_register_callbacks cb 0x7fe6b40f3390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55414/0 (socket says 172.21.15.87:55414) 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/928133152 learned_addr learned my addr 172.21.15.87:0/928133152 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c53a6640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6bffff640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 0x557a73fadfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/928133152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 msgr2=0x557a73fadfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 0x557a73fadfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 msgr2=0x557a73fae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 -- 172.21.15.87:0/928133152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a73faed40 con 0x557a73fa9170 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6bffff640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 0x557a73fadfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c4ba5640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b9ff0 tx=0x7fe6b40c4830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4d990a4fbfffc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe69ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4049040 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9583+0+0 (secure 0 0 0) 0x7fe6b40bb190 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac0047c0 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6b40bb6a0 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 10 client.17034 fetch_fsmap learned FSMap version 34 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.804+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a7402f0b0 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0047c0 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe6b40ed140 con 0x557a73fa9170 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 20 client.17034 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a7402f720 0x557a74031b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 4 client.17034 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:57:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a73f1ac20 con 0x557a7402f720 2024-06-06T07:57:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.805+0000 7fe6bffff640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a7402f720 0x557a74031b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.806+0000 7fe69ffff640 10 client.17034 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:45.806+0000 7fe6bffff640 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a7402f720 0x557a74031b40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f28f50ff740756d server_cookie=4aa94f9cb8af6beb in_seq=0 out_seq=0 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.050+0000 7fe69ffff640 1 -- 172.21.15.87:0/928133152 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a73f1ac20 con 0x557a7402f720 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.050+0000 7fe69ffff640 10 client.17034 handle_command_reply: tid=0 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.050+0000 7fe6c9bf0740 1 client.17034 shutdown 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 msgr2=0x557a73fbcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b9ff0 tx=0x7fe6b40c4830 comp rx=0 tx=0).stop 2024-06-06T07:57:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 shutdown_connections 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x557a7402f720 0x557a74031b40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe69ffff640 0 client.17034 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a73fb4350 0x557a73fae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a73fa9a90 0x557a73fadfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 --2- 172.21.15.87:0/928133152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557a73fa9170 0x557a73fbcaa0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 >> 172.21.15.87:0/928133152 conn(0x557a73fa5680 msgr2=0x557a73fab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 shutdown_connections 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 1 -- 172.21.15.87:0/928133152 wait complete. 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.051+0000 7fe6c9bf0740 20 client.17034 trim_cache size 0 max 16384 2024-06-06T07:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:57:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:57:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:57:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 msgr2=0x7fe6b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 0x7fe6b00a5260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005fd0 tx=0x7fe6b8014040 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 msgr2=0x7fe6c0199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009230 tx=0x7fe6b4004040 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 shutdown_connections 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe6b00a2da0 0x7fe6b00a5260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6c0106890 0x7fe6c01a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c0101290 0x7fe6c01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 --2- 172.21.15.87:0/3062610124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01008d0 0x7fe6c0199080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 >> 172.21.15.87:0/3062610124 conn(0x7fe6c0077a20 msgr2=0x7fe6c01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 shutdown_connections 2024-06-06T07:57:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:46.052+0000 7fe6c6e09640 1 -- 172.21.15.87:0/3062610124 wait complete. 2024-06-06T07:57:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:46 smithi181 ceph-mon[36790]: pgmap v302: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-06-06T07:57:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:46 smithi087 ceph-mon[32276]: pgmap v302: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-06-06T07:57:46.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 554744 6.68 MB/sec execute 350 sec latency 182.462 ms 2024-06-06T07:57:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:46 smithi195 ceph-mon[37040]: pgmap v302: 129 pgs: 129 active+clean; 93 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-06-06T07:57:46.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:57:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T07:57:46.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:57:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T07:57:46.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:57:46 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/1597912190 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d32af5a40 con 0x558d32ac9000 2024-06-06T07:57:46.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:57:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T07:57:47.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:47 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T07:57:47.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:47 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T07:57:47.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 556756 6.68 MB/sec execute 351 sec latency 53.774 ms 2024-06-06T07:57:47.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:47 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: pgmap v303: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/1306245437,v1:172.21.15.195:6837/1306245437] up:boot 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:57:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:48 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:57:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: pgmap v303: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-06-06T07:57:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:57:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/1306245437,v1:172.21.15.195:6837/1306245437] up:boot 2024-06-06T07:57:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:57:48.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:57:48.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:48 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:57:48.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 558774 6.70 MB/sec execute 352 sec latency 18.833 ms 2024-06-06T07:57:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: pgmap v303: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-06-06T07:57:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:57:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/1306245437,v1:172.21.15.195:6837/1306245437] up:boot 2024-06-06T07:57:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:57:48.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:57:48.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:48 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:57:49.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 561320 6.72 MB/sec execute 353 sec latency 65.934 ms 2024-06-06T07:57:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:50 smithi181 ceph-mon[36790]: pgmap v305: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:50 smithi087 ceph-mon[32276]: pgmap v305: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:50.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 562760 6.72 MB/sec execute 354 sec latency 40.466 ms 2024-06-06T07:57:50.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:50 smithi195 ceph-mon[37040]: pgmap v305: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:51.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 565191 6.74 MB/sec execute 355 sec latency 15.901 ms 2024-06-06T07:57:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:52 smithi181 ceph-mon[36790]: pgmap v306: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:52 smithi087 ceph-mon[32276]: pgmap v306: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:52.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 567130 6.74 MB/sec execute 356 sec latency 37.296 ms 2024-06-06T07:57:52.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:52 smithi195 ceph-mon[37040]: pgmap v306: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-06-06T07:57:53.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 569077 6.76 MB/sec execute 357 sec latency 18.178 ms 2024-06-06T07:57:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:54 smithi181 ceph-mon[36790]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:54 smithi087 ceph-mon[32276]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:54.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 570785 6.76 MB/sec execute 358 sec latency 34.188 ms 2024-06-06T07:57:54.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:54 smithi195 ceph-mon[37040]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:55.677 INFO:tasks.workunit.client.0.smithi087.stdout: 1 572443 6.78 MB/sec execute 359 sec latency 93.129 ms 2024-06-06T07:57:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:56 smithi181 ceph-mon[36790]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:56.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:56 smithi087 ceph-mon[32276]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:56.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 574470 6.78 MB/sec execute 360 sec latency 25.402 ms 2024-06-06T07:57:56.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:56 smithi195 ceph-mon[37040]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-06-06T07:57:57.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 576239 6.79 MB/sec execute 361 sec latency 22.666 ms 2024-06-06T07:57:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:57:58 smithi181 ceph-mon[36790]: pgmap v309: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-06-06T07:57:58.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:57:58 smithi087 ceph-mon[32276]: pgmap v309: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-06-06T07:57:58.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 578630 6.81 MB/sec execute 362 sec latency 17.250 ms 2024-06-06T07:57:58.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:57:58 smithi195 ceph-mon[37040]: pgmap v309: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.146+0000 7f16c0e5d640 1 -- 172.21.15.87:0/3828574469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc106890 msgr2=0x7f16bc106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.146+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/3828574469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc106890 0x7f16bc106c70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009930 tx=0x7f16ac02a420 comp rx=0 tx=0).stop 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 -- 172.21.15.87:0/3828574469 shutdown_connections 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/3828574469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 0x7f16bc10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/3828574469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc1008d0 0x7f16bc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/3828574469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc106890 0x7f16bc106c70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 -- 172.21.15.87:0/3828574469 >> 172.21.15.87:0/3828574469 conn(0x7f16bc077a20 msgr2=0x7f16bc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 -- 172.21.15.87:0/3828574469 shutdown_connections 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 -- 172.21.15.87:0/3828574469 wait complete. 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 Processor -- start 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.147+0000 7f16c0e5d640 1 -- start start 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 0x7f16bc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 0x7f16bc1a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc110ca0 con 0x7f16bc1008d0 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc110b20 con 0x7f16bc106890 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc110e20 con 0x7f16bc101290 2024-06-06T07:57:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59112/0 (socket says 172.21.15.87:59112) 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 -- 172.21.15.87:0/1851868398 learned_addr learned my addr 172.21.15.87:0/1851868398 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 0x7f16bc1a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 0x7f16bc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 -- 172.21.15.87:0/1851868398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 msgr2=0x7f16bc199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 0x7f16bc199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 -- 172.21.15.87:0/1851868398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 msgr2=0x7f16bc1a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 0x7f16bc1a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 -- 172.21.15.87:0/1851868398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1a45c0 con 0x7f16bc1008d0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 0x7f16bc1a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009230 tx=0x7f16ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16b37fe640 1 -- 172.21.15.87:0/1851868398 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac03f030 con 0x7f16bc1008d0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc1a4790 con 0x7f16bc1008d0 2024-06-06T07:57:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16b37fe640 1 -- 172.21.15.87:0/1851868398 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac004440 con 0x7f16bc1008d0 2024-06-06T07:57:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.148+0000 7f16b37fe640 1 -- 172.21.15.87:0/1851868398 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac035070 con 0x7f16bc1008d0 2024-06-06T07:57:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.149+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a4ba0 con 0x7f16bc1008d0 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.151+0000 7f16b37fe640 1 -- 172.21.15.87:0/1851868398 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f16ac042030 con 0x7f16bc1008d0 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.151+0000 7f16c3b6a740 1 Processor -- start 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 -- start start 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55719f42d4f0 con 0x55719f4c9a90 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55719f3cb870 con 0x55719f4c9170 2024-06-06T07:57:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55719f4294b0 con 0x55719f4d4350 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16b37fe640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 0x7f16a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:42258/0 (socket says 172.21.15.87:42258) 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59114/0 (socket says 172.21.15.87:59114) 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 -- 172.21.15.87:0/758419123 learned_addr learned my addr 172.21.15.87:0/758419123 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 0x7f16a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16b3fff640 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16b37fe640 1 -- 172.21.15.87:0/1851868398 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9495+0+0 (secure 0 0 0) 0x7f16ac0efc20 con 0x7f16bc1008d0 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 -- 172.21.15.87:0/758419123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 msgr2=0x55719f4d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 -- 172.21.15.87:0/758419123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 msgr2=0x55719f4d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.152+0000 7f16bad9d640 1 -- 172.21.15.87:0/758419123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719f3e79b0 con 0x55719f4c9170 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.153+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 0x7f16a80a5260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16b40099e0 tx=0x7f16b401e000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.153+0000 7f16bad9d640 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16b400f5c0 tx=0x7f16b400fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5429ec31c7fd3962 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.153+0000 7f16b0ff9640 1 -- 172.21.15.87:0/758419123 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b402b070 con 0x55719f4c9170 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.153+0000 7f16b0ff9640 1 -- 172.21.15.87:0/758419123 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b40268f0 con 0x55719f4c9170 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.153+0000 7f16b0ff9640 1 -- 172.21.15.87:0/758419123 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4022070 con 0x55719f4c9170 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- 172.21.15.87:0/758419123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 msgr2=0x55719f4c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16b400f5c0 tx=0x7f16b400fa80 comp rx=0 tx=0).stop 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- 172.21.15.87:0/758419123 shutdown_connections 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/758419123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4c9550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- 172.21.15.87:0/758419123 >> 172.21.15.87:0/758419123 conn(0x55719f4c5680 msgr2=0x55719f4c5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- 172.21.15.87:0/758419123 shutdown_connections 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- 172.21.15.87:0/758419123 wait complete. 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 -- start start 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 Processor -- start 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.154+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55719f4d6f20 con 0x55719f4c9a90 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55719f3b6ae0 con 0x55719f4c9170 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55719f4d70a0 con 0x55719f4d4350 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 10 client.0 _ll_register_callbacks cb 0x55719f3ef9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59118/0 (socket says 172.21.15.87:59118) 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 -- 172.21.15.87:0/1034294692 learned_addr learned my addr 172.21.15.87:0/1034294692 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 -- 172.21.15.87:0/1034294692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 msgr2=0x55719f4d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 -- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 msgr2=0x55719f4d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 -- 172.21.15.87:0/1034294692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719f141210 con 0x55719f4c9a90 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f16a4013710 tx=0x7f16a4013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aea21f35725c25 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4020060 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9495+0+0 (secure 0 0 0) 0x7f16a4011dd0 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b40268f0 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16a4027050 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 10 client.17049 fetch_fsmap learned FSMap version 36 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.155+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55719f20d620 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b40268f0 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f16a401a030 con 0x55719f4c9a90 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 10 client.17049 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 20 client.17049 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f4e4550 0x55719f4e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 4 client.17049 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55719f440f10 con 0x55719f4e4550 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.156+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f4e4550 0x55719f4e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16a1ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f4e4550 0x55719f4e6960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d548ac0d57100fab server_cookie=9dba99ee94b8396c in_seq=0 out_seq=0 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1034294692 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55719f440f10 con 0x55719f4e4550 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16a1ffb640 10 client.17049 handle_command_reply: tid=0 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 client.17049 shutdown 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 msgr2=0x55719f4d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f16a4013710 tx=0x7f16a4013bd0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 shutdown_connections 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f4e4550 0x55719f4e6960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4d1850 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1034294692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16a1ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.157+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 >> 172.21.15.87:0/1034294692 conn(0x55719f4c5680 msgr2=0x55719f4cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.158+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 shutdown_connections 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.158+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1034294692 wait complete. 2024-06-06T07:57:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.158+0000 7f16c3b6a740 20 client.17049 trim_cache size 0 max 16384 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 Processor -- start 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 -- start start 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 0x55719f4d3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4d4350 0x55719f4ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b16b0 con 0x55719f4c9a90 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1aef80 con 0x55719f4d4350 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1acdd0 con 0x55719f4c9170 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59122/0 (socket says 172.21.15.87:59122) 2024-06-06T07:57:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 -- 172.21.15.87:0/1415605817 learned_addr learned my addr 172.21.15.87:0/1415605817 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4d4350 0x55719f4ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 0x55719f4d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 -- 172.21.15.87:0/1415605817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 msgr2=0x55719f4d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 0x55719f4d3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 -- 172.21.15.87:0/1415605817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4d4350 msgr2=0x55719f4ec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4d4350 0x55719f4ec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 -- 172.21.15.87:0/1415605817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719f4e30f0 con 0x55719f4c9a90 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 0x55719f4d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009830 tx=0x7f16a4009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6775c4ef2bd2ec74 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.173+0000 7f16a3fff640 1 -- 172.21.15.87:0/1415605817 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a402f020 con 0x55719f4c9a90 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16a3fff640 1 -- 172.21.15.87:0/1415605817 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4005520 con 0x55719f4c9a90 2024-06-06T07:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16a3fff640 1 -- 172.21.15.87:0/1415605817 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4004620 con 0x55719f4c9a90 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1415605817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 msgr2=0x55719f4ec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009830 tx=0x7f16a4009d40 comp rx=0 tx=0).stop 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1415605817 shutdown_connections 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4d4350 0x55719f4ec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4ec270 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1415605817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4c9170 0x55719f4d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1415605817 >> 172.21.15.87:0/1415605817 conn(0x55719f4c5680 msgr2=0x55719f4d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1415605817 shutdown_connections 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1415605817 wait complete. 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.174+0000 7f16c3b6a740 1 -- start start 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 Processor -- start 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1ad440 con 0x55719f4c9a90 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1ae8b0 con 0x55719f4c9170 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1adb10 con 0x55719f4d4350 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 10 client.0 _ll_register_callbacks cb 0x7f16a4009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59136/0 (socket says 172.21.15.87:59136) 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 -- 172.21.15.87:0/1931436110 learned_addr learned my addr 172.21.15.87:0/1931436110 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16bad9d640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 -- 172.21.15.87:0/1931436110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 msgr2=0x55719f4d7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 -- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 msgr2=0x55719f4dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 -- 172.21.15.87:0/1931436110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719f4d8220 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16b3fff640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f16a4013c10 tx=0x7f16a4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c07d11ada4403ae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4038070 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9495+0+0 (secure 0 0 0) 0x7f16a40050d0 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.175+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a4005520 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16a4041050 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 10 client.17055 fetch_fsmap learned FSMap version 36 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55719f54ed40 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4005520 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f16a403b030 con 0x55719f4c9a90 2024-06-06T07:57:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 10 client.17055 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 20 client.17055 populate_metadata read hostname 'smithi087' 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f54f350 0x55719f551770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 4 client.17055 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55719f43ac20 con 0x55719f54f350 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f54f350 0x55719f551770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16ba59c640 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f54f350 0x55719f551770 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4070810b328f36e0 server_cookie=c62335c54ab4b5ac in_seq=0 out_seq=0 2024-06-06T07:57:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.176+0000 7f16a1ffb640 10 client.17055 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16a1ffb640 1 -- 172.21.15.87:0/1931436110 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55719f43ac20 con 0x55719f54f350 2024-06-06T07:57:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16a1ffb640 10 client.17055 handle_command_reply: tid=0 2024-06-06T07:57:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 client.17055 shutdown 2024-06-06T07:57:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 msgr2=0x55719f4dcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f16a4013c10 tx=0x7f16a4009cb0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 shutdown_connections 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55719f54f350 0x55719f551770 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55719f4d4350 0x55719f4d7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55719f4c9a90 0x55719f4dcb20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16a1ffb640 0 client.17055 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 --2- 172.21.15.87:0/1931436110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55719f4c9170 0x55719f4dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 >> 172.21.15.87:0/1931436110 conn(0x55719f4c5680 msgr2=0x55719f4d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 shutdown_connections 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 1 -- 172.21.15.87:0/1931436110 wait complete. 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.483+0000 7f16c3b6a740 20 client.17055 trim_cache size 0 max 16384 2024-06-06T07:57:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:57:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:57:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:57:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 msgr2=0x7f16a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 0x7f16a80a5260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16b40099e0 tx=0x7f16b401e000 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 msgr2=0x7f16bc199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009230 tx=0x7f16ac004040 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 shutdown_connections 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f16a80a2da0 0x7f16a80a5260 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc106890 0x7f16bc1a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16bc101290 0x7f16bc199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 --2- 172.21.15.87:0/1851868398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc1008d0 0x7f16bc199000 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 >> 172.21.15.87:0/1851868398 conn(0x7f16bc077a20 msgr2=0x7f16bc1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 shutdown_connections 2024-06-06T07:57:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:57:59.484+0000 7f16c0e5d640 1 -- 172.21.15.87:0/1851868398 wait complete. 2024-06-06T07:57:59.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 579925 6.81 MB/sec execute 363 sec latency 75.631 ms 2024-06-06T07:58:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:00 smithi181 ceph-mon[36790]: pgmap v310: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-06-06T07:58:00.561 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:00 smithi195 ceph-mon[37040]: pgmap v310: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-06-06T07:58:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:00 smithi087 ceph-mon[32276]: pgmap v310: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-06-06T07:58:00.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 581462 6.81 MB/sec execute 364 sec latency 128.658 ms 2024-06-06T07:58:01.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 582715 6.82 MB/sec execute 365 sec latency 66.949 ms 2024-06-06T07:58:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:02 smithi181 ceph-mon[36790]: pgmap v311: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-06-06T07:58:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:02 smithi087 ceph-mon[32276]: pgmap v311: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-06-06T07:58:02.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 583915 6.82 MB/sec execute 366 sec latency 61.090 ms 2024-06-06T07:58:02.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:02 smithi195 ceph-mon[37040]: pgmap v311: 129 pgs: 129 active+clean; 94 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-06-06T07:58:03.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 586161 6.83 MB/sec execute 367 sec latency 21.226 ms 2024-06-06T07:58:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:04 smithi181 ceph-mon[36790]: pgmap v312: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-06T07:58:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:04 smithi087 ceph-mon[32276]: pgmap v312: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-06T07:58:04.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 587601 6.83 MB/sec execute 368 sec latency 15.808 ms 2024-06-06T07:58:04.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:04 smithi195 ceph-mon[37040]: pgmap v312: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-06T07:58:05.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 589691 6.85 MB/sec execute 369 sec latency 57.049 ms 2024-06-06T07:58:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:06 smithi181 ceph-mon[36790]: pgmap v313: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-06T07:58:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:06 smithi087 ceph-mon[32276]: pgmap v313: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-06T07:58:06.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 590600 6.83 MB/sec execute 370 sec latency 42.205 ms 2024-06-06T07:58:06.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:06 smithi195 ceph-mon[37040]: pgmap v313: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-06-06T07:58:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:07 smithi087 ceph-mon[32276]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:58:07.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 591189 6.83 MB/sec execute 371 sec latency 59.744 ms 2024-06-06T07:58:07.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:07 smithi195 ceph-mon[37040]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:58:07.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:07 smithi181 ceph-mon[36790]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:58:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:08 smithi087 ceph-mon[32276]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 90 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-06-06T07:58:08.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 592389 6.82 MB/sec execute 372 sec latency 49.671 ms 2024-06-06T07:58:08.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:08 smithi195 ceph-mon[37040]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 90 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-06-06T07:58:08.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:08 smithi181 ceph-mon[36790]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 90 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-06-06T07:58:09.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 594067 6.83 MB/sec execute 373 sec latency 24.554 ms 2024-06-06T07:58:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:10 smithi087 ceph-mon[32276]: pgmap v316: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:10.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 595582 6.83 MB/sec execute 374 sec latency 39.747 ms 2024-06-06T07:58:10.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:10 smithi195 ceph-mon[37040]: pgmap v316: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:10.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:10 smithi181 ceph-mon[36790]: pgmap v316: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:11.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 597434 6.84 MB/sec execute 375 sec latency 18.468 ms 2024-06-06T07:58:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 -- 172.21.15.87:0/1424859117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65281019b0 msgr2=0x7f6528101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 --2- 172.21.15.87:0/1424859117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65281019b0 0x7f6528101e30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6518009930 tx=0x7f651802a420 comp rx=0 tx=0).stop 2024-06-06T07:58:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 -- 172.21.15.87:0/1424859117 shutdown_connections 2024-06-06T07:58:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 --2- 172.21.15.87:0/1424859117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6528102370 0x7f652810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 --2- 172.21.15.87:0/1424859117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65281019b0 0x7f6528101e30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 --2- 172.21.15.87:0/1424859117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281079c0 0x7f6528107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 -- 172.21.15.87:0/1424859117 >> 172.21.15.87:0/1424859117 conn(0x7f65280fd460 msgr2=0x7f65280ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 -- 172.21.15.87:0/1424859117 shutdown_connections 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.594+0000 7f652fe5d640 1 -- 172.21.15.87:0/1424859117 wait complete. 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 Processor -- start 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 -- start start 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 0x7f6528198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111dd0 con 0x7f6528102370 2024-06-06T07:58:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111c50 con 0x7f65281079c0 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652fe5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111f50 con 0x7f65281019b0 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652d3f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652e3fa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652dbf9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 0x7f6528198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652d3f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36904/0 (socket says 172.21.15.87:36904) 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652e3fa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53232/0 (socket says 172.21.15.87:53232) 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.595+0000 7f652d3f8640 1 -- 172.21.15.87:0/4019352807 learned_addr learned my addr 172.21.15.87:0/4019352807 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 -- 172.21.15.87:0/4019352807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 msgr2=0x7f6528198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 0x7f6528198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 -- 172.21.15.87:0/4019352807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 msgr2=0x7f652819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 -- 172.21.15.87:0/4019352807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652819d200 con 0x7f6528102370 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652dbf9640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 0x7f6528198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652e3fa640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6518009230 tx=0x7f6518004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f651effd640 1 -- 172.21.15.87:0/4019352807 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651803f030 con 0x7f6528102370 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f651effd640 1 -- 172.21.15.87:0/4019352807 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6518004440 con 0x7f6528102370 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652819d490 con 0x7f6528102370 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f651effd640 1 -- 172.21.15.87:0/4019352807 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518035070 con 0x7f6528102370 2024-06-06T07:58:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.596+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281a4d80 con 0x7f6528102370 2024-06-06T07:58:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.598+0000 7f651effd640 1 -- 172.21.15.87:0/4019352807 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6518042030 con 0x7f6528102370 2024-06-06T07:58:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 Processor -- start 2024-06-06T07:58:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 -- start start 2024-06-06T07:58:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad70170 0x55863ad70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad7b350 0x55863ad7d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55863acd44f0 con 0x55863ad7b350 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55863ac72870 con 0x55863ad70a90 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f6532b6e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55863acd04b0 con 0x55863ad70170 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f652dbf9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.599+0000 7f652d3f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad7b350 0x55863ad7d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53238/0 (socket says 172.21.15.87:53238) 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652e3fa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad70170 0x55863ad70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 -- 172.21.15.87:0/3910202249 learned_addr learned my addr 172.21.15.87:0/3910202249 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f651effd640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 0x7f65140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 -- 172.21.15.87:0/3910202249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad70170 msgr2=0x55863ad70550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad70170 0x55863ad70550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 -- 172.21.15.87:0/3910202249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad7b350 msgr2=0x55863ad7d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad7b350 0x55863ad7d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 -- 172.21.15.87:0/3910202249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863ac8e9b0 con 0x55863ad70a90 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652e3fa640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 0x7f65140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f651effd640 1 -- 172.21.15.87:0/4019352807 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9671+0+0 (secure 0 0 0) 0x7f65180efbb0 con 0x7f6528102370 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f652dbf9640 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65240114c0 tx=0x7f6524011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9a66a35b2aca96 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f6513fff640 1 -- 172.21.15.87:0/3910202249 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524004020 con 0x55863ad70a90 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f6513fff640 1 -- 172.21.15.87:0/3910202249 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240041c0 con 0x55863ad70a90 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.600+0000 7f6513fff640 1 -- 172.21.15.87:0/3910202249 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652400f9c0 con 0x55863ad70a90 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 -- 172.21.15.87:0/3910202249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 msgr2=0x55863ad7ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f652e3fa640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 0x7f65140a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fd0 tx=0x7f6520004a40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65240114c0 tx=0x7f6524011980 comp rx=0 tx=0).stop 2024-06-06T07:58:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 -- 172.21.15.87:0/3910202249 shutdown_connections 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad7b350 0x55863ad7d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad7ae10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3910202249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad70170 0x55863ad70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 -- 172.21.15.87:0/3910202249 >> 172.21.15.87:0/3910202249 conn(0x55863ad6c680 msgr2=0x55863ad6ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 -- 172.21.15.87:0/3910202249 shutdown_connections 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.601+0000 7f6532b6e740 1 -- 172.21.15.87:0/3910202249 wait complete. 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 -- start start 2024-06-06T07:58:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 Processor -- start 2024-06-06T07:58:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad78320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55863ad7df80 con 0x55863ad70a90 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55863ac5dae0 con 0x55863ad70170 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55863ad7e100 con 0x55863ad7b350 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f6532b6e740 10 client.0 _ll_register_callbacks cb 0x55863ac969b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f652d3f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f652d3f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36936/0 (socket says 172.21.15.87:36936) 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f652d3f8640 1 -- 172.21.15.87:0/4062459250 learned_addr learned my addr 172.21.15.87:0/4062459250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f652dbf9640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad78320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.602+0000 7f652e3fa640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 -- 172.21.15.87:0/4062459250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 msgr2=0x55863ad78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 -- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 msgr2=0x55863ad78320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad78320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 -- 172.21.15.87:0/4062459250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863a9e8210 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f652d3f8640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f65180ed480 tx=0x7f6518005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f5f9c383eb8fea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f65117fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518049070 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9671+0+0 (secure 0 0 0) 0x7f65180b8370 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65240041c0 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f65180d1070 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f6532b6e740 10 client.17067 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55863aab4620 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.603+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240041c0 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f65180c4430 con 0x55863ad70a90 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 10 client.17067 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 20 client.17067 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863ad93d50 0x55863ad961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 4 client.17067 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863ace7f10 con 0x55863ad93d50 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f652dbf9640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863ad93d50 0x55863ad961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f652dbf9640 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863ad93d50 0x55863ad961d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abc62cfc1056d9d server_cookie=e5190a718998d671 in_seq=0 out_seq=0 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.604+0000 7f65117fa640 10 client.17067 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.606+0000 7f65117fa640 1 -- 172.21.15.87:0/4062459250 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55863ace7f10 con 0x55863ad93d50 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.606+0000 7f65117fa640 10 client.17067 handle_command_reply: tid=0 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.606+0000 7f6532b6e740 1 client.17067 shutdown 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 msgr2=0x55863ad78860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f65180ed480 tx=0x7f6518005c90 comp rx=0 tx=0).stop 2024-06-06T07:58:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 shutdown_connections 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863ad93d50 0x55863ad961d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad78860 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 --2- 172.21.15.87:0/4062459250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad78320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f65117fa640 0 client.17067 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 >> 172.21.15.87:0/4062459250 conn(0x55863ad6c680 msgr2=0x55863ad723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 shutdown_connections 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 1 -- 172.21.15.87:0/4062459250 wait complete. 2024-06-06T07:58:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.607+0000 7f6532b6e740 20 client.17067 trim_cache size 0 max 16384 2024-06-06T07:58:12.612 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:12 smithi087 ceph-mon[32276]: pgmap v317: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 Processor -- start 2024-06-06T07:58:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 -- start start 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad9bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad9c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65281af600 con 0x55863ad70a90 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65281aced0 con 0x55863ad70170 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.623+0000 7f6532b6e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65281aad40 con 0x55863ad7b350 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:60074/0 (socket says 172.21.15.87:60074) 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 -- 172.21.15.87:0/3029579498 learned_addr learned my addr 172.21.15.87:0/3029579498 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652d3f8640 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad9c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652dbf9640 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad9bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 -- 172.21.15.87:0/3029579498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 msgr2=0x55863ad9bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad9bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 -- 172.21.15.87:0/3029579498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 msgr2=0x55863ad9c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad9c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 -- 172.21.15.87:0/3029579498 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863ad73120 con 0x55863ad7b350 2024-06-06T07:58:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f652e3fa640 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6520011a40 tx=0x7f6520018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e4c779fc38f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f65137fe640 1 -- 172.21.15.87:0/3029579498 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652001da20 con 0x55863ad7b350 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f65137fe640 1 -- 172.21.15.87:0/3029579498 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652002e070 con 0x55863ad7b350 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.624+0000 7f65137fe640 1 -- 172.21.15.87:0/3029579498 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652000f620 con 0x55863ad7b350 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 -- 172.21.15.87:0/3029579498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 msgr2=0x55863ad75580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6520011a40 tx=0x7f6520018000 comp rx=0 tx=0).stop 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 -- 172.21.15.87:0/3029579498 shutdown_connections 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad75580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70a90 0x55863ad9c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 --2- 172.21.15.87:0/3029579498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70170 0x55863ad9bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 -- 172.21.15.87:0/3029579498 >> 172.21.15.87:0/3029579498 conn(0x55863ad6c680 msgr2=0x55863ad73630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 -- 172.21.15.87:0/3029579498 shutdown_connections 2024-06-06T07:58:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.625+0000 7f6532b6e740 1 -- 172.21.15.87:0/3029579498 wait complete. 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 -- start start 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 Processor -- start 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65281ab390 con 0x55863ad70170 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65281ac800 con 0x55863ad70a90 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65281aba60 con 0x55863ad7b350 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f6532b6e740 10 client.0 _ll_register_callbacks cb 0x7f65180f33f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f652dbf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f652dbf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36964/0 (socket says 172.21.15.87:36964) 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f652dbf9640 1 -- 172.21.15.87:0/2937064409 learned_addr learned my addr 172.21.15.87:0/2937064409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f652e3fa640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.626+0000 7f652d3f8640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 -- 172.21.15.87:0/2937064409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 msgr2=0x55863ad87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad87a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 -- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 msgr2=0x55863ad874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad874d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 -- 172.21.15.87:0/2937064409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863ad88050 con 0x55863ad70170 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f652dbf9640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796367dd74e42349 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652401d040 con 0x55863ad70170 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9671+0+0 (secure 0 0 0) 0x7f6524011020 con 0x55863ad70170 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652002e070 con 0x55863ad70170 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6524039050 con 0x55863ad70170 2024-06-06T07:58:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 10 client.17073 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55863adfe930 con 0x55863ad70170 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652002e070 con 0x55863ad70170 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6524003a00 con 0x55863ad70170 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 10 client.17073 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 20 client.17073 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863adfef10 0x55863ae01330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 4 client.17073 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.627+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55863ace1c20 con 0x55863adfef10 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.628+0000 7f652d3f8640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863adfef10 0x55863ae01330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.628+0000 7f65117fa640 10 client.17073 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.628+0000 7f652d3f8640 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863adfef10 0x55863ae01330 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270feb5cf00b4b40 server_cookie=cd6e59ddd9e5f762 in_seq=0 out_seq=0 2024-06-06T07:58:12.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 599472 6.85 MB/sec execute 376 sec latency 21.142 ms 2024-06-06T07:58:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:12 smithi195 ceph-mon[37040]: pgmap v317: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:12 smithi181 ceph-mon[36790]: pgmap v317: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 91 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-06-06T07:58:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f65117fa640 1 -- 172.21.15.87:0/2937064409 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55863ace1c20 con 0x55863adfef10 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f65117fa640 10 client.17073 handle_command_reply: tid=0 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 client.17073 shutdown 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 msgr2=0x55863ad8c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524005da0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 shutdown_connections 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55863adfef10 0x55863ae01330 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ad7b350 0x55863ad87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f65117fa640 0 client.17073 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55863ad70a90 0x55863ad874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 --2- 172.21.15.87:0/2937064409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55863ad70170 0x55863ad8c160 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.847+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 >> 172.21.15.87:0/2937064409 conn(0x55863ad6c680 msgr2=0x55863ad71b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.848+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 shutdown_connections 2024-06-06T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.848+0000 7f6532b6e740 1 -- 172.21.15.87:0/2937064409 wait complete. 2024-06-06T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.848+0000 7f6532b6e740 20 client.17073 trim_cache size 0 max 16384 2024-06-06T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.848+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 msgr2=0x7f65140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.848+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 0x7f65140a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fd0 tx=0x7f6520004a40 comp rx=0 tx=0).stop 2024-06-06T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 msgr2=0x7f6528198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6518009230 tx=0x7f6518004040 comp rx=0 tx=0).stop 2024-06-06T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 shutdown_connections 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f65140a2da0 0x7f65140a5260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65281079c0 0x7f652819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6528102370 0x7f6528198750 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 --2- 172.21.15.87:0/4019352807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65281019b0 0x7f6528198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 >> 172.21.15.87:0/4019352807 conn(0x7f65280fd460 msgr2=0x7f65280fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 shutdown_connections 2024-06-06T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:12.849+0000 7f652fe5d640 1 -- 172.21.15.87:0/4019352807 wait complete. 2024-06-06T07:58:13.501 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T07:58:13.501 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-06T07:58:13.502 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:58:13.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 600987 6.86 MB/sec execute 377 sec latency 136.360 ms 2024-06-06T07:58:13.771 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:14 smithi087 ceph-mon[32276]: pgmap v318: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:14.678 INFO:tasks.workunit.client.0.smithi087.stdout: 1 603267 6.87 MB/sec execute 378 sec latency 14.509 ms 2024-06-06T07:58:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:14 smithi195 ceph-mon[37040]: pgmap v318: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:14 smithi181 ceph-mon[36790]: pgmap v318: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:15.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 604470 6.87 MB/sec execute 379 sec latency 95.074 ms 2024-06-06T07:58:15.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:15 smithi087 ceph-mon[32276]: pgmap v319: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:15.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:15 smithi087 ceph-mon[32276]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.870+0000 7f614c98d700 1 -- 172.21.15.87:0/1315454410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 msgr2=0x7f614810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.870+0000 7f614c98d700 1 --2- 172.21.15.87:0/1315454410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f614810f330 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f614000b0a0 tx=0x7f614000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 -- 172.21.15.87:0/1315454410 shutdown_connections 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 --2- 172.21.15.87:0/1315454410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f614810f330 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 --2- 172.21.15.87:0/1315454410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61481084f0 0x7f61481088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 --2- 172.21.15.87:0/1315454410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6148102550 0x7f61481029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 -- 172.21.15.87:0/1315454410 >> 172.21.15.87:0/1315454410 conn(0x7f6148079650 msgr2=0x7f6148079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 -- 172.21.15.87:0/1315454410 shutdown_connections 2024-06-06T07:58:15.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.871+0000 7f614c98d700 1 -- 172.21.15.87:0/1315454410 wait complete. 2024-06-06T07:58:15.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 Processor -- start 2024-06-06T07:58:15.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 -- start start 2024-06-06T07:58:15.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 0x7f6148074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:15.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:15.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61481084f0 0x7f6148075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112770 con 0x7f6148102f10 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61481125f0 con 0x7f61481084f0 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614c98d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61481128f0 con 0x7f6148102550 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f6145d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f614659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 0x7f6148074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f6145d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36984/0 (socket says 172.21.15.87:36984) 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f6145d9b700 1 -- 172.21.15.87:0/1074771425 learned_addr learned my addr 172.21.15.87:0/1074771425 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.872+0000 7f6146d9d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61481084f0 0x7f6148075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 -- 172.21.15.87:0/1074771425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61481084f0 msgr2=0x7f6148075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61481084f0 0x7f6148075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 -- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 msgr2=0x7f6148074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:15.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 -- 172.21.15.87:0/1074771425 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61481a7170 con 0x7f6148102550 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f6145d9b700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614659c700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 0x7f6148074790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f613800e9f0 tx=0x7f613800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138004220 con 0x7f6148102550 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61481a7460 con 0x7f6148102550 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6138004d10 con 0x7f6148102550 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61481a79a0 con 0x7f6148102550 2024-06-06T07:58:15.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.873+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138005020 con 0x7f6148102550 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.874+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6148069180 con 0x7f6148102550 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.875+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6138042b60 con 0x7f6148102550 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.876+0000 7f61377fe700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 0x7f61300a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.876+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f61380cf600 con 0x7f6148102550 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.876+0000 7f6145d9b700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 0x7f61300a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:15.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.876+0000 7f6145d9b700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 0x7f61300a52d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6148079bc0 tx=0x7f613c009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:15.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:15.879+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f613801f3a0 con 0x7f6148102550 2024-06-06T07:58:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:15 smithi181 ceph-mon[36790]: pgmap v319: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:15 smithi181 ceph-mon[36790]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:58:16.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.141+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f614802dca0 con 0x7f6148102550 2024-06-06T07:58:16.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.142+0000 7f61377fe700 1 -- 172.21.15.87:0/1074771425 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11877 (secure 0 0 0) 0x7f613809be00 con 0x7f6148102550 2024-06-06T07:58:16.142 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:58:16.142 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:57:47.279422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26299":{"gid":26299,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1306245437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1306245437},{"type":"v1","addr":"172.21.15.195:6837","nonce":1306245437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:58:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.144+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 msgr2=0x7f61300a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.144+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 0x7f61300a52d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6148079bc0 tx=0x7f613c009260 comp rx=0 tx=0).stop 2024-06-06T07:58:16.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.144+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 msgr2=0x7f6148074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.144+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 0x7f6148074790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f613800e9f0 tx=0x7f613800ed00 comp rx=0 tx=0).stop 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 shutdown_connections 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61481084f0 0x7f6148075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61300a2e10 0x7f61300a52d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6148102550 0x7f6148074790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 --2- 172.21.15.87:0/1074771425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148102f10 0x7f6148074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 >> 172.21.15.87:0/1074771425 conn(0x7f6148079650 msgr2=0x7f61481005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 shutdown_connections 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:16.145+0000 7f614c98d700 1 -- 172.21.15.87:0/1074771425 wait complete. 2024-06-06T07:58:16.145 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 36 2024-06-06T07:58:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:15 smithi195 ceph-mon[37040]: pgmap v319: 129 pgs: 129 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-06-06T07:58:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:15 smithi195 ceph-mon[37040]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:58:16.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 606366 6.87 MB/sec execute 380 sec latency 99.538 ms 2024-06-06T07:58:16.856 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-06T07:58:16.856 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-06T07:58:17.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:16 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1074771425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:17.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:16 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1074771425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:17.112 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:17.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:16 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1074771425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:17.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 608228 6.89 MB/sec execute 381 sec latency 53.266 ms 2024-06-06T07:58:18.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:17 smithi087 ceph-mon[32276]: pgmap v321: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-06-06T07:58:18.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:17 smithi181 ceph-mon[36790]: pgmap v321: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-06-06T07:58:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:17 smithi195 ceph-mon[37040]: pgmap v321: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-06-06T07:58:18.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 610567 6.90 MB/sec execute 382 sec latency 29.085 ms 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 -- 172.21.15.87:0/3460707710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c100e50 msgr2=0x7f186c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 --2- 172.21.15.87:0/3460707710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c100e50 0x7f186c1012d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f185c009990 tx=0x7f185c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 -- 172.21.15.87:0/3460707710 shutdown_connections 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 --2- 172.21.15.87:0/3460707710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 --2- 172.21.15.87:0/3460707710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c101810 0x7f186c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 --2- 172.21.15.87:0/3460707710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c100e50 0x7f186c1012d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 -- 172.21.15.87:0/3460707710 >> 172.21.15.87:0/3460707710 conn(0x7f186c0766b0 msgr2=0x7f186c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 -- 172.21.15.87:0/3460707710 shutdown_connections 2024-06-06T07:58:19.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.635+0000 7f187424e700 1 -- 172.21.15.87:0/3460707710 wait complete. 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 Processor -- start 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 -- start start 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 0x7f186c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 0x7f186c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1127d0 con 0x7f186c101810 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112650 con 0x7f186c107570 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f187424e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112950 con 0x7f186c100e50 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 0x7f186c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f1871fea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 0x7f186c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f18727eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 0x7f186c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46428/0 (socket says 172.21.15.87:46428) 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f1871fea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 0x7f186c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52628/0 (socket says 172.21.15.87:52628) 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f18717e9700 1 -- 172.21.15.87:0/4135467392 learned_addr learned my addr 172.21.15.87:0/4135467392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.636+0000 7f18727eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:47650/0 (socket says 172.21.15.87:47650) 2024-06-06T07:58:19.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 -- 172.21.15.87:0/4135467392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 msgr2=0x7f186c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 0x7f186c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 -- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 msgr2=0x7f186c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 0x7f186c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 -- 172.21.15.87:0/4135467392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1990f0 con 0x7f186c107570 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f18727eb700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f186400efc0 tx=0x7f186400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f1862ffd700 1 -- 172.21.15.87:0/4135467392 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640092e0 con 0x7f186c107570 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c199380 con 0x7f186c107570 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a5650 con 0x7f186c107570 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f1862ffd700 1 -- 172.21.15.87:0/4135467392 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1864004bb0 con 0x7f186c107570 2024-06-06T07:58:19.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.637+0000 7f1862ffd700 1 -- 172.21.15.87:0/4135467392 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864004280 con 0x7f186c107570 2024-06-06T07:58:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.639+0000 7f1862ffd700 1 -- 172.21.15.87:0/4135467392 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f186401b030 con 0x7f186c107570 2024-06-06T07:58:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.640+0000 7f1862ffd700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 0x7f18580a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.640+0000 7f1871fea700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 0x7f18580a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.640+0000 7f1862ffd700 1 -- 172.21.15.87:0/4135467392 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f18640cf330 con 0x7f186c107570 2024-06-06T07:58:19.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.640+0000 7f1871fea700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 0x7f18580a5370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f186c107d50 tx=0x7f1868009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 Processor -- start 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 -- start start 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 0x56472e34f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e349cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56472e2969a0 con 0x56472e34fa00 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56472e23e8b0 con 0x56472e349870 2024-06-06T07:58:19.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.641+0000 7f187ead1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56472e292960 con 0x56472e34f0e0 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f1871fea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 0x56472e34f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46444/0 (socket says 172.21.15.87:46444) 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 -- 172.21.15.87:0/1176733850 learned_addr learned my addr 172.21.15.87:0/1176733850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18727eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e349cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 -- 172.21.15.87:0/1176733850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 msgr2=0x56472e34f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 0x56472e34f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 -- 172.21.15.87:0/1176733850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 msgr2=0x56472e349cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e349cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 -- 172.21.15.87:0/1176733850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472e2350e0 con 0x56472e34fa00 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f1871fea700 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 0x56472e34f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18727eb700 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e349cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f18717e9700 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f185c009720 tx=0x7f185c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9e74e3459fab4d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f1857fff700 1 -- 172.21.15.87:0/1176733850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c027070 con 0x56472e34fa00 2024-06-06T07:58:19.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f1857fff700 1 -- 172.21.15.87:0/1176733850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c005550 con 0x56472e34fa00 2024-06-06T07:58:19.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.642+0000 7f1857fff700 1 -- 172.21.15.87:0/1176733850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c003cb0 con 0x56472e34fa00 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 -- 172.21.15.87:0/1176733850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 msgr2=0x56472e349330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f185c009720 tx=0x7f185c005c80 comp rx=0 tx=0).stop 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 -- 172.21.15.87:0/1176733850 shutdown_connections 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e349cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34f0e0 0x56472e34f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1176733850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e349330 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 -- 172.21.15.87:0/1176733850 >> 172.21.15.87:0/1176733850 conn(0x56472e26fe50 msgr2=0x56472e3401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 -- 172.21.15.87:0/1176733850 shutdown_connections 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.643+0000 7f187ead1b80 1 -- 172.21.15.87:0/1176733850 wait complete. 2024-06-06T07:58:19.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 -- start start 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 Processor -- start 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e349870 0x56472e3556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e3529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56472e356410 con 0x56472e34fa00 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56472e236d80 con 0x56472e34f0e0 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56472e356590 con 0x56472e349870 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f187ead1b80 10 client.0 _ll_register_callbacks cb 0x56472e230ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f18727eb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f18717e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e3529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f18727eb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46454/0 (socket says 172.21.15.87:46454) 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f18727eb700 1 -- 172.21.15.87:0/2863497581 learned_addr learned my addr 172.21.15.87:0/2863497581 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.644+0000 7f1871fea700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e349870 0x56472e3556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 -- 172.21.15.87:0/2863497581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e349870 msgr2=0x56472e3556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e349870 0x56472e3556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 -- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 msgr2=0x56472e3529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e3529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 -- 172.21.15.87:0/2863497581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472e1e5a60 con 0x56472e34fa00 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18717e9700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e3529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:19.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18727eb700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f186400c6f0 tx=0x7f186401b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf0a56c4e3975df server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640b4060 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f18640a3bb0 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c005550 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f186409bb60 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f187ead1b80 10 client.17091 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56472e360d30 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c005550 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.645+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f1864004420 con 0x56472e34fa00 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 10 client.17091 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 10 client.17091 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 20 client.17091 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:19.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3632c0 0x56472e3656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 4 client.17091 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472e2a3ea0 con 0x56472e3632c0 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f1871fea700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3632c0 0x56472e3656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f18557fa700 10 client.17091 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.646+0000 7f1871fea700 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3632c0 0x56472e3656b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08b4dbc710f0255 server_cookie=436d4201665ce30c in_seq=0 out_seq=0 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.647+0000 7f18557fa700 1 -- 172.21.15.87:0/2863497581 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56472e2a3ea0 con 0x56472e3632c0 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.647+0000 7f18557fa700 10 client.17091 handle_command_reply: tid=0 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 client.17091 shutdown 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 msgr2=0x56472e352f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f186400c6f0 tx=0x7f186401b320 comp rx=0 tx=0).stop 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 shutdown_connections 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3632c0 0x56472e3656b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34fa00 0x56472e352f30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f18557fa700 0 client.17091 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e349870 0x56472e3556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2863497581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e3529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 >> 172.21.15.87:0/2863497581 conn(0x56472e26fe50 msgr2=0x56472e349c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 shutdown_connections 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 1 -- 172.21.15.87:0/2863497581 wait complete. 2024-06-06T07:58:19.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.648+0000 7f187ead1b80 20 client.17091 trim_cache size 0 max 16384 2024-06-06T07:58:19.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 Processor -- start 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 -- start start 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e36aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e35b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1aed40 con 0x56472e34f0e0 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ac5d0 con 0x56472e349870 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f187ead1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1aa440 con 0x56472e34fa00 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f18717e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46456/0 (socket says 172.21.15.87:46456) 2024-06-06T07:58:19.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f18717e9700 1 -- 172.21.15.87:0/2162443027 learned_addr learned my addr 172.21.15.87:0/2162443027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f18727eb700 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e35b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f1871fea700 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e36aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.669+0000 7f18717e9700 1 -- 172.21.15.87:0/2162443027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 msgr2=0x56472e35b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18717e9700 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e35b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18717e9700 1 -- 172.21.15.87:0/2162443027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 msgr2=0x56472e36aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18717e9700 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e36aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18717e9700 1 -- 172.21.15.87:0/2162443027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472e355330 con 0x56472e34f0e0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18717e9700 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f185c025040 tx=0x7f185c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec0aee7dd36aba1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18577fe700 1 -- 172.21.15.87:0/2162443027 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c027050 con 0x56472e34f0e0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18577fe700 1 -- 172.21.15.87:0/2162443027 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c028040 con 0x56472e34f0e0 2024-06-06T07:58:19.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.670+0000 7f18577fe700 1 -- 172.21.15.87:0/2162443027 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00f460 con 0x56472e34f0e0 2024-06-06T07:58:19.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- 172.21.15.87:0/2162443027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 msgr2=0x56472e36b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f185c025040 tx=0x7f185c01a040 comp rx=0 tx=0).stop 2024-06-06T07:58:19.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- 172.21.15.87:0/2162443027 shutdown_connections 2024-06-06T07:58:19.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e349870 0x56472e36aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e35b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 --2- 172.21.15.87:0/2162443027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e34f0e0 0x56472e36b400 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- 172.21.15.87:0/2162443027 >> 172.21.15.87:0/2162443027 conn(0x56472e26fe50 msgr2=0x56472e3529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- 172.21.15.87:0/2162443027 shutdown_connections 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- 172.21.15.87:0/2162443027 wait complete. 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 -- start start 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.671+0000 7f187ead1b80 1 Processor -- start 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e358850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e358d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1aaa90 con 0x56472e349870 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1abf00 con 0x56472e34f0e0 2024-06-06T07:58:19.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ab160 con 0x56472e34fa00 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 10 client.0 _ll_register_callbacks cb 0x56472e230ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46470/0 (socket says 172.21.15.87:46470) 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 -- 172.21.15.87:0/1775170531 learned_addr learned my addr 172.21.15.87:0/1775170531 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18727eb700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e358d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18717e9700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e358850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 -- 172.21.15.87:0/1775170531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 msgr2=0x56472e358d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e358d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 -- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 msgr2=0x56472e358850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e358850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 -- 172.21.15.87:0/1775170531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472e3592d0 con 0x56472e349870 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18717e9700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e358850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f1871fea700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1868005bf0 tx=0x7f18680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724e13d47eb9366b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18557fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:19.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868037060 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c028040 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.672+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1868032430 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f186803e050 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 10 client.17097 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56472e3ceb40 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c028040 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f186801d530 con 0x56472e349870 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 10 client.17097 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 10 client.17097 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 20 client.17097 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3cf210 0x56472e3d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 4 client.17097 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-06T07:58:19.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56472e29e420 con 0x56472e3cf210 2024-06-06T07:58:19.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.673+0000 7f18717e9700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3cf210 0x56472e3d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:19.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.674+0000 7f18557fa700 10 client.17097 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:19.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.674+0000 7f18717e9700 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3cf210 0x56472e3d1620 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e51b3d4f2a55181 server_cookie=930eab4a83e7056c in_seq=0 out_seq=0 2024-06-06T07:58:19.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612178 6.91 MB/sec execute 383 sec latency 61.471 ms 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f18557fa700 1 -- 172.21.15.87:0/1775170531 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56472e29e420 con 0x56472e3cf210 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f18557fa700 10 client.17097 handle_command_reply: tid=0 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 client.17097 shutdown 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 msgr2=0x56472e35bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1868005bf0 tx=0x7f18680047c0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 shutdown_connections 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56472e3cf210 0x56472e3d1620 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56472e34fa00 0x56472e358d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f18557fa700 0 client.17097 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56472e349870 0x56472e35bca0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 --2- 172.21.15.87:0/1775170531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56472e34f0e0 0x56472e358850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 >> 172.21.15.87:0/1775170531 conn(0x56472e26fe50 msgr2=0x56472e3512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.773+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 shutdown_connections 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187ead1b80 1 -- 172.21.15.87:0/1775170531 wait complete. 2024-06-06T07:58:19.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187ead1b80 20 client.17097 trim_cache size 0 max 16384 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 36, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 30, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "45a5c96b": { 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 30, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:58:19.776 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:19.777 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 msgr2=0x7f18580a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 0x7f18580a5370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f186c107d50 tx=0x7f1868009260 comp rx=0 tx=0).stop 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 msgr2=0x7f186c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f186400efc0 tx=0x7f186400c3a0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 shutdown_connections 2024-06-06T07:58:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f186c107570 0x7f186c198990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18580a2eb0 0x7f18580a5370 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f186c100e50 0x7f186c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 --2- 172.21.15.87:0/4135467392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c101810 0x7f186c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 >> 172.21.15.87:0/4135467392 conn(0x7f186c0766b0 msgr2=0x7f186c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 shutdown_connections 2024-06-06T07:58:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:19.774+0000 7f187424e700 1 -- 172.21.15.87:0/4135467392 wait complete. 2024-06-06T07:58:20.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:19 smithi181 ceph-mon[36790]: pgmap v322: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:19 smithi087 ceph-mon[32276]: pgmap v322: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:19 smithi195 ceph-mon[37040]: pgmap v322: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:20.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.89 MB/sec execute 384 sec latency 893.621 ms 2024-06-06T07:58:21.285 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '45a5c96b', quiesce took 7.8 seconds. Will release after: 22.2 2024-06-06T07:58:21.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.88 MB/sec execute 385 sec latency 1893.653 ms 2024-06-06T07:58:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:21 smithi181 ceph-mon[36790]: pgmap v323: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:21 smithi087 ceph-mon[32276]: pgmap v323: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:21 smithi195 ceph-mon[37040]: pgmap v323: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-06-06T07:58:22.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.86 MB/sec execute 386 sec latency 2893.726 ms 2024-06-06T07:58:23.679 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.84 MB/sec execute 387 sec latency 3893.780 ms 2024-06-06T07:58:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: pgmap v324: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:58:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:23 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: pgmap v324: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:58:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:23 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: pgmap v324: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:23 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:24.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.82 MB/sec execute 388 sec latency 4893.806 ms 2024-06-06T07:58:25.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.81 MB/sec execute 389 sec latency 5894.126 ms 2024-06-06T07:58:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- 172.21.15.87:0/1370279988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 msgr2=0x7f055410ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 --2- 172.21.15.87:0/1370279988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f055410ba30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0544009900 tx=0x7f054402a510 comp rx=0 tx=0).stop 2024-06-06T07:58:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- 172.21.15.87:0/1370279988 shutdown_connections 2024-06-06T07:58:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 --2- 172.21.15.87:0/1370279988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 0x7f055410e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 --2- 172.21.15.87:0/1370279988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f055410ba30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 --2- 172.21.15.87:0/1370279988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 0x7f0554103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- 172.21.15.87:0/1370279988 >> 172.21.15.87:0/1370279988 conn(0x7f0554077a20 msgr2=0x7f0554077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- 172.21.15.87:0/1370279988 shutdown_connections 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- 172.21.15.87:0/1370279988 wait complete. 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 Processor -- start 2024-06-06T07:58:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.942+0000 7f0559370640 1 -- start start 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 0x7f05541981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 0x7f055419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0554111d70 con 0x7f0554103550 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0554111bf0 con 0x7f0554102c30 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0559370640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0554111ef0 con 0x7f055410bf70 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46494/0 (socket says 172.21.15.87:46494) 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 -- 172.21.15.87:0/684909033 learned_addr learned my addr 172.21.15.87:0/684909033 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05537fe640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 0x7f055419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f0552ffd640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 0x7f05541981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 -- 172.21.15.87:0/684909033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 msgr2=0x7f055419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 0x7f055419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 -- 172.21.15.87:0/684909033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 msgr2=0x7f05541981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 0x7f05541981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 -- 172.21.15.87:0/684909033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055419d1e0 con 0x7f0554103550 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.943+0000 7f05527fc640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0544006010 tx=0x7f05440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.944+0000 7f054bfff640 1 -- 172.21.15.87:0/684909033 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054403f030 con 0x7f0554103550 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.944+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055419d470 con 0x7f0554103550 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.944+0000 7f054bfff640 1 -- 172.21.15.87:0/684909033 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0544003ce0 con 0x7f0554103550 2024-06-06T07:58:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.944+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05541a4d60 con 0x7f0554103550 2024-06-06T07:58:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.944+0000 7f054bfff640 1 -- 172.21.15.87:0/684909033 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544035070 con 0x7f0554103550 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 Processor -- start 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 -- start start 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514f6350 0x55f8514f8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f85144f4f0 con 0x55f8514f6350 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f8513ed870 con 0x55f8514eb170 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.946+0000 7f055bd22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f85144b4b0 con 0x55f8514eba90 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52680/0 (socket says 172.21.15.87:52680) 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 -- 172.21.15.87:0/1190029091 learned_addr learned my addr 172.21.15.87:0/1190029091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f05537fe640 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 -- 172.21.15.87:0/1190029091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 msgr2=0x55f8514eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 -- 172.21.15.87:0/1190029091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514f6350 msgr2=0x55f8514f8740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514f6350 0x55f8514f8740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 -- 172.21.15.87:0/1190029091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8514099b0 con 0x55f8514eba90 2024-06-06T07:58:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f05537fe640 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f054bfff640 1 -- 172.21.15.87:0/684909033 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0544042030 con 0x7f0554103550 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f053c00bcd0 tx=0x7f053c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f192a635df75778a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f05497fa640 1 -- 172.21.15.87:0/1190029091 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c005220 con 0x55f8514eba90 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f05497fa640 1 -- 172.21.15.87:0/1190029091 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c004c80 con 0x55f8514eba90 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.947+0000 7f05497fa640 1 -- 172.21.15.87:0/1190029091 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c011780 con 0x55f8514eba90 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f054bfff640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 0x7f05400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f05537fe640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 0x7f05400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f054bfff640 1 -- 172.21.15.87:0/684909033 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f05440ef8f0 con 0x7f0554103550 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 -- 172.21.15.87:0/1190029091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 msgr2=0x55f8514f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f053c00bcd0 tx=0x7f053c0107a0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f05537fe640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 0x7f05400a5260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f054c005fd0 tx=0x7f054c009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 -- 172.21.15.87:0/1190029091 shutdown_connections 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514f6350 0x55f8514f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eba90 0x55f8514f5e10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 --2- 172.21.15.87:0/1190029091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 -- 172.21.15.87:0/1190029091 >> 172.21.15.87:0/1190029091 conn(0x55f8514e7680 msgr2=0x55f8514e7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 -- 172.21.15.87:0/1190029091 shutdown_connections 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.948+0000 7f055bd22740 1 -- 172.21.15.87:0/1190029091 wait complete. 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 -- start start 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 Processor -- start 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514f6350 0x55f8514f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f8514f8f80 con 0x55f8514eba90 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f8513d8ae0 con 0x55f8514f6350 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8514f9100 con 0x55f8514eb170 2024-06-06T07:58:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f055bd22740 10 client.0 _ll_register_callbacks cb 0x55f8514119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52682/0 (socket says 172.21.15.87:52682) 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46508/0 (socket says 172.21.15.87:46508) 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.949+0000 7f0552ffd640 1 -- 172.21.15.87:0/1767645703 learned_addr learned my addr 172.21.15.87:0/1767645703 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f05537fe640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514f6350 0x55f8514f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 -- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514f6350 msgr2=0x55f8514f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514f6350 0x55f8514f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 -- 172.21.15.87:0/1767645703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 msgr2=0x55f8514f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 -- 172.21.15.87:0/1767645703 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f851163210 con 0x55f8514eb170 2024-06-06T07:58:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f05527fc640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f0552ffd640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f053c005700 tx=0x7f053c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86a8d9ad5460a29 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c00b8c0 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f053c005040 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f053c004c80 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f053c005240 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 10 client.26365 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f85122f620 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c004c80 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f053c003b90 con 0x55f8514eb170 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 10 client.26365 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.950+0000 7f055bd22740 20 client.26365 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851508890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f055bd22740 4 client.26365 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f851462f10 con 0x55f8515064a0 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f05527fc640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851508890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f053a7fc640 10 client.26365 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.951+0000 7f05527fc640 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851508890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f158c5783228d server_cookie=e667a320227245b9 in_seq=0 out_seq=0 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f053a7fc640 1 -- 172.21.15.87:0/1767645703 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f851462f10 con 0x55f8515064a0 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f053a7fc640 10 client.26365 handle_command_reply: tid=0 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 client.26365 shutdown 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 msgr2=0x55f8514f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f053c005700 tx=0x7f053c0048f0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 shutdown_connections 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851508890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514f6350 0x55f8514f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f053a7fc640 0 client.26365 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 --2- 172.21.15.87:0/1767645703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514eb170 0x55f8514f3380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 >> 172.21.15.87:0/1767645703 conn(0x55f8514e7680 msgr2=0x55f8514ed3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 shutdown_connections 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 1 -- 172.21.15.87:0/1767645703 wait complete. 2024-06-06T07:58:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.952+0000 7f055bd22740 20 client.26365 trim_cache size 0 max 16384 2024-06-06T07:58:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.967+0000 7f055bd22740 1 Processor -- start 2024-06-06T07:58:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.967+0000 7f055bd22740 1 -- start start 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f85150df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f8514f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05541b1910 con 0x55f8514eba90 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05541af1e0 con 0x55f8514eb170 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f055bd22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05541ad050 con 0x55f8514f6350 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46524/0 (socket says 172.21.15.87:46524) 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 -- 172.21.15.87:0/2976257277 learned_addr learned my addr 172.21.15.87:0/2976257277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05537fe640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f8514f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f0552ffd640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f85150df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 -- 172.21.15.87:0/2976257277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 msgr2=0x55f8514f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f8514f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 -- 172.21.15.87:0/2976257277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 msgr2=0x55f85150df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f85150df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f05527fc640 1 -- 172.21.15.87:0/2976257277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8514eb8c0 con 0x55f8514eba90 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.968+0000 7f0552ffd640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f85150df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.969+0000 7f05527fc640 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0544005ec0 tx=0x7f054402a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27267f82048b9eb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.969+0000 7f0548ff9640 1 -- 172.21.15.87:0/2976257277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544049040 con 0x55f8514eba90 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.969+0000 7f0548ff9640 1 -- 172.21.15.87:0/2976257277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05440bbb60 con 0x55f8514eba90 2024-06-06T07:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.969+0000 7f0548ff9640 1 -- 172.21.15.87:0/2976257277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440ecea0 con 0x55f8514eba90 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- 172.21.15.87:0/2976257277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 msgr2=0x55f85150e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0544005ec0 tx=0x7f054402a510 comp rx=0 tx=0).stop 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- 172.21.15.87:0/2976257277 shutdown_connections 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f8514f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f85150e4c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 --2- 172.21.15.87:0/2976257277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f85150df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- 172.21.15.87:0/2976257277 >> 172.21.15.87:0/2976257277 conn(0x55f8514e7680 msgr2=0x55f8514f1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- 172.21.15.87:0/2976257277 shutdown_connections 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- 172.21.15.87:0/2976257277 wait complete. 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 -- start start 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.970+0000 7f055bd22740 1 Processor -- start 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f85156ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05541ad6a0 con 0x55f8514eba90 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05541aeb10 con 0x55f8514eb170 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05541add70 con 0x55f8514f6350 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 10 client.0 _ll_register_callbacks cb 0x7f053c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46528/0 (socket says 172.21.15.87:46528) 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 -- 172.21.15.87:0/3555445640 learned_addr learned my addr 172.21.15.87:0/3555445640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05537fe640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f85156ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f0552ffd640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 -- 172.21.15.87:0/3555445640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 msgr2=0x55f85156ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f85156ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 -- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 msgr2=0x55f8514f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 -- 172.21.15.87:0/3555445640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85156f480 con 0x55f8514eba90 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f0552ffd640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f05527fc640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0544005940 tx=0x7f0544031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce869f8896b9a4b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f053a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544049070 con 0x55f8514eba90 2024-06-06T07:58:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f05440b81d0 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05440bbb60 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f05440ecea0 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 10 client.17112 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.971+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f851570590 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05440bbb60 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f054404c060 con 0x55f8514eba90 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 10 client.17112 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 10 client.17112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 20 client.17112 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851572bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 4 client.17112 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f85145cc20 con 0x55f8515064a0 2024-06-06T07:58:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.972+0000 7f0552ffd640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851572bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.973+0000 7f0552ffd640 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851572bb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17a6b573253f382 server_cookie=c12eb4303db7fc85 in_seq=0 out_seq=0 2024-06-06T07:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:25.973+0000 7f053a7fc640 10 client.17112 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f053a7fc640 1 -- 172.21.15.87:0/3555445640 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f85145cc20 con 0x55f8515064a0 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f053a7fc640 10 client.17112 handle_command_reply: tid=0 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 client.17112 shutdown 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 msgr2=0x55f8514f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0544005940 tx=0x7f0544031a60 comp rx=0 tx=0).stop 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 shutdown_connections 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f8515064a0 0x55f851572bb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8514f6350 0x55f85156ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f8514eba90 0x55f8514f9a30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f053a7fc640 0 client.17112 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 --2- 172.21.15.87:0/3555445640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8514eb170 0x55f8514f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 >> 172.21.15.87:0/3555445640 conn(0x55f8514e7680 msgr2=0x55f8514ecb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 shutdown_connections 2024-06-06T07:58:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 1 -- 172.21.15.87:0/3555445640 wait complete. 2024-06-06T07:58:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.139+0000 7f055bd22740 20 client.17112 trim_cache size 0 max 16384 2024-06-06T07:58:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:58:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:58:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 msgr2=0x7f05400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 0x7f05400a5260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f054c005fd0 tx=0x7f054c009500 comp rx=0 tx=0).stop 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 msgr2=0x7f0554198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0544006010 tx=0x7f05440042f0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 shutdown_connections 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f05400a2da0 0x7f05400a5260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f055410bf70 0x7f055419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0554103550 0x7f0554198730 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 --2- 172.21.15.87:0/684909033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0554102c30 0x7f05541981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 >> 172.21.15.87:0/684909033 conn(0x7f0554077a20 msgr2=0x7f05540fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 shutdown_connections 2024-06-06T07:58:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:26.140+0000 7f0559370640 1 -- 172.21.15.87:0/684909033 wait complete. 2024-06-06T07:58:26.525 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: pgmap v325: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:26.525 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.525 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.526 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:26 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.79 MB/sec execute 390 sec latency 6894.156 ms 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: pgmap v325: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:26 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: pgmap v325: 129 pgs: 129 active+clean; 101 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-06-06T07:58:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:26.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:26 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:27.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.77 MB/sec execute 391 sec latency 7894.266 ms 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:27.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:27 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:27.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:27 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:27.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:27 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:28.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.75 MB/sec execute 392 sec latency 8894.382 ms 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: pgmap v326: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:58:28.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: pgmap v326: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:58:28.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:58:28.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: pgmap v326: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2024-06-06T07:58:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:58:28.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:58:28.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:58:28.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:58:29.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.74 MB/sec execute 393 sec latency 9894.430 ms 2024-06-06T07:58:29.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:29 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:58:29.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:29 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:58:29.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:29 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:58:29.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:29 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:58:29.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:29 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T07:58:29.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:29 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T07:58:30.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.72 MB/sec execute 394 sec latency 10894.464 ms 2024-06-06T07:58:30.707 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:30 smithi181 ceph-mon[36790]: pgmap v327: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-06-06T07:58:30.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:30 smithi195 ceph-mon[37040]: pgmap v327: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-06-06T07:58:30.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:30 smithi087 ceph-mon[32276]: pgmap v327: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-06-06T07:58:31.676 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:31 smithi195 ceph-mon[37040]: pgmap v328: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-06-06T07:58:31.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.70 MB/sec execute 395 sec latency 11894.494 ms 2024-06-06T07:58:31.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:31 smithi181 ceph-mon[36790]: pgmap v328: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-06-06T07:58:31.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:31 smithi087 ceph-mon[32276]: pgmap v328: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-06-06T07:58:32.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.69 MB/sec execute 396 sec latency 12894.528 ms 2024-06-06T07:58:33.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.67 MB/sec execute 397 sec latency 13894.632 ms 2024-06-06T07:58:34.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:33 smithi195 ceph-mon[37040]: pgmap v329: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2024-06-06T07:58:34.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:33 smithi181 ceph-mon[36790]: pgmap v329: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2024-06-06T07:58:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:33 smithi087 ceph-mon[32276]: pgmap v329: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2024-06-06T07:58:34.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.65 MB/sec execute 398 sec latency 14894.695 ms 2024-06-06T07:58:35.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.64 MB/sec execute 399 sec latency 15894.751 ms 2024-06-06T07:58:36.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:35 smithi195 ceph-mon[37040]: pgmap v330: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T07:58:36.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:35 smithi181 ceph-mon[36790]: pgmap v330: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T07:58:36.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:35 smithi087 ceph-mon[32276]: pgmap v330: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T07:58:36.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.62 MB/sec execute 400 sec latency 16894.806 ms 2024-06-06T07:58:37.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.60 MB/sec execute 401 sec latency 17894.910 ms 2024-06-06T07:58:38.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:37 smithi195 ceph-mon[37040]: pgmap v331: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-06-06T07:58:38.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:37 smithi181 ceph-mon[36790]: pgmap v331: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-06-06T07:58:38.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:37 smithi087 ceph-mon[32276]: pgmap v331: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-06-06T07:58:38.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.59 MB/sec execute 402 sec latency 18894.942 ms 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 -- 172.21.15.87:0/3507981167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334101260 msgr2=0x7f033410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 --2- 172.21.15.87:0/3507981167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334101260 0x7f033410d770 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f032c009cd0 tx=0x7f032c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 -- 172.21.15.87:0/3507981167 shutdown_connections 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 --2- 172.21.15.87:0/3507981167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334101260 0x7f033410d770 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 --2- 172.21.15.87:0/3507981167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 0x7f0334100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 --2- 172.21.15.87:0/3507981167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03341068b0 0x7f0334106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 -- 172.21.15.87:0/3507981167 >> 172.21.15.87:0/3507981167 conn(0x7f03340779b0 msgr2=0x7f0334077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 -- 172.21.15.87:0/3507981167 shutdown_connections 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.236+0000 7f0339509640 1 -- 172.21.15.87:0/3507981167 wait complete. 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 Processor -- start 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 -- start start 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 0x7f0334198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 0x7f03341987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110cc0 con 0x7f03341068b0 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110b40 con 0x7f03341008a0 2024-06-06T07:58:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f0339509640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110e40 con 0x7f0334101260 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f03337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f03337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35232/0 (socket says 172.21.15.87:35232) 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.237+0000 7f03337fe640 1 -- 172.21.15.87:0/3079347714 learned_addr learned my addr 172.21.15.87:0/3079347714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0332ffd640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 0x7f0334198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f032bfff640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 0x7f03341987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 -- 172.21.15.87:0/3079347714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 msgr2=0x7f03341987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 0x7f03341987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 -- 172.21.15.87:0/3079347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 msgr2=0x7f0334198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 0x7f0334198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 -- 172.21.15.87:0/3079347714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033419d260 con 0x7f03341068b0 2024-06-06T07:58:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f032bfff640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 0x7f03341987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f03337fe640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f032c005cc0 tx=0x7f032c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0330ff9640 1 -- 172.21.15.87:0/3079347714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c004040 con 0x7f03341068b0 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0330ff9640 1 -- 172.21.15.87:0/3079347714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032c0046c0 con 0x7f03341068b0 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033419d4f0 con 0x7f03341068b0 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0330ff9640 1 -- 172.21.15.87:0/3079347714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c046480 con 0x7f03341068b0 2024-06-06T07:58:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.238+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a4de0 con 0x7f03341068b0 2024-06-06T07:58:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.240+0000 7f0330ff9640 1 -- 172.21.15.87:0/3079347714 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f032c0305a0 con 0x7f03341068b0 2024-06-06T07:58:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 Processor -- start 2024-06-06T07:58:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 -- start start 2024-06-06T07:58:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556ea6a1a4f0 con 0x556ea6ab6070 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.241+0000 7f033c20f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ea69b8870 con 0x556ea6ac5760 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f033c20f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556ea6a164b0 con 0x556ea6ab6990 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f0330ff9640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 0x7f03200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35238/0 (socket says 172.21.15.87:35238) 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 -- 172.21.15.87:0/1203395522 learned_addr learned my addr 172.21.15.87:0/1203395522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f0332ffd640 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f03337fe640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 0x7f03200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f0330ff9640 1 -- 172.21.15.87:0/3079347714 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f032c02e030 con 0x7f03341068b0 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 -- 172.21.15.87:0/1203395522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 msgr2=0x556ea6ac5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 -- 172.21.15.87:0/1203395522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 msgr2=0x556ea6ac7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 -- 172.21.15.87:0/1203395522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea69d49b0 con 0x556ea6ab6070 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f032bfff640 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f031c0145f0 tx=0x7f031c00b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ffb9cfcc04942d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f03337fe640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 0x7f03200a52e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f032c0bcec0 tx=0x7f032c03d040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f03297fa640 1 -- 172.21.15.87:0/1203395522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c005220 con 0x556ea6ab6070 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f03297fa640 1 -- 172.21.15.87:0/1203395522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031c005e80 con 0x556ea6ab6070 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.242+0000 7f03297fa640 1 -- 172.21.15.87:0/1203395522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c012770 con 0x556ea6ab6070 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 -- 172.21.15.87:0/1203395522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 msgr2=0x556ea6ab6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f031c0145f0 tx=0x7f031c00b5d0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 -- 172.21.15.87:0/1203395522 shutdown_connections 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 --2- 172.21.15.87:0/1203395522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ab6450 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 -- 172.21.15.87:0/1203395522 >> 172.21.15.87:0/1203395522 conn(0x556ea6ab2680 msgr2=0x556ea6ab2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 -- 172.21.15.87:0/1203395522 shutdown_connections 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.243+0000 7f033c20f740 1 -- 172.21.15.87:0/1203395522 wait complete. 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 -- start start 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 Processor -- start 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556ea6ac8390 con 0x556ea6ab6070 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ea69a3ae0 con 0x556ea6ac5760 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556ea6ac8510 con 0x556ea6ab6990 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f033c20f740 10 client.0 _ll_register_callbacks cb 0x556ea69dc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35248/0 (socket says 172.21.15.87:35248) 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 -- 172.21.15.87:0/1230115094 learned_addr learned my addr 172.21.15.87:0/1230115094 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f0332ffd640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 -- 172.21.15.87:0/1230115094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 msgr2=0x556ea6ac2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 -- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 msgr2=0x556ea6ac3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f032bfff640 1 -- 172.21.15.87:0/1230115094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea672e210 con 0x556ea6ab6070 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.244+0000 7f0332ffd640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f032bfff640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f031c0055b0 tx=0x7f031c00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5212399b0e5603 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0048c0 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f031c004a60 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ea6a2df10 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f031c031050 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 10 client.17124 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556ea67fa620 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556ea6a2df10 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f031c026020 con 0x556ea6ab6070 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 10 client.17124 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 20 client.17124 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6ad5860 0x556ea6ad7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 4 client.17124 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.245+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ea6a27c20 con 0x556ea6ad5860 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.246+0000 7f0332ffd640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6ad5860 0x556ea6ad7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.246+0000 7f0332ffd640 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6ad5860 0x556ea6ad7c50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76855c0a80dfa3c server_cookie=b59a3e776c8ffc83 in_seq=0 out_seq=0 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.246+0000 7f031a7fc640 10 client.17124 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f031a7fc640 1 -- 172.21.15.87:0/1230115094 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ea6a27c20 con 0x556ea6ad5860 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f031a7fc640 10 client.17124 handle_command_reply: tid=0 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 client.17124 shutdown 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 msgr2=0x556ea6ac25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f031c0055b0 tx=0x7f031c00bd90 comp rx=0 tx=0).stop 2024-06-06T07:58:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 shutdown_connections 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6ad5860 0x556ea6ad7c50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6990 0x556ea6ac2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f031a7fc640 0 client.17124 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 --2- 172.21.15.87:0/1230115094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6070 0x556ea6ac25c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 >> 172.21.15.87:0/1230115094 conn(0x556ea6ab2680 msgr2=0x556ea6ab84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 shutdown_connections 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 1 -- 172.21.15.87:0/1230115094 wait complete. 2024-06-06T07:58:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.247+0000 7f033c20f740 20 client.17124 trim_cache size 0 max 16384 2024-06-06T07:58:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 Processor -- start 2024-06-06T07:58:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 -- start start 2024-06-06T07:58:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 0x556ea6ac15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ac5760 0x556ea6add8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03341af5d0 con 0x556ea6ab6990 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03341acea0 con 0x556ea6ab6070 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.262+0000 7f033c20f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aad30 con 0x556ea6ac5760 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35258/0 (socket says 172.21.15.87:35258) 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 -- 172.21.15.87:0/2444686131 learned_addr learned my addr 172.21.15.87:0/2444686131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f032bfff640 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 0x556ea6ac15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 -- 172.21.15.87:0/2444686131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ac5760 msgr2=0x556ea6add8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ac5760 0x556ea6add8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 -- 172.21.15.87:0/2444686131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 msgr2=0x556ea6ac15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 0x556ea6ac15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 -- 172.21.15.87:0/2444686131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea6ab67c0 con 0x556ea6ab6990 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f032bfff640 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 0x556ea6ac15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0324005d00 tx=0x7f032400b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5bfa7e83c8f90f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0328ff9640 1 -- 172.21.15.87:0/2444686131 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240140d0 con 0x556ea6ab6990 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0328ff9640 1 -- 172.21.15.87:0/2444686131 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032401f020 con 0x556ea6ab6990 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.263+0000 7f0328ff9640 1 -- 172.21.15.87:0/2444686131 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324003da0 con 0x556ea6ab6990 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- 172.21.15.87:0/2444686131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 msgr2=0x556ea6add370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0324005d00 tx=0x7f032400b4d0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- 172.21.15.87:0/2444686131 shutdown_connections 2024-06-06T07:58:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ac5760 0x556ea6add8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6add370 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- 172.21.15.87:0/2444686131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ab6070 0x556ea6ac15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- 172.21.15.87:0/2444686131 >> 172.21.15.87:0/2444686131 conn(0x556ea6ab2680 msgr2=0x556ea6ac0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- 172.21.15.87:0/2444686131 shutdown_connections 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- 172.21.15.87:0/2444686131 wait complete. 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 -- start start 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 Processor -- start 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 0x556ea6ac8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.264+0000 7f033c20f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ab380 con 0x556ea6ab6990 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ac7d0 con 0x556ea6ac5760 2024-06-06T07:58:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aba30 con 0x556ea6ab6070 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 10 client.0 _ll_register_callbacks cb 0x7f032400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35264/0 (socket says 172.21.15.87:35264) 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 -- 172.21.15.87:0/2365474071 learned_addr learned my addr 172.21.15.87:0/2365474071 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f032bfff640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 0x556ea6ac8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f032bfff640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 -- 172.21.15.87:0/2365474071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 msgr2=0x556ea6ac8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 0x556ea6ac8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 -- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 msgr2=0x556ea6ac9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 -- 172.21.15.87:0/2365474071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea6b3e330 con 0x556ea6ab6990 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f032bfff640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 0x556ea6ac8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f0332ffd640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f032400bc50 tx=0x7f0324003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc457b08fa0df9e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f031a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324014290 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f0324005020 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f032401f020 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0324005220 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 10 client.17130 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.265+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556ea6b3f5a0 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032401f020 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f0324003ef0 con 0x556ea6ab6990 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 10 client.17130 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 20 client.17130 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6b3fd70 0x556ea6b42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 4 client.17130 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:58:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ea6a21f70 con 0x556ea6b3fd70 2024-06-06T07:58:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.266+0000 7f032bfff640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6b3fd70 0x556ea6b42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.267+0000 7f032bfff640 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6b3fd70 0x556ea6b42190 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f4edd09c81c62f server_cookie=1f61fdcf3734ee53 in_seq=0 out_seq=0 2024-06-06T07:58:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.267+0000 7f031a7fc640 10 client.17130 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f031a7fc640 1 -- 172.21.15.87:0/2365474071 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ea6a21f70 con 0x556ea6b3fd70 2024-06-06T07:58:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f031a7fc640 10 client.17130 handle_command_reply: tid=0 2024-06-06T07:58:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 client.17130 shutdown 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 msgr2=0x556ea6ac8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f032400bc50 tx=0x7f0324003820 comp rx=0 tx=0).stop 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 shutdown_connections 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556ea6b3fd70 0x556ea6b42190 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ea6ac5760 0x556ea6ac9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f031a7fc640 0 client.17130 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ea6ab6990 0x556ea6ac8cd0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 --2- 172.21.15.87:0/2365474071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556ea6ab6070 0x556ea6ac8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 >> 172.21.15.87:0/2365474071 conn(0x556ea6ab2680 msgr2=0x556ea6ab7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 shutdown_connections 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 1 -- 172.21.15.87:0/2365474071 wait complete. 2024-06-06T07:58:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.325+0000 7f033c20f740 20 client.17130 trim_cache size 0 max 16384 2024-06-06T07:58:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:58:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:58:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 msgr2=0x7f03200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 0x7f03200a52e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f032c0bcec0 tx=0x7f032c03d040 comp rx=0 tx=0).stop 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 msgr2=0x7f033419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f032c005cc0 tx=0x7f032c003970 comp rx=0 tx=0).stop 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 shutdown_connections 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f03200a2e20 0x7f03200a52e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03341068b0 0x7f033419cb30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0334101260 0x7f03341987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 --2- 172.21.15.87:0/3079347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03341008a0 0x7f0334198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 >> 172.21.15.87:0/3079347714 conn(0x7f03340779b0 msgr2=0x7f0334069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 shutdown_connections 2024-06-06T07:58:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:39.326+0000 7f0339509640 1 -- 172.21.15.87:0/3079347714 wait complete. 2024-06-06T07:58:39.680 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.57 MB/sec execute 403 sec latency 19894.978 ms 2024-06-06T07:58:40.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:39 smithi195 ceph-mon[37040]: pgmap v332: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-06-06T07:58:40.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:39 smithi181 ceph-mon[36790]: pgmap v332: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-06-06T07:58:40.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:39 smithi087 ceph-mon[32276]: pgmap v332: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-06-06T07:58:40.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.55 MB/sec execute 404 sec latency 20895.071 ms 2024-06-06T07:58:41.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.54 MB/sec execute 405 sec latency 21895.159 ms 2024-06-06T07:58:42.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:41 smithi195 ceph-mon[37040]: pgmap v333: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1022 KiB/s wr, 62 op/s 2024-06-06T07:58:42.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:41 smithi181 ceph-mon[36790]: pgmap v333: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1022 KiB/s wr, 62 op/s 2024-06-06T07:58:42.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:41 smithi087 ceph-mon[32276]: pgmap v333: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1022 KiB/s wr, 62 op/s 2024-06-06T07:58:42.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.52 MB/sec execute 406 sec latency 22895.202 ms 2024-06-06T07:58:43.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:58:43.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.51 MB/sec execute 407 sec latency 23895.251 ms 2024-06-06T07:58:43.781 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:43.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:43 smithi087 ceph-mon[32276]: pgmap v334: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-06-06T07:58:44.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:43 smithi195 ceph-mon[37040]: pgmap v334: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-06-06T07:58:44.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:43 smithi181 ceph-mon[36790]: pgmap v334: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-06-06T07:58:44.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.49 MB/sec execute 408 sec latency 24895.342 ms 2024-06-06T07:58:45.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.47 MB/sec execute 409 sec latency 25895.386 ms 2024-06-06T07:58:45.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.767+0000 7f8d481d2700 1 -- 172.21.15.87:0/2903165567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40101d50 msgr2=0x7f8d4010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.767+0000 7f8d481d2700 1 --2- 172.21.15.87:0/2903165567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40101d50 0x7f8d4010e240 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800b0a0 tx=0x7f8d3800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 -- 172.21.15.87:0/2903165567 shutdown_connections 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 --2- 172.21.15.87:0/2903165567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40107400 0x7f8d401077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 --2- 172.21.15.87:0/2903165567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40101d50 0x7f8d4010e240 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 --2- 172.21.15.87:0/2903165567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d40101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 -- 172.21.15.87:0/2903165567 >> 172.21.15.87:0/2903165567 conn(0x7f8d40078340 msgr2=0x7f8d40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 -- 172.21.15.87:0/2903165567 shutdown_connections 2024-06-06T07:58:45.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 -- 172.21.15.87:0/2903165567 wait complete. 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 Processor -- start 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.768+0000 7f8d481d2700 1 -- start start 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d401999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 0x7f8d40199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40111670 con 0x7f8d40107400 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401114f0 con 0x7f8d40101d50 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d481d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401117f0 con 0x7f8d40101390 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35284/0 (socket says 172.21.15.87:35284) 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 -- 172.21.15.87:0/3374335608 learned_addr learned my addr 172.21.15.87:0/3374335608 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4576d700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 0x7f8d40199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d45f6e700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d401999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 -- 172.21.15.87:0/3374335608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 msgr2=0x7f8d401999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d401999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 -- 172.21.15.87:0/3374335608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 msgr2=0x7f8d40199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 0x7f8d40199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 -- 172.21.15.87:0/3374335608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401a4fb0 con 0x7f8d40107400 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4576d700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 0x7f8d40199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d45f6e700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d401999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.769+0000 7f8d4676f700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8d380062a0 tx=0x7f8d38004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38021070 con 0x7f8d40107400 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401a5180 con 0x7f8d40107400 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a5680 con 0x7f8d40107400 2024-06-06T07:58:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d38007c80 con 0x7f8d40107400 2024-06-06T07:58:45.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380074e0 con 0x7f8d40107400 2024-06-06T07:58:45.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.770+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d24005330 con 0x7f8d40107400 2024-06-06T07:58:45.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.772+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d38004000 con 0x7f8d40107400 2024-06-06T07:58:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.772+0000 7f8d36ffd700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 0x7f8d2c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.772+0000 7f8d45f6e700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 0x7f8d2c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.772+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f8d380d0840 con 0x7f8d40107400 2024-06-06T07:58:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.773+0000 7f8d45f6e700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 0x7f8d2c0a52d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005fd0 tx=0x7f8d30005d40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:45.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:45.776+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d3802a070 con 0x7f8d40107400 2024-06-06T07:58:45.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:45 smithi087 ceph-mon[32276]: pgmap v335: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 891 KiB/s wr, 70 op/s 2024-06-06T07:58:46.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:45 smithi195 ceph-mon[37040]: pgmap v335: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 891 KiB/s wr, 70 op/s 2024-06-06T07:58:46.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:45 smithi181 ceph-mon[36790]: pgmap v335: 129 pgs: 129 active+clean; 85 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 891 KiB/s wr, 70 op/s 2024-06-06T07:58:46.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.066+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d24005620 con 0x7f8d40107400 2024-06-06T07:58:46.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.067+0000 7f8d36ffd700 1 -- 172.21.15.87:0/3374335608 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11877 (secure 0 0 0) 0x7f8d3809d150 con 0x7f8d40107400 2024-06-06T07:58:46.067 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:58:46.068 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:57:47.279422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26299":{"gid":26299,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1306245437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1306245437},{"type":"v1","addr":"172.21.15.195:6837","nonce":1306245437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:58:46.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 msgr2=0x7f8d2c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 0x7f8d2c0a52d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005fd0 tx=0x7f8d30005d40 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 msgr2=0x7f8d401a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8d380062a0 tx=0x7f8d38004e20 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 shutdown_connections 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d2c0a2e10 0x7f8d2c0a52d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d40107400 0x7f8d401a4a70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d40101390 0x7f8d401999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 --2- 172.21.15.87:0/3374335608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d40101d50 0x7f8d40199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 >> 172.21.15.87:0/3374335608 conn(0x7f8d40078340 msgr2=0x7f8d400ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 shutdown_connections 2024-06-06T07:58:46.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:46.070+0000 7f8d481d2700 1 -- 172.21.15.87:0/3374335608 wait complete. 2024-06-06T07:58:46.071 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 36 2024-06-06T07:58:46.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.46 MB/sec execute 410 sec latency 26895.414 ms 2024-06-06T07:58:46.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 45a5c96b --release --await' 2024-06-06T07:58:46.821 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 45a5c96b --release --await 2024-06-06T07:58:47.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:46 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3374335608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:47.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:46 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3374335608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:47.067 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:47.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:46 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3374335608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:47.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.44 MB/sec execute 411 sec latency 27895.732 ms 2024-06-06T07:58:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:47 smithi087 ceph-mon[32276]: pgmap v336: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-06T07:58:48.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:47 smithi195 ceph-mon[37040]: pgmap v336: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-06T07:58:48.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:47 smithi181 ceph-mon[36790]: pgmap v336: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-06-06T07:58:48.681 INFO:tasks.workunit.client.0.smithi087.stdout: 1 612297 6.43 MB/sec execute 412 sec latency 28895.771 ms 2024-06-06T07:58:48.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 -- 172.21.15.87:0/2862674345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102530 msgr2=0x7f6dc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/2862674345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102530 0x7f6dc81029b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0099c0 tx=0x7f6dbc009cd0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 -- 172.21.15.87:0/2862674345 shutdown_connections 2024-06-06T07:58:48.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/2862674345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 0x7f6dc81089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/2862674345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102ef0 0x7f6dc810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/2862674345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102530 0x7f6dc81029b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 -- 172.21.15.87:0/2862674345 >> 172.21.15.87:0/2862674345 conn(0x7f6dc8079640 msgr2=0x7f6dc8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 -- 172.21.15.87:0/2862674345 shutdown_connections 2024-06-06T07:58:48.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.923+0000 7f6dce6c8700 1 -- 172.21.15.87:0/2862674345 wait complete. 2024-06-06T07:58:48.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 Processor -- start 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 -- start start 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102530 0x7f6dc8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 0x7f6dc8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8112810 con 0x7f6dc8102ef0 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8112690 con 0x7f6dc81085d0 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.924+0000 7f6dce6c8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8112990 con 0x7f6dc8102530 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53760/0 (socket says 172.21.15.87:53760) 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 -- 172.21.15.87:0/3181544630 learned_addr learned my addr 172.21.15.87:0/3181544630 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dccc65700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 0x7f6dc8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102530 0x7f6dc8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 -- 172.21.15.87:0/3181544630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102530 msgr2=0x7f6dc8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102530 0x7f6dc8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 -- 172.21.15.87:0/3181544630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 msgr2=0x7f6dc8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 0x7f6dc8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 -- 172.21.15.87:0/3181544630 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81a71b0 con 0x7f6dc8102ef0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc57fa700 1 -- 172.21.15.87:0/3181544630 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc027070 con 0x7f6dc8102ef0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81a7440 con 0x7f6dc8102ef0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc57fa700 1 -- 172.21.15.87:0/3181544630 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dbc0040d0 con 0x7f6dc8102ef0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dc57fa700 1 -- 172.21.15.87:0/3181544630 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00f460 con 0x7f6dc8102ef0 2024-06-06T07:58:48.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.925+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81a7890 con 0x7f6dc8102ef0 2024-06-06T07:58:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.927+0000 7f6dc57fa700 1 -- 172.21.15.87:0/3181544630 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6dbc02a070 con 0x7f6dc8102ef0 2024-06-06T07:58:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.928+0000 7f6dc57fa700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 0x7f6db00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.928+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 0x7f6db00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.928+0000 7f6dc57fa700 1 -- 172.21.15.87:0/3181544630 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f6dbc0d08a0 con 0x7f6dc8102ef0 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 0x7f6db00a5250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6db8005fd0 tx=0x7f6db8005db0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 Processor -- start 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 -- start start 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50340710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50341070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fb50233060 con 0x55fb50346500 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb50291950 con 0x55fb50346e20 2024-06-06T07:58:48.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb501f9560 con 0x55fb50340c50 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53772/0 (socket says 172.21.15.87:53772) 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 -- 172.21.15.87:0/3616550088 learned_addr learned my addr 172.21.15.87:0/3616550088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50340710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dccc65700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50341070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 -- 172.21.15.87:0/3616550088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 msgr2=0x55fb50341070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50341070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 -- 172.21.15.87:0/3616550088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 msgr2=0x55fb50340710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.929+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50340710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dc7fff700 1 -- 172.21.15.87:0/3616550088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb501e09d0 con 0x55fb50346500 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dccc65700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50341070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6db800ebd0 tx=0x7f6db8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4befb1c01b3ff5fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dae7fc700 1 -- 172.21.15.87:0/3616550088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8021070 con 0x55fb50346500 2024-06-06T07:58:48.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dae7fc700 1 -- 172.21.15.87:0/3616550088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db8010cd0 con 0x55fb50346500 2024-06-06T07:58:48.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.930+0000 7f6dae7fc700 1 -- 172.21.15.87:0/3616550088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800f6b0 con 0x55fb50346500 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/3616550088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 msgr2=0x55fb503468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6db800ebd0 tx=0x7f6db8011000 comp rx=0 tx=0).stop 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/3616550088 shutdown_connections 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50341070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb503468e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/3616550088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50340710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/3616550088 >> 172.21.15.87:0/3616550088 conn(0x55fb5026d180 msgr2=0x55fb5033ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/3616550088 shutdown_connections 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/3616550088 wait complete. 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 -- start start 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.931+0000 7f6dd8f4bb80 1 Processor -- start 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50340c50 0x55fb5034cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 0x55fb5034a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fb502320c0 con 0x55fb50346500 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb5028d910 con 0x55fb50340c50 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb5034d800 con 0x55fb50346e20 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 10 client.0 _ll_register_callbacks cb 0x55fb502276b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:48.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53774/0 (socket says 172.21.15.87:53774) 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 -- 172.21.15.87:0/2552238180 learned_addr learned my addr 172.21.15.87:0/2552238180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50340c50 0x55fb5034cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dccc65700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 0x55fb5034a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 -- 172.21.15.87:0/2552238180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 msgr2=0x55fb5034a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 0x55fb5034a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 -- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50340c50 msgr2=0x55fb5034cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50340c50 0x55fb5034cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 -- 172.21.15.87:0/2552238180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb50257850 con 0x55fb50346500 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dccc65700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 0x55fb5034a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009990 tx=0x7f6dbc0228f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12511fc7ceb6dc41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6da77fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0b5060 con 0x55fb50346500 2024-06-06T07:58:48.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f6dbc0a30b0 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.932+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db8010cd0 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6dbc0ba030 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 10 client.17148 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb50357f20 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db8010cd0 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6dbc0a5180 con 0x55fb50346500 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 10 client.17148 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 10 client.17148 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 20 client.17148 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb5035c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 4 client.17148 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5029ee50 con 0x55fb5035a4e0 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.933+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb5035c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.934+0000 7f6da77fe700 10 client.17148 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.934+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb5035c900 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ce93748f2df264 server_cookie=f7fe6c1e60e2acd5 in_seq=0 out_seq=0 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6da77fe700 1 -- 172.21.15.87:0/2552238180 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb5029ee50 con 0x55fb5035a4e0 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6da77fe700 10 client.17148 handle_command_reply: tid=0 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 client.17148 shutdown 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 msgr2=0x55fb50349e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009990 tx=0x7f6dbc0228f0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 shutdown_connections 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb5035c900 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50340c50 0x55fb5034cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346e20 0x55fb5034a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6da77fe700 0 client.17148 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/2552238180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346500 0x55fb50349e80 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 >> 172.21.15.87:0/2552238180 conn(0x55fb5026d180 msgr2=0x55fb5033abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 shutdown_connections 2024-06-06T07:58:48.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/2552238180 wait complete. 2024-06-06T07:58:48.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.935+0000 7f6dd8f4bb80 20 client.17148 trim_cache size 0 max 16384 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 Processor -- start 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 -- start start 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50348e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346500 0x55fb5033fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81b3200 con 0x55fb50346e20 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81b0a90 con 0x55fb50346500 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81ae900 con 0x55fb50340c50 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53778/0 (socket says 172.21.15.87:53778) 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 -- 172.21.15.87:0/1671087970 learned_addr learned my addr 172.21.15.87:0/1671087970 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:48.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50348e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 -- 172.21.15.87:0/1671087970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 msgr2=0x55fb50348e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50348e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 -- 172.21.15.87:0/1671087970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346500 msgr2=0x55fb5033fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346500 0x55fb5033fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dccc65700 1 -- 172.21.15.87:0/1671087970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5034a740 con 0x55fb50346e20 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.955+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50348e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dccc65700 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0008710 tx=0x7f6dc00078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb055749ab707bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dadffb700 1 -- 172.21.15.87:0/1671087970 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc001a020 con 0x55fb50346e20 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dadffb700 1 -- 172.21.15.87:0/1671087970 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0016ae0 con 0x55fb50346e20 2024-06-06T07:58:48.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dadffb700 1 -- 172.21.15.87:0/1671087970 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0015bb0 con 0x55fb50346e20 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/1671087970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 msgr2=0x55fb50340350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0008710 tx=0x7f6dc00078d0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/1671087970 shutdown_connections 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50346e20 0x55fb50340350 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50340c50 0x55fb50348e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/1671087970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346500 0x55fb5033fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/1671087970 >> 172.21.15.87:0/1671087970 conn(0x55fb5026d180 msgr2=0x55fb5033f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/1671087970 shutdown_connections 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.956+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/1671087970 wait complete. 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 -- start start 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 Processor -- start 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 0x55fb5034fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50350000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81aef50 con 0x55fb50340c50 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81b03c0 con 0x55fb50346e20 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81af620 con 0x55fb50346500 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dd8f4bb80 10 client.0 _ll_register_callbacks cb 0x55fb502276b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53788/0 (socket says 172.21.15.87:53788) 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.957+0000 7f6dc7fff700 1 -- 172.21.15.87:0/879651210 learned_addr learned my addr 172.21.15.87:0/879651210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 0x55fb5034fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dccc65700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50350000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 -- 172.21.15.87:0/879651210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 msgr2=0x55fb5034fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 0x55fb5034fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 -- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 msgr2=0x55fb50350000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50350000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 -- 172.21.15.87:0/879651210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb503506b0 con 0x55fb50340c50 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 0x55fb5034fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dccc65700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50350000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dc7fff700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6db801dee0 tx=0x7f6db8010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8c909e9adfcb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6da77fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8014040 con 0x55fb50340c50 2024-06-06T07:58:48.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f6db8005250 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc0016ae0 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6db8040050 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dd8f4bb80 10 client.17154 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb503c5fe0 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.958+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0016ae0 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6db801d450 con 0x55fb50340c50 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 10 client.17154 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 10 client.17154 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 20 client.17154 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb503c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 4 client.17154 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "45a5c96b", "release": true, "await": true}] 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "45a5c96b", "release": true, "await": true}) -- 0x55fb502993d0 con 0x55fb5035a4e0 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb503c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:48.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6dc77fe700 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb503c8510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46db21431d5e35a5 server_cookie=b070883c1640d784 in_seq=0 out_seq=0 2024-06-06T07:58:48.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.959+0000 7f6da77fe700 10 client.17154 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6da77fe700 1 -- 172.21.15.87:0/879651210 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb502993d0 con 0x55fb5035a4e0 2024-06-06T07:58:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6da77fe700 10 client.17154 handle_command_reply: tid=0 2024-06-06T07:58:48.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 client.17154 shutdown 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 msgr2=0x55fb50353150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6db801dee0 tx=0x7f6db8010b80 comp rx=0 tx=0).stop 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 shutdown_connections 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fb5035a4e0 0x55fb503c8510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb50346e20 0x55fb50350000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6da77fe700 0 client.17154 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fb50340c50 0x55fb50353150 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 --2- 172.21.15.87:0/879651210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb50346500 0x55fb5034fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 >> 172.21.15.87:0/879651210 conn(0x55fb5026d180 msgr2=0x55fb50349fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 shutdown_connections 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.978+0000 7f6dd8f4bb80 1 -- 172.21.15.87:0/879651210 wait complete. 2024-06-06T07:58:48.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dd8f4bb80 20 client.17154 trim_cache size 0 max 16384 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 36, 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 32, 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "45a5c96b": { 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 32, 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:58:48.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:48.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:48.982 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:58:48.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 msgr2=0x7f6db00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 0x7f6db00a5250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6db8005fd0 tx=0x7f6db8005db0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 msgr2=0x7f6dc8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc003680 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 shutdown_connections 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc81085d0 0x7f6dc8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6db00a2d90 0x7f6db00a5250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dc8102530 0x7f6dc8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 --2- 172.21.15.87:0/3181544630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6dc8102ef0 0x7f6dc8074cd0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 >> 172.21.15.87:0/3181544630 conn(0x7f6dc8079640 msgr2=0x7f6dc81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 shutdown_connections 2024-06-06T07:58:48.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:48.979+0000 7f6dce6c8700 1 -- 172.21.15.87:0/3181544630 wait complete. 2024-06-06T07:58:49.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 613659 6.43 MB/sec execute 413 sec latency 29194.396 ms 2024-06-06T07:58:49.727 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '45a5c96b', total seconds elapsed: 36.2 2024-06-06T07:58:49.727 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-06T07:58:49.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:58:49.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:49 smithi087 ceph-mon[32276]: pgmap v337: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-06-06T07:58:49.996 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:50.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:49 smithi195 ceph-mon[37040]: pgmap v337: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-06-06T07:58:50.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:49 smithi181 ceph-mon[36790]: pgmap v337: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-06-06T07:58:50.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 615360 6.43 MB/sec execute 414 sec latency 21.891 ms 2024-06-06T07:58:51.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 617196 6.44 MB/sec execute 415 sec latency 87.671 ms 2024-06-06T07:58:51.704 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:51 smithi087 ceph-mon[32276]: pgmap v338: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-06-06T07:58:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81f93b3700 1 -- 172.21.15.87:0/1349855491 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4004230 con 0x7f81f41099d0 2024-06-06T07:58:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 -- 172.21.15.87:0/1349855491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 msgr2=0x7f81f4109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 --2- 172.21.15.87:0/1349855491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4109e30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009990 tx=0x7f81e4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 -- 172.21.15.87:0/1349855491 shutdown_connections 2024-06-06T07:58:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 --2- 172.21.15.87:0/1349855491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4117520 0x7f81f4119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 --2- 172.21.15.87:0/1349855491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f410cbb0 0x7f81f4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 --2- 172.21.15.87:0/1349855491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4109e30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.764+0000 7f81fc619700 1 -- 172.21.15.87:0/1349855491 >> 172.21.15.87:0/1349855491 conn(0x7f81f406d600 msgr2=0x7f81f406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.765+0000 7f81fc619700 1 -- 172.21.15.87:0/1349855491 shutdown_connections 2024-06-06T07:58:51.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.765+0000 7f81fc619700 1 -- 172.21.15.87:0/1349855491 wait complete. 2024-06-06T07:58:51.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 Processor -- start 2024-06-06T07:58:51.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 -- start start 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f410cbb0 0x7f81f4113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 0x7f81f4114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81f411d220 con 0x7f81f41099d0 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81f411d0a0 con 0x7f81f410cbb0 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fc619700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81f411d3a0 con 0x7f81f4117520 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fa3b5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fa3b5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53804/0 (socket says 172.21.15.87:53804) 2024-06-06T07:58:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.766+0000 7f81fa3b5700 1 -- 172.21.15.87:0/1321379478 learned_addr learned my addr 172.21.15.87:0/1321379478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81f9bb4700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f410cbb0 0x7f81f4113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fabb6700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 0x7f81f4114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fa3b5700 1 -- 172.21.15.87:0/1321379478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 msgr2=0x7f81f4114320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fa3b5700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 0x7f81f4114320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fa3b5700 1 -- 172.21.15.87:0/1321379478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f410cbb0 msgr2=0x7f81f4113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fa3b5700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f410cbb0 0x7f81f4113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fa3b5700 1 -- 172.21.15.87:0/1321379478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f4114b90 con 0x7f81f41099d0 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.767+0000 7f81fabb6700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 0x7f81f4114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.768+0000 7f81fa3b5700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f81f000e9a0 tx=0x7f81f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.769+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f000cbd0 con 0x7f81f41099d0 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.769+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f000cd70 con 0x7f81f41099d0 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.769+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0018750 con 0x7f81f41099d0 2024-06-06T07:58:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.769+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f41b3fc0 con 0x7f81f41099d0 2024-06-06T07:58:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.769+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41b43c0 con 0x7f81f41099d0 2024-06-06T07:58:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.770+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f406eb60 con 0x7f81f41099d0 2024-06-06T07:58:51.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.774+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81f0010430 con 0x7f81f41099d0 2024-06-06T07:58:51.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.776+0000 7f81eb7fe700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 0x7f81e00a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:51.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.776+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f81f00cfec0 con 0x7f81f41099d0 2024-06-06T07:58:51.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.777+0000 7f81f9bb4700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 0x7f81e00a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:51.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.778+0000 7f81f9bb4700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 0x7f81e00a53a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004000 tx=0x7f81e4005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:51.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:51.780+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f81f009c6e0 con 0x7f81f41099d0 2024-06-06T07:58:52.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:51 smithi195 ceph-mon[37040]: pgmap v338: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-06-06T07:58:52.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.054+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81f4069180 con 0x7f81f41099d0 2024-06-06T07:58:52.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.055+0000 7f81eb7fe700 1 -- 172.21.15.87:0/1321379478 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11877 (secure 0 0 0) 0x7f81f009bfe0 con 0x7f81f41099d0 2024-06-06T07:58:52.055 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:58:52.055 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:57:47.279422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26299":{"gid":26299,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1306245437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1306245437},{"type":"v1","addr":"172.21.15.195:6837","nonce":1306245437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:58:52.055 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:51 smithi181 ceph-mon[36790]: pgmap v338: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 msgr2=0x7f81e00a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 0x7f81e00a53a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004000 tx=0x7f81e4005960 comp rx=0 tx=0).stop 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 msgr2=0x7f81f4116ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f81f000e9a0 tx=0x7f81f000ecb0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 shutdown_connections 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81e00a2ee0 0x7f81e00a53a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81f4117520 0x7f81f4114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81f41099d0 0x7f81f4116ce0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 --2- 172.21.15.87:0/1321379478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f410cbb0 0x7f81f4113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 >> 172.21.15.87:0/1321379478 conn(0x7f81f406d600 msgr2=0x7f81f4117900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 shutdown_connections 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:52.058+0000 7f81fc619700 1 -- 172.21.15.87:0/1321379478 wait complete. 2024-06-06T07:58:52.059 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 36 2024-06-06T07:58:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/1150095959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc88103550 msgr2=0x7fdc8810ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/1150095959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc88103550 0x7fdc8810ba30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009900 tx=0x7fdc7c02a480 comp rx=0 tx=0).stop 2024-06-06T07:58:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/1150095959 shutdown_connections 2024-06-06T07:58:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/1150095959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc8810bf70 0x7fdc8810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/1150095959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc88103550 0x7fdc8810ba30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/1150095959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/1150095959 >> 172.21.15.87:0/1150095959 conn(0x7fdc88077a50 msgr2=0x7fdc88077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/1150095959 shutdown_connections 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.417+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/1150095959 wait complete. 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 Processor -- start 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 -- start start 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc8810bf70 0x7fdc8819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111d70 con 0x7fdc88103550 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111bf0 con 0x7fdc88102c30 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc8dfa9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111ef0 con 0x7fdc8810bf70 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.418+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53832/0 (socket says 172.21.15.87:53832) 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc86ffd640 1 -- 172.21.15.87:0/3983480873 learned_addr learned my addr 172.21.15.87:0/3983480873 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc87fff640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc8810bf70 0x7fdc8819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 -- 172.21.15.87:0/3983480873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc8810bf70 msgr2=0x7fdc8819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc8810bf70 0x7fdc8819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 -- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 msgr2=0x7fdc88198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 -- 172.21.15.87:0/3983480873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819d1b0 con 0x7fdc88102c30 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88198210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e9f0 tx=0x7fdc7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc84ff9640 1 -- 172.21.15.87:0/3983480873 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800ccc0 con 0x7fdc88102c30 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819d4a0 con 0x7fdc88102c30 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc84ff9640 1 -- 172.21.15.87:0/3983480873 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78005020 con 0x7fdc88102c30 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc84ff9640 1 -- 172.21.15.87:0/3983480873 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780049e0 con 0x7fdc88102c30 2024-06-06T07:58:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.419+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a4d30 con 0x7fdc88102c30 2024-06-06T07:58:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.421+0000 7fdc84ff9640 1 -- 172.21.15.87:0/3983480873 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdc78005560 con 0x7fdc88102c30 2024-06-06T07:58:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 Processor -- start 2024-06-06T07:58:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 -- start start 2024-06-06T07:58:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc84ff9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 0x7fdc700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01bf7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc87fff640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 0x7fdc700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc84ff9640 1 -- 172.21.15.87:0/3983480873 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fdc780cf870 con 0x7fdc88102c30 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55be01b4e4f0 con 0x55be01bea170 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55be01aec870 con 0x55be01bf5350 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc90d98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55be01b4a4b0 con 0x55be01beaa90 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53834/0 (socket says 172.21.15.87:53834) 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc877fe640 1 -- 172.21.15.87:0/2939655192 learned_addr learned my addr 172.21.15.87:0/2939655192 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.422+0000 7fdc87fff640 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01bf7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 -- 172.21.15.87:0/2939655192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 msgr2=0x55be01bf4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 -- 172.21.15.87:0/2939655192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 msgr2=0x55be01bf7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01bf7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 -- 172.21.15.87:0/2939655192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be01b089b0 con 0x55be01bea170 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc87fff640 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01bf7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc87fff640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 0x7fdc700a5260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009fd0 tx=0x7fdc80012a90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc877fe640 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdc780a40b0 tx=0x7fdc7809c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945f71873ab18b68 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc6dffb640 1 -- 172.21.15.87:0/2939655192 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780b4070 con 0x55be01bea170 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc6dffb640 1 -- 172.21.15.87:0/2939655192 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc780a45e0 con 0x55be01bea170 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.423+0000 7fdc6dffb640 1 -- 172.21.15.87:0/2939655192 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801c040 con 0x55be01bea170 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- 172.21.15.87:0/2939655192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 msgr2=0x55be01bea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdc780a40b0 tx=0x7fdc7809c070 comp rx=0 tx=0).stop 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- 172.21.15.87:0/2939655192 shutdown_connections 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01bf7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2939655192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bea550 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- 172.21.15.87:0/2939655192 >> 172.21.15.87:0/2939655192 conn(0x55be01be6680 msgr2=0x55be01be6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- 172.21.15.87:0/2939655192 shutdown_connections 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- 172.21.15.87:0/2939655192 wait complete. 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 -- start start 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.424+0000 7fdc90d98740 1 Processor -- start 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bea170 0x55be01bf2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55be01bf7f80 con 0x55be01bf5350 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55be01ad7ae0 con 0x55be01beaa90 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55be01bf8100 con 0x55be01bea170 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 10 client.0 _ll_register_callbacks cb 0x55be01b109b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bea170 0x55be01bf2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53842/0 (socket says 172.21.15.87:53842) 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 -- 172.21.15.87:0/2537904166 learned_addr learned my addr 172.21.15.87:0/2537904166 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 -- 172.21.15.87:0/2537904166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bea170 msgr2=0x55be01bf2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bea170 0x55be01bf2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 -- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 msgr2=0x55be01bf2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 -- 172.21.15.87:0/2537904166 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be01862210 con 0x55be01bf5350 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc87fff640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000eeb0 tx=0x7fdc80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861df3259816ce74 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002a680 con 0x55be01bf5350 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fdc8002a820 con 0x55be01bf5350 2024-06-06T07:58:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.425+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc780a45e0 con 0x55be01bf5350 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdc80033050 con 0x55be01bf5350 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 10 client.17172 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be0192e620 con 0x55be01bf5350 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc780a45e0 con 0x55be01bf5350 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fdc8001e070 con 0x55be01bf5350 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 10 client.17172 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 20 client.17172 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c05530 0x55be01c07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 4 client.17172 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be01b61f10 con 0x55be01c05530 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.426+0000 7fdc877fe640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c05530 0x55be01c07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc66ffd640 10 client.17172 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc877fe640 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c05530 0x55be01c07940 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c129850a17db2c server_cookie=56001393660b81f3 in_seq=0 out_seq=0 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc66ffd640 1 -- 172.21.15.87:0/2537904166 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be01b61f10 con 0x55be01c05530 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc66ffd640 10 client.17172 handle_command_reply: tid=0 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 client.17172 shutdown 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 msgr2=0x55be01bf2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000eeb0 tx=0x7fdc80004ab0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 shutdown_connections 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c05530 0x55be01c07940 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bf5350 0x55be01bf2dc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc66ffd640 0 client.17172 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 --2- 172.21.15.87:0/2537904166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bea170 0x55be01bf2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 >> 172.21.15.87:0/2537904166 conn(0x55be01be6680 msgr2=0x55be01bec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.427+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 shutdown_connections 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.428+0000 7fdc90d98740 1 -- 172.21.15.87:0/2537904166 wait complete. 2024-06-06T07:58:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.428+0000 7fdc90d98740 20 client.17172 trim_cache size 0 max 16384 2024-06-06T07:58:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 Processor -- start 2024-06-06T07:58:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 -- start start 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bf5350 0x55be01c0d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b1840 con 0x55be01bea170 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881af110 con 0x55be01beaa90 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc90d98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881acf80 con 0x55be01bf5350 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53850/0 (socket says 172.21.15.87:53850) 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 -- 172.21.15.87:0/1110066863 learned_addr learned my addr 172.21.15.87:0/1110066863 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc87fff640 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bf5350 0x55be01c0d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 -- 172.21.15.87:0/1110066863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bf5350 msgr2=0x55be01c0d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bf5350 0x55be01c0d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 -- 172.21.15.87:0/1110066863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 msgr2=0x55be01bf2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.445+0000 7fdc877fe640 1 -- 172.21.15.87:0/1110066863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be01be9f30 con 0x55be01bea170 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc877fe640 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdc78099c20 tx=0x7fdc780096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2950a5f4390e21 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc6d7fa640 1 -- 172.21.15.87:0/1110066863 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801c040 con 0x55be01bea170 2024-06-06T07:58:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc6d7fa640 1 -- 172.21.15.87:0/1110066863 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78097fe0 con 0x55be01bea170 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc6d7fa640 1 -- 172.21.15.87:0/1110066863 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780b4040 con 0x55be01bea170 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc90d98740 1 -- 172.21.15.87:0/1110066863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 msgr2=0x55be01bf2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.446+0000 7fdc90d98740 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdc78099c20 tx=0x7fdc780096e0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- 172.21.15.87:0/1110066863 shutdown_connections 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01bf5350 0x55be01c0d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01beaa90 0x55be01bf2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- 172.21.15.87:0/1110066863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf2a10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- 172.21.15.87:0/1110066863 >> 172.21.15.87:0/1110066863 conn(0x55be01be6680 msgr2=0x55be01bf2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- 172.21.15.87:0/1110066863 shutdown_connections 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- 172.21.15.87:0/1110066863 wait complete. 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- start start 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 Processor -- start 2024-06-06T07:58:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01c6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ad5d0 con 0x55be01bea170 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881aea40 con 0x55be01bf5350 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881adca0 con 0x55be01beaa90 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc90d98740 10 client.0 _ll_register_callbacks cb 0x7fdc8000e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.447+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53852/0 (socket says 172.21.15.87:53852) 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 -- 172.21.15.87:0/3977731612 learned_addr learned my addr 172.21.15.87:0/3977731612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc87fff640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01c6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 -- 172.21.15.87:0/3977731612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 msgr2=0x55be01bf8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 -- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 msgr2=0x55be01c6e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01c6e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 -- 172.21.15.87:0/3977731612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be01c6e750 con 0x55be01bea170 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc87fff640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01c6e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc877fe640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdc78097dd0 tx=0x7fdc7809bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b6ca941c8196d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801c030 con 0x55be01bea170 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fdc780860e0 con 0x55be01bea170 2024-06-06T07:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc78097fe0 con 0x55be01bea170 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdc780b4040 con 0x55be01bea170 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc90d98740 10 client.17178 fetch_fsmap learned FSMap version 36 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be01c6f990 con 0x55be01bea170 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.448+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78097fe0 con 0x55be01bea170 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fdc780ce040 con 0x55be01bea170 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 10 client.17178 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 20 client.17178 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c6fc10 0x55be01c72000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 4 client.17178 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be01b5bc20 con 0x55be01c6fc10 2024-06-06T07:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c6fc10 0x55be01c72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc86ffd640 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c6fc10 0x55be01c72000 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd847392cf817ec server_cookie=81e4b004922f7f95 in_seq=0 out_seq=0 2024-06-06T07:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.449+0000 7fdc66ffd640 10 client.17178 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.578+0000 7fdc66ffd640 1 -- 172.21.15.87:0/3977731612 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be01b5bc20 con 0x55be01c6fc10 2024-06-06T07:58:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.578+0000 7fdc66ffd640 10 client.17178 handle_command_reply: tid=0 2024-06-06T07:58:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.578+0000 7fdc90d98740 1 client.17178 shutdown 2024-06-06T07:58:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 msgr2=0x55be01bf8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdc78097dd0 tx=0x7fdc7809bbd0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 shutdown_connections 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55be01c6fc10 0x55be01c72000 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be01bf5350 0x55be01c6e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55be01beaa90 0x55be01bf8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 --2- 172.21.15.87:0/3977731612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be01bea170 0x55be01bf8720 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 >> 172.21.15.87:0/3977731612 conn(0x55be01be6680 msgr2=0x55be01bf2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc66ffd640 0 client.17178 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 shutdown_connections 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 1 -- 172.21.15.87:0/3977731612 wait complete. 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc90d98740 20 client.17178 trim_cache size 0 max 16384 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:58:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:58:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 msgr2=0x7fdc700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 0x7fdc700a5260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009fd0 tx=0x7fdc80012a90 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 msgr2=0x7fdc88198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88198210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e9f0 tx=0x7fdc7800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 shutdown_connections 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdc700a2da0 0x7fdc700a5260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc8810bf70 0x7fdc8819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc88103550 0x7fdc88198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 --2- 172.21.15.87:0/3983480873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88102c30 0x7fdc88198210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.579+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 >> 172.21.15.87:0/3983480873 conn(0x7fdc88077a50 msgr2=0x7fdc880fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.580+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 shutdown_connections 2024-06-06T07:58:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:58:52.580+0000 7fdc8dfa9640 1 -- 172.21.15.87:0/3983480873 wait complete. 2024-06-06T07:58:52.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 618706 6.44 MB/sec execute 416 sec latency 29.784 ms 2024-06-06T07:58:52.750 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:52 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1321379478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:52.783 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-06T07:58:52.783 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-06T07:58:53.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:52 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1321379478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:53.010 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:58:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T07:58:53.010 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:58:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T07:58:53.010 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:58:52 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/2259855518 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556436eb9a40 con 0x556436e8d000 2024-06-06T07:58:53.010 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:58:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T07:58:53.039 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:58:53.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:52 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1321379478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:58:53.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 620879 6.46 MB/sec execute 417 sec latency 18.115 ms 2024-06-06T07:58:53.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:53 smithi087 ceph-mon[32276]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-06T07:58:53.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:53 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T07:58:54.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:53 smithi195 ceph-mon[37040]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-06T07:58:54.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:53 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T07:58:54.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:53 smithi181 ceph-mon[36790]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-06-06T07:58:54.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:53 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T07:58:54.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 622517 6.46 MB/sec execute 418 sec latency 24.442 ms 2024-06-06T07:58:54.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 -- 172.21.15.87:0/2072371790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a41066c0 msgr2=0x7fb3a410e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/2072371790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a41066c0 0x7fb3a410e290 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00b0d0 tx=0x7fb39c00b3e0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 -- 172.21.15.87:0/2072371790 shutdown_connections 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/2072371790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a41066c0 0x7fb3a410e290 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/2072371790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/2072371790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a4101fa0 0x7fb3a4105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 -- 172.21.15.87:0/2072371790 >> 172.21.15.87:0/2072371790 conn(0x7fb3a4078330 msgr2=0x7fb3a4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 -- 172.21.15.87:0/2072371790 shutdown_connections 2024-06-06T07:58:54.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.939+0000 7fb3a98b0700 1 -- 172.21.15.87:0/2072371790 wait complete. 2024-06-06T07:58:54.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:54 smithi087 ceph-mon[32276]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:58:54.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:54 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/1032843361,v1:172.21.15.195:6837/1032843361] up:boot 2024-06-06T07:58:54.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:54 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:58:54.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:54 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:58:54.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:54 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:58:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.940+0000 7fb3a98b0700 1 Processor -- start 2024-06-06T07:58:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.940+0000 7fb3a98b0700 1 -- start start 2024-06-06T07:58:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.940+0000 7fb3a98b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a98b0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a4101fa0 0x7fb3a419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a98b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 0x7fb3a41a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a98b0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41116d0 con 0x7fb3a4101fa0 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a98b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4111550 con 0x7fb3a41066c0 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a98b0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4111850 con 0x7fb3a41015f0 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:41712/0 (socket says 172.21.15.87:41712) 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3772630362 learned_addr learned my addr 172.21.15.87:0/3772630362 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a37fe700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 0x7fb3a41a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3772630362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 msgr2=0x7fb3a41a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 0x7fb3a41a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a4101fa0 msgr2=0x7fb3a419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a4101fa0 0x7fb3a419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3772630362 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a41a4180 con 0x7fb3a41015f0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.941+0000 7fb3a37fe700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 0x7fb3a41a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e9c0 tx=0x7fb39400ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a88ae700 1 -- 172.21.15.87:0/3772630362 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39400cbd0 con 0x7fb3a41015f0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a88ae700 1 -- 172.21.15.87:0/3772630362 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39400cd70 con 0x7fb3a41015f0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a88ae700 1 -- 172.21.15.87:0/3772630362 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394010430 con 0x7fb3a41015f0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a41a4470 con 0x7fb3a41015f0 2024-06-06T07:58:54.945 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.942+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a41abcf0 con 0x7fb3a41015f0 2024-06-06T07:58:54.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.945+0000 7fb3a88ae700 1 -- 172.21.15.87:0/3772630362 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb394004750 con 0x7fb3a41015f0 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.946+0000 7fb3a88ae700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 0x7fb38c0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.946+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 0x7fb38c0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.946+0000 7fb3a88ae700 1 -- 172.21.15.87:0/3772630362 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7fb3940cfca0 con 0x7fb3a41015f0 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.946+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 0x7fb38c0a52f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4107a20 tx=0x7fb398009620 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 Processor -- start 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 -- start start 2024-06-06T07:58:54.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe158a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563fe147c0f0 con 0x563fe1589ce0 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563fe14da9e0 con 0x563fe158feb0 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563fe14425f0 con 0x563fe158f590 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:41724/0 (socket says 172.21.15.87:41724) 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/34968437 learned_addr learned my addr 172.21.15.87:0/34968437 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a37fe700 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe158a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/34968437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 msgr2=0x563fe15897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/34968437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 msgr2=0x563fe158a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe158a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/34968437 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe1429a60 con 0x563fe158f590 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.947+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb394098250 tx=0x7fb39409b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c54403c3d3997bd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb389ffb700 1 -- 172.21.15.87:0/34968437 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39409bdf0 con 0x563fe158f590 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb389ffb700 1 -- 172.21.15.87:0/34968437 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39409f070 con 0x563fe158f590 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb389ffb700 1 -- 172.21.15.87:0/34968437 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39409c2b0 con 0x563fe158f590 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 -- 172.21.15.87:0/34968437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 msgr2=0x563fe158f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb394098250 tx=0x7fb39409b9b0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 -- 172.21.15.87:0/34968437 shutdown_connections 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe158a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe158f970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/34968437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 -- 172.21.15.87:0/34968437 >> 172.21.15.87:0/34968437 conn(0x563fe14b6210 msgr2=0x563fe1584a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 -- 172.21.15.87:0/34968437 shutdown_connections 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.948+0000 7fb3b4133b80 1 -- 172.21.15.87:0/34968437 wait complete. 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 -- start start 2024-06-06T07:58:54.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 Processor -- start 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe1593020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1593560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563fe147b150 con 0x563fe1589ce0 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563fe14d69a0 con 0x563fe158f590 2024-06-06T07:58:54.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563fe1596890 con 0x563fe158feb0 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3b4133b80 10 client.0 _ll_register_callbacks cb 0x563fe1470740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.949+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53886/0 (socket says 172.21.15.87:53886) 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe1593020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3376724313 learned_addr learned my addr 172.21.15.87:0/3376724313 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a37fe700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1593560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3376724313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 msgr2=0x563fe1593560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1593560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 msgr2=0x563fe1593020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe1593020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/3376724313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe14a08e0 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe1593020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c5b0 tx=0x7fb394004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84022cd145cc709 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940a01a0 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7fb39409c260 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39409f070 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb39409bdf0 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3b4133b80 10 client.17193 fetch_fsmap learned FSMap version 38 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563fe15a10f0 con 0x563fe1589ce0 2024-06-06T07:58:54.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.950+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39409f070 con 0x563fe1589ce0 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb394062070 con 0x563fe1589ce0 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 10 client.17193 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 20 client.17193 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe15a35c0 0x563fe15a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 4 client.17193 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe14e7ee0 con 0x563fe15a35c0 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe15a35c0 0x563fe15a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe15a35c0 0x563fe15a59b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a7d98a70d9279d server_cookie=78fc7f262e0f38fa in_seq=0 out_seq=0 2024-06-06T07:58:54.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.951+0000 7fb382ffd700 10 client.17193 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb382ffd700 1 -- 172.21.15.87:0/3376724313 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fe14e7ee0 con 0x563fe15a35c0 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb382ffd700 10 client.17193 handle_command_reply: tid=0 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 client.17193 shutdown 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 msgr2=0x563fe1595d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c5b0 tx=0x7fb394004c50 comp rx=0 tx=0).stop 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 shutdown_connections 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe15a35c0 0x563fe15a59b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1593560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe1595d20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/3376724313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe1593020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 >> 172.21.15.87:0/3376724313 conn(0x563fe14b6210 msgr2=0x563fe15833b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:54.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb382ffd700 0 client.17193 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:54.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 shutdown_connections 2024-06-06T07:58:54.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 1 -- 172.21.15.87:0/3376724313 wait complete. 2024-06-06T07:58:54.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.953+0000 7fb3b4133b80 20 client.17193 trim_cache size 0 max 16384 2024-06-06T07:58:54.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.974+0000 7fb3b4133b80 1 Processor -- start 2024-06-06T07:58:54.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.974+0000 7fb3b4133b80 1 -- start start 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe158d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1590b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b74c0 con 0x563fe1589ce0 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b4d50 con 0x563fe158f590 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b2ba0 con 0x563fe158feb0 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1590b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53894/0 (socket says 172.21.15.87:53894) 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2169268482 learned_addr learned my addr 172.21.15.87:0/2169268482 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe158d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2169268482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 msgr2=0x563fe1590b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1590b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2169268482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 msgr2=0x563fe158d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe158d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2169268482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe157c530 con 0x563fe1589ce0 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb39409a0a0 tx=0x7fb394098110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0cc864703dba00 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3897fa700 1 -- 172.21.15.87:0/2169268482 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940a4a60 con 0x563fe1589ce0 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3897fa700 1 -- 172.21.15.87:0/2169268482 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39409f070 con 0x563fe1589ce0 2024-06-06T07:58:54.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.975+0000 7fb3897fa700 1 -- 172.21.15.87:0/2169268482 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940a45d0 con 0x563fe1589ce0 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2169268482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 msgr2=0x563fe15897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb39409a0a0 tx=0x7fb394098110 comp rx=0 tx=0).stop 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2169268482 shutdown_connections 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158feb0 0x563fe1590b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15897a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2169268482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158f590 0x563fe158d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2169268482 >> 172.21.15.87:0/2169268482 conn(0x563fe14b6210 msgr2=0x563fe158e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:54.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2169268482 shutdown_connections 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.976+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2169268482 wait complete. 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 -- start start 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 Processor -- start 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe15a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b3210 con 0x563fe1589ce0 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b4680 con 0x563fe158feb0 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41b38e0 con 0x563fe158f590 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3b4133b80 10 client.0 _ll_register_callbacks cb 0x563fe1470740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53906/0 (socket says 172.21.15.87:53906) 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2008027796 learned_addr learned my addr 172.21.15.87:0/2008027796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3a37fe700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.977+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe15a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2008027796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 msgr2=0x563fe15a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe15a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 msgr2=0x563fe15a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 -- 172.21.15.87:0/2008027796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe15a6b80 con 0x563fe1589ce0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3a2ffd700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb394098250 tx=0x7fb394005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995ceaf9e11c431f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39409b930 con 0x563fe1589ce0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7fb39409b240 con 0x563fe1589ce0 2024-06-06T07:58:54.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39409f070 con 0x563fe1589ce0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3940a9050 con 0x563fe1589ce0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 10 client.17199 fetch_fsmap learned FSMap version 38 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563fe15a6b80 con 0x563fe1589ce0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb39409f070 con 0x563fe1589ce0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb394062040 con 0x563fe1589ce0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 10 client.17199 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 10 client.17199 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 20 client.17199 populate_metadata read hostname 'smithi087' 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe160dff0 0x563fe16103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 4 client.17199 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.978+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x563fe14e2460 con 0x563fe160dff0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.979+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe160dff0 0x563fe16103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.979+0000 7fb382ffd700 10 client.17199 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:54.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:54.979+0000 7fb3a27fc700 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe160dff0 0x563fe16103e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b00029acf5775bf server_cookie=1531517ddcf55fd0 in_seq=0 out_seq=0 2024-06-06T07:58:55.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:54 smithi195 ceph-mon[37040]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:58:55.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:54 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/1032843361,v1:172.21.15.195:6837/1032843361] up:boot 2024-06-06T07:58:55.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:54 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:58:55.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:54 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:58:55.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:54 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:58:55.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:54 smithi181 ceph-mon[36790]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:58:55.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:54 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/1032843361,v1:172.21.15.195:6837/1032843361] up:boot 2024-06-06T07:58:55.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:54 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:58:55.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:54 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:58:55.057 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:54 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:58:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb382ffd700 1 -- 172.21.15.87:0/2008027796 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563fe14e2460 con 0x563fe160dff0 2024-06-06T07:58:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb382ffd700 10 client.17199 handle_command_reply: tid=0 2024-06-06T07:58:55.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 client.17199 shutdown 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 msgr2=0x563fe15a8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb394098250 tx=0x7fb394005b40 comp rx=0 tx=0).stop 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 shutdown_connections 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563fe160dff0 0x563fe16103e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb382ffd700 0 client.17199 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fe158feb0 0x563fe15a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563fe1589ce0 0x563fe15a8bf0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 --2- 172.21.15.87:0/2008027796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fe158f590 0x563fe15a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.071+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 >> 172.21.15.87:0/2008027796 conn(0x563fe14b6210 msgr2=0x563fe158ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 shutdown_connections 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3b4133b80 1 -- 172.21.15.87:0/2008027796 wait complete. 2024-06-06T07:58:55.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3b4133b80 20 client.17199 trim_cache size 0 max 16384 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 38, 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 34, 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "1494725a": { 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 34, 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:55.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:55.074 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 msgr2=0x7fb38c0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 0x7fb38c0a52f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4107a20 tx=0x7fb398009620 comp rx=0 tx=0).stop 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 msgr2=0x7fb3a419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e9c0 tx=0x7fb39400ed80 comp rx=0 tx=0).stop 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 shutdown_connections 2024-06-06T07:58:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3a41066c0 0x7fb3a41a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb38c0a2e30 0x7fb38c0a52f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3a41015f0 0x7fb3a419f580 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 --2- 172.21.15.87:0/3772630362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a4101fa0 0x7fb3a419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 >> 172.21.15.87:0/3772630362 conn(0x7fb3a4078330 msgr2=0x7fb3a406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 shutdown_connections 2024-06-06T07:58:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:58:55.072+0000 7fb3a98b0700 1 -- 172.21.15.87:0/3772630362 wait complete. 2024-06-06T07:58:55.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.46 MB/sec execute 419 sec latency 601.101 ms 2024-06-06T07:58:55.758 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1494725a', quiesce took 6.0 seconds. Will release after: 24.0 2024-06-06T07:58:56.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:55 smithi195 ceph-mon[37040]: pgmap v341: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 114 op/s 2024-06-06T07:58:56.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:55 smithi181 ceph-mon[36790]: pgmap v341: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 114 op/s 2024-06-06T07:58:56.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:55 smithi087 ceph-mon[32276]: pgmap v341: 129 pgs: 129 active+clean; 89 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 114 op/s 2024-06-06T07:58:56.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.44 MB/sec execute 420 sec latency 1601.131 ms 2024-06-06T07:58:57.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.43 MB/sec execute 421 sec latency 2601.168 ms 2024-06-06T07:58:58.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:57 smithi195 ceph-mon[37040]: pgmap v342: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:58:58.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:57 smithi181 ceph-mon[36790]: pgmap v342: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:58:58.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:57 smithi087 ceph-mon[32276]: pgmap v342: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:58:58.682 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.41 MB/sec execute 422 sec latency 3601.202 ms 2024-06-06T07:58:59.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.39 MB/sec execute 423 sec latency 4601.324 ms 2024-06-06T07:59:00.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:58:59 smithi195 ceph-mon[37040]: pgmap v343: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:00.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:58:59 smithi181 ceph-mon[36790]: pgmap v343: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:00.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:58:59 smithi087 ceph-mon[32276]: pgmap v343: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:00.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.38 MB/sec execute 424 sec latency 5601.388 ms 2024-06-06T07:59:01.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.36 MB/sec execute 425 sec latency 6601.496 ms 2024-06-06T07:59:02.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:01 smithi195 ceph-mon[37040]: pgmap v344: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:02.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:01 smithi181 ceph-mon[36790]: pgmap v344: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:02.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:01 smithi087 ceph-mon[32276]: pgmap v344: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:59:02.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.35 MB/sec execute 426 sec latency 7601.609 ms 2024-06-06T07:59:03.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.33 MB/sec execute 427 sec latency 8601.701 ms 2024-06-06T07:59:04.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:03 smithi195 ceph-mon[37040]: pgmap v345: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 756 KiB/s wr, 177 op/s 2024-06-06T07:59:04.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:03 smithi181 ceph-mon[36790]: pgmap v345: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 756 KiB/s wr, 177 op/s 2024-06-06T07:59:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:03 smithi087 ceph-mon[32276]: pgmap v345: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 756 KiB/s wr, 177 op/s 2024-06-06T07:59:04.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.32 MB/sec execute 428 sec latency 9601.784 ms 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- 172.21.15.87:0/1421477866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58401008d0 msgr2=0x7f5840100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 --2- 172.21.15.87:0/1421477866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58401008d0 0x7f5840100d50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f582c009930 tx=0x7f582c02a420 comp rx=0 tx=0).stop 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- 172.21.15.87:0/1421477866 shutdown_connections 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 --2- 172.21.15.87:0/1421477866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5840101290 0x7f584010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 --2- 172.21.15.87:0/1421477866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58401008d0 0x7f5840100d50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 --2- 172.21.15.87:0/1421477866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 0x7f5840106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- 172.21.15.87:0/1421477866 >> 172.21.15.87:0/1421477866 conn(0x7f5840077a20 msgr2=0x7f5840077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- 172.21.15.87:0/1421477866 shutdown_connections 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- 172.21.15.87:0/1421477866 wait complete. 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 Processor -- start 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.677+0000 7f5846bb7640 1 -- start start 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58401008d0 0x7f58401982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 0x7f584019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5840110ca0 con 0x7f5840101290 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5840110b20 con 0x7f58401008d0 2024-06-06T07:59:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5840110e20 con 0x7f5840106890 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53172/0 (socket says 172.21.15.87:53172) 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 -- 172.21.15.87:0/855571098 learned_addr learned my addr 172.21.15.87:0/855571098 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5845154640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 0x7f584019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5844953640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58401008d0 0x7f58401982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 -- 172.21.15.87:0/855571098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 msgr2=0x7f584019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 0x7f584019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 -- 172.21.15.87:0/855571098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58401008d0 msgr2=0x7f58401982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58401008d0 0x7f58401982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 -- 172.21.15.87:0/855571098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584019d2d0 con 0x7f5840101290 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583ffff640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f582c009230 tx=0x7f582c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583dffb640 1 -- 172.21.15.87:0/855571098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c03f030 con 0x7f5840101290 2024-06-06T07:59:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f583dffb640 1 -- 172.21.15.87:0/855571098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c003ad0 con 0x7f5840101290 2024-06-06T07:59:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.678+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584019d560 con 0x7f5840101290 2024-06-06T07:59:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.679+0000 7f583dffb640 1 -- 172.21.15.87:0/855571098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c035070 con 0x7f5840101290 2024-06-06T07:59:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.679+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58401a4e50 con 0x7f5840101290 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.681+0000 7f583dffb640 1 -- 172.21.15.87:0/855571098 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f582c042030 con 0x7f5840101290 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.681+0000 7f58499a5740 1 Processor -- start 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 -- start start 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f583dffb640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 0x7f58300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 0x55db43e71450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 0x55db43e82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55db43dd54f0 con 0x55db43e80760 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db43d73870 con 0x55db43e71990 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f58499a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db43dd14b0 con 0x55db43e71070 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5845154640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 0x7f58300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f583dffb640 1 -- 172.21.15.87:0/855571098 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7f582c0efbc0 con 0x7f5840101290 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5845154640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 0x55db43e82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5845154640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 0x55db43e82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53188/0 (socket says 172.21.15.87:53188) 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5845154640 1 -- 172.21.15.87:0/212737222 learned_addr learned my addr 172.21.15.87:0/212737222 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 0x55db43e71450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f583ffff640 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 -- 172.21.15.87:0/212737222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 msgr2=0x55db43e80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e80220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 -- 172.21.15.87:0/212737222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 msgr2=0x55db43e82b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 0x55db43e82b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5844953640 1 -- 172.21.15.87:0/212737222 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db43d8f9b0 con 0x55db43e71070 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.682+0000 7f5845154640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 0x7f58300a52e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5838012c80 tx=0x7f5838009e80 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f5844953640 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 0x55db43e71450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58340045b0 tx=0x7f583400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c5f170ddc75900 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f582affd640 1 -- 172.21.15.87:0/212737222 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834003bd0 con 0x55db43e71070 2024-06-06T07:59:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f582affd640 1 -- 172.21.15.87:0/212737222 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834015070 con 0x55db43e71070 2024-06-06T07:59:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f582affd640 1 -- 172.21.15.87:0/212737222 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834004d00 con 0x55db43e71070 2024-06-06T07:59:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 -- 172.21.15.87:0/212737222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 msgr2=0x55db43e71450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 0x55db43e71450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58340045b0 tx=0x7f583400bb20 comp rx=0 tx=0).stop 2024-06-06T07:59:05.687 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.31 MB/sec execute 429 sec latency 10601.817 ms 2024-06-06T07:59:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 -- 172.21.15.87:0/212737222 shutdown_connections 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e80760 0x55db43e82b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.683+0000 7f58499a5740 1 --2- 172.21.15.87:0/212737222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71070 0x55db43e71450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- 172.21.15.87:0/212737222 >> 172.21.15.87:0/212737222 conn(0x55db43e6d680 msgr2=0x55db43e6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- 172.21.15.87:0/212737222 shutdown_connections 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- 172.21.15.87:0/212737222 wait complete. 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- start start 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 Processor -- start 2024-06-06T07:59:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71990 0x55db43e7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e80760 0x55db43e7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55db43e83390 con 0x55db43e71070 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db43d5eae0 con 0x55db43e80760 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db43e83510 con 0x55db43e71990 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.684+0000 7f58499a5740 10 client.0 _ll_register_callbacks cb 0x55db43d979b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53198/0 (socket says 172.21.15.87:53198) 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 -- 172.21.15.87:0/3422470044 learned_addr learned my addr 172.21.15.87:0/3422470044 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f583ffff640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71990 0x55db43e7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5845154640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e80760 0x55db43e7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 -- 172.21.15.87:0/3422470044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71990 msgr2=0x55db43e7db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71990 0x55db43e7db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 -- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e80760 msgr2=0x55db43e7e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e80760 0x55db43e7e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 -- 172.21.15.87:0/3422470044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db43ae9210 con 0x55db43e71070 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f5844953640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f583400b5f0 tx=0x7f58340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2ca9aadfbde7c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f581ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834028060 con 0x55db43e71070 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7f5834011640 con 0x55db43e71070 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5834015070 con 0x55db43e71070 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f583402f050 con 0x55db43e71070 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f58499a5740 10 client.17211 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.685+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55db43bb5620 con 0x55db43e71070 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834015070 con 0x55db43e71070 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f5834022070 con 0x55db43e71070 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 10 client.17211 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 20 client.17211 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43e90820 0x55db43e92c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 4 client.17211 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db43de8f10 con 0x55db43e90820 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.686+0000 7f583ffff640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43e90820 0x55db43e92c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f583ffff640 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43e90820 0x55db43e92c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8246569826a7211c server_cookie=dafff8734a245d04 in_seq=0 out_seq=0 2024-06-06T07:59:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f581ffff640 10 client.17211 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f581ffff640 1 -- 172.21.15.87:0/3422470044 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db43de8f10 con 0x55db43e90820 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f581ffff640 10 client.17211 handle_command_reply: tid=0 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f58499a5740 1 client.17211 shutdown 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 msgr2=0x55db43e7d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f583400b5f0 tx=0x7f58340042f0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 shutdown_connections 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.687+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43e90820 0x55db43e92c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e80760 0x55db43e7e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e71990 0x55db43e7db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 --2- 172.21.15.87:0/3422470044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e7d610 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f581ffff640 0 client.17211 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 >> 172.21.15.87:0/3422470044 conn(0x55db43e6d680 msgr2=0x55db43e734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 shutdown_connections 2024-06-06T07:59:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 1 -- 172.21.15.87:0/3422470044 wait complete. 2024-06-06T07:59:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.688+0000 7f58499a5740 20 client.17211 trim_cache size 0 max 16384 2024-06-06T07:59:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 Processor -- start 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 -- start start 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71070 0x55db43e98330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58401af4a0 con 0x55db43e71990 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58401acd70 con 0x55db43e71070 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.703+0000 7f58499a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58401aabc0 con 0x55db43e80760 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53214/0 (socket says 172.21.15.87:53214) 2024-06-06T07:59:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 -- 172.21.15.87:0/1022796381 learned_addr learned my addr 172.21.15.87:0/1022796381 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f5845154640 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 -- 172.21.15.87:0/1022796381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 msgr2=0x55db43e7ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 -- 172.21.15.87:0/1022796381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71070 msgr2=0x55db43e98330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71070 0x55db43e98330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 -- 172.21.15.87:0/1022796381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db43e82ef0 con 0x55db43e71990 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f583ffff640 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f582c0b8220 tx=0x7f582c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52df7162cb9dcb39 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f582a7fc640 1 -- 172.21.15.87:0/1022796381 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c049040 con 0x55db43e71990 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f582a7fc640 1 -- 172.21.15.87:0/1022796381 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c0b9e60 con 0x55db43e71990 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.704+0000 7f582a7fc640 1 -- 172.21.15.87:0/1022796381 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0b8430 con 0x55db43e71990 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- 172.21.15.87:0/1022796381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 msgr2=0x55db43e98870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f582c0b8220 tx=0x7f582c005bd0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- 172.21.15.87:0/1022796381 shutdown_connections 2024-06-06T07:59:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71990 0x55db43e98870 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 --2- 172.21.15.87:0/1022796381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71070 0x55db43e98330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- 172.21.15.87:0/1022796381 >> 172.21.15.87:0/1022796381 conn(0x55db43e6d680 msgr2=0x55db43e6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- 172.21.15.87:0/1022796381 shutdown_connections 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- 172.21.15.87:0/1022796381 wait complete. 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 -- start start 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.705+0000 7f58499a5740 1 Processor -- start 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e7a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58401ab230 con 0x55db43e71070 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58401ac6a0 con 0x55db43e71990 2024-06-06T07:59:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58401ab900 con 0x55db43e80760 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 10 client.0 _ll_register_callbacks cb 0x7f58380127d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53222/0 (socket says 172.21.15.87:53222) 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 -- 172.21.15.87:0/4240179085 learned_addr learned my addr 172.21.15.87:0/4240179085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5845154640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f583ffff640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e7a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 -- 172.21.15.87:0/4240179085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 msgr2=0x55db43e7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 -- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 msgr2=0x55db43e7a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e7a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 -- 172.21.15.87:0/4240179085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db43e7ae00 con 0x55db43e71070 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f5844953640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5834019b90 tx=0x7f5834004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f064e7259f76ca0f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f581ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834028070 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9495+0+0 (secure 0 0 0) 0x7f5834019420 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.706+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c0b9e60 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5834019660 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 10 client.17217 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55db43efb1a0 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c0b9e60 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f5834022070 con 0x55db43e71070 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 10 client.17217 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 20 client.17217 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43efb8a0 0x55db43efdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 4 client.17217 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:59:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db43de2c20 con 0x55db43efb8a0 2024-06-06T07:59:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.707+0000 7f583ffff640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43efb8a0 0x55db43efdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.708+0000 7f581ffff640 10 client.17217 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.708+0000 7f583ffff640 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43efb8a0 0x55db43efdcc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3e9d53e09e429 server_cookie=1188502fd2701183 in_seq=0 out_seq=0 2024-06-06T07:59:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f581ffff640 1 -- 172.21.15.87:0/4240179085 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55db43de2c20 con 0x55db43efb8a0 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f581ffff640 10 client.17217 handle_command_reply: tid=0 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 client.17217 shutdown 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 msgr2=0x55db43e88dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5834019b90 tx=0x7f5834004430 comp rx=0 tx=0).stop 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 shutdown_connections 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db43efb8a0 0x55db43efdcc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db43e80760 0x55db43e7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f581ffff640 0 client.17217 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db43e71990 0x55db43e7a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 --2- 172.21.15.87:0/4240179085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db43e71070 0x55db43e88dc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 >> 172.21.15.87:0/4240179085 conn(0x55db43e6d680 msgr2=0x55db43e6bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 shutdown_connections 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.791+0000 7f58499a5740 1 -- 172.21.15.87:0/4240179085 wait complete. 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f58499a5740 20 client.17217 trim_cache size 0 max 16384 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:59:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:59:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 msgr2=0x7f58300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 0x7f58300a52e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5838012c80 tx=0x7f5838009e80 comp rx=0 tx=0).stop 2024-06-06T07:59:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 msgr2=0x7f5840198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f582c009230 tx=0x7f582c004270 comp rx=0 tx=0).stop 2024-06-06T07:59:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 shutdown_connections 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f58300a2e20 0x7f58300a52e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5840106890 0x7f584019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5840101290 0x7f5840198820 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 --2- 172.21.15.87:0/855571098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58401008d0 0x7f58401982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 >> 172.21.15.87:0/855571098 conn(0x7f5840077a20 msgr2=0x7f58400693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 shutdown_connections 2024-06-06T07:59:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:05.792+0000 7f5846bb7640 1 -- 172.21.15.87:0/855571098 wait complete. 2024-06-06T07:59:06.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:05 smithi195 ceph-mon[37040]: pgmap v346: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 693 KiB/s wr, 162 op/s 2024-06-06T07:59:06.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:05 smithi181 ceph-mon[36790]: pgmap v346: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 693 KiB/s wr, 162 op/s 2024-06-06T07:59:06.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:05 smithi087 ceph-mon[32276]: pgmap v346: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 693 KiB/s wr, 162 op/s 2024-06-06T07:59:06.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.29 MB/sec execute 430 sec latency 11601.897 ms 2024-06-06T07:59:07.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.28 MB/sec execute 431 sec latency 12601.998 ms 2024-06-06T07:59:08.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:07 smithi195 ceph-mon[37040]: pgmap v347: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 651 KiB/s wr, 177 op/s 2024-06-06T07:59:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:07 smithi181 ceph-mon[36790]: pgmap v347: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 651 KiB/s wr, 177 op/s 2024-06-06T07:59:08.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:07 smithi087 ceph-mon[32276]: pgmap v347: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 651 KiB/s wr, 177 op/s 2024-06-06T07:59:08.683 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.26 MB/sec execute 432 sec latency 13602.092 ms 2024-06-06T07:59:09.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.25 MB/sec execute 433 sec latency 14602.198 ms 2024-06-06T07:59:10.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:09 smithi195 ceph-mon[37040]: pgmap v348: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 80 KiB/s wr, 58 op/s 2024-06-06T07:59:10.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:09 smithi181 ceph-mon[36790]: pgmap v348: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 80 KiB/s wr, 58 op/s 2024-06-06T07:59:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:09 smithi087 ceph-mon[32276]: pgmap v348: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 80 KiB/s wr, 58 op/s 2024-06-06T07:59:10.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.23 MB/sec execute 434 sec latency 15602.302 ms 2024-06-06T07:59:11.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.22 MB/sec execute 435 sec latency 16602.341 ms 2024-06-06T07:59:12.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:11 smithi195 ceph-mon[37040]: pgmap v349: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:59:12.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:11 smithi195 ceph-mon[37040]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:59:12.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:11 smithi181 ceph-mon[36790]: pgmap v349: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:59:12.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:11 smithi181 ceph-mon[36790]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:59:12.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:11 smithi087 ceph-mon[32276]: pgmap v349: 129 pgs: 129 active+clean; 80 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:59:12.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:11 smithi087 ceph-mon[32276]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:59:12.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.20 MB/sec execute 436 sec latency 17602.391 ms 2024-06-06T07:59:13.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.19 MB/sec execute 437 sec latency 18602.431 ms 2024-06-06T07:59:14.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:13 smithi195 ceph-mon[37040]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:14.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:13 smithi181 ceph-mon[36790]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:14.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:13 smithi087 ceph-mon[32276]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:14.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.18 MB/sec execute 438 sec latency 19602.503 ms 2024-06-06T07:59:15.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.16 MB/sec execute 439 sec latency 20602.548 ms 2024-06-06T07:59:16.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:15 smithi195 ceph-mon[37040]: pgmap v352: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:16.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:15 smithi181 ceph-mon[36790]: pgmap v352: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:15 smithi087 ceph-mon[32276]: pgmap v352: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 459 KiB/s wr, 47 op/s 2024-06-06T07:59:16.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.15 MB/sec execute 440 sec latency 21602.601 ms 2024-06-06T07:59:17.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.13 MB/sec execute 441 sec latency 22602.695 ms 2024-06-06T07:59:18.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:17 smithi195 ceph-mon[37040]: pgmap v353: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 648 KiB/s wr, 28 op/s 2024-06-06T07:59:18.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:17 smithi181 ceph-mon[36790]: pgmap v353: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 648 KiB/s wr, 28 op/s 2024-06-06T07:59:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:17 smithi087 ceph-mon[32276]: pgmap v353: 129 pgs: 9 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 97 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 648 KiB/s wr, 28 op/s 2024-06-06T07:59:18.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.12 MB/sec execute 442 sec latency 23602.755 ms 2024-06-06T07:59:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 -- 172.21.15.87:0/216466017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c106890 msgr2=0x7f066c106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 --2- 172.21.15.87:0/216466017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c106890 0x7f066c106c70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f065c009930 tx=0x7f065c02a420 comp rx=0 tx=0).stop 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 -- 172.21.15.87:0/216466017 shutdown_connections 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 --2- 172.21.15.87:0/216466017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 0x7f066c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 --2- 172.21.15.87:0/216466017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c1008d0 0x7f066c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 --2- 172.21.15.87:0/216466017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c106890 0x7f066c106c70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 -- 172.21.15.87:0/216466017 >> 172.21.15.87:0/216466017 conn(0x7f066c077a20 msgr2=0x7f066c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 -- 172.21.15.87:0/216466017 shutdown_connections 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.889+0000 7f0670fa9640 1 -- 172.21.15.87:0/216466017 wait complete. 2024-06-06T07:59:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 Processor -- start 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 -- start start 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 0x7f066c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c106890 0x7f066c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f066c110ca0 con 0x7f066c1008d0 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f066c110b20 con 0x7f066c101290 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f0670fa9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f066c110e20 con 0x7f066c106890 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f066a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f066a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60816/0 (socket says 172.21.15.87:60816) 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.890+0000 7f066a59c640 1 -- 172.21.15.87:0/1496133287 learned_addr learned my addr 172.21.15.87:0/1496133287 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f0669d9b640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 0x7f066c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066ad9d640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c106890 0x7f066c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 -- 172.21.15.87:0/1496133287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c106890 msgr2=0x7f066c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c106890 0x7f066c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 -- 172.21.15.87:0/1496133287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 msgr2=0x7f066c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 0x7f066c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 -- 172.21.15.87:0/1496133287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c19d2e0 con 0x7f066c1008d0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f066a59c640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f065c009230 tx=0x7f065c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f065b7fe640 1 -- 172.21.15.87:0/1496133287 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c03f030 con 0x7f066c1008d0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f065b7fe640 1 -- 172.21.15.87:0/1496133287 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c003ad0 con 0x7f066c1008d0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c19d570 con 0x7f066c1008d0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f065b7fe640 1 -- 172.21.15.87:0/1496133287 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c035070 con 0x7f066c1008d0 2024-06-06T07:59:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.891+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a4e60 con 0x7f066c1008d0 2024-06-06T07:59:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.894+0000 7f065b7fe640 1 -- 172.21.15.87:0/1496133287 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f065c042030 con 0x7f066c1008d0 2024-06-06T07:59:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.894+0000 7f0673d9a740 1 Processor -- start 2024-06-06T07:59:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 -- start start 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce131f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ce131594f0 con 0x55ce13200350 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ce130f7870 con 0x55ce131f5a90 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0673d9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ce131554b0 con 0x55ce131f5170 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f065b7fe640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 0x7f06540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60828/0 (socket says 172.21.15.87:60828) 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 -- 172.21.15.87:0/2830883933 learned_addr learned my addr 172.21.15.87:0/2830883933 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f066ad9d640 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce131f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f066a59c640 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f066ad9d640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 0x7f06540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f065b7fe640 1 -- 172.21.15.87:0/1496133287 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9671+0+0 (secure 0 0 0) 0x7f065c0efe60 con 0x7f066c1008d0 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 -- 172.21.15.87:0/2830883933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 msgr2=0x55ce131f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce131f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 -- 172.21.15.87:0/2830883933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 msgr2=0x55ce131ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.895+0000 7f0669d9b640 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f0669d9b640 1 -- 172.21.15.87:0/2830883933 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce131139b0 con 0x55ce13200350 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f0669d9b640 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f066000abe0 tx=0x7f066000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e74a2519cd08e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f066ad9d640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 0x7f06540a5260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0664005fd0 tx=0x7f0664009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f0658ff9640 1 -- 172.21.15.87:0/2830883933 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066001d070 con 0x55ce13200350 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f0658ff9640 1 -- 172.21.15.87:0/2830883933 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660018b20 con 0x55ce13200350 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.896+0000 7f0658ff9640 1 -- 172.21.15.87:0/2830883933 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660010b60 con 0x55ce13200350 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- 172.21.15.87:0/2830883933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 msgr2=0x55ce13202740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f066000abe0 tx=0x7f066000f790 comp rx=0 tx=0).stop 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- 172.21.15.87:0/2830883933 shutdown_connections 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce13200350 0x55ce13202740 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- 172.21.15.87:0/2830883933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce131f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- 172.21.15.87:0/2830883933 >> 172.21.15.87:0/2830883933 conn(0x55ce131f1680 msgr2=0x55ce131f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- 172.21.15.87:0/2830883933 shutdown_connections 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- 172.21.15.87:0/2830883933 wait complete. 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- start start 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 Processor -- start 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce131fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ce13202f80 con 0x55ce131f5170 2024-06-06T07:59:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ce130e2ae0 con 0x55ce131f5a90 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.897+0000 7f0673d9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ce13203100 con 0x55ce13200350 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0673d9a740 10 client.0 _ll_register_callbacks cb 0x55ce1311b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60838/0 (socket says 172.21.15.87:60838) 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 -- 172.21.15.87:0/3822514914 learned_addr learned my addr 172.21.15.87:0/3822514914 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066ad9d640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce131fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 -- 172.21.15.87:0/3822514914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 msgr2=0x55ce131fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce131fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 -- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 msgr2=0x55ce131fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 -- 172.21.15.87:0/3822514914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce12e6d210 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f066a59c640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f065c0bbcb0 tx=0x7f065c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3211950d788db69 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c049040 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9671+0+0 (secure 0 0 0) 0x7f065c003c80 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0660018b20 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f065c0bbfb0 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0673d9a740 10 client.17229 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.898+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ce12f39620 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660018b20 con 0x55ce131f5170 2024-06-06T07:59:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f065c0b8620 con 0x55ce131f5170 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 10 client.17229 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 20 client.17229 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce132128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 4 client.17229 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce1316cf10 con 0x55ce132104a0 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.899+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce132128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0651ffb640 10 client.17229 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce132128f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6ec5fe4558e626 server_cookie=12584952fb0aff76 in_seq=0 out_seq=0 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0651ffb640 1 -- 172.21.15.87:0/3822514914 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce1316cf10 con 0x55ce132104a0 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0651ffb640 10 client.17229 handle_command_reply: tid=0 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 client.17229 shutdown 2024-06-06T07:59:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 msgr2=0x55ce131fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f065c0bbcb0 tx=0x7f065c03b000 comp rx=0 tx=0).stop 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 shutdown_connections 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce132128f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce131fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0651ffb640 0 client.17229 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5a90 0x55ce131fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3822514914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5170 0x55ce131fd2d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 >> 172.21.15.87:0/3822514914 conn(0x55ce131f1680 msgr2=0x55ce131f73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 shutdown_connections 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.900+0000 7f0673d9a740 1 -- 172.21.15.87:0/3822514914 wait complete. 2024-06-06T07:59:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.901+0000 7f0673d9a740 20 client.17229 trim_cache size 0 max 16384 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 Processor -- start 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 -- start start 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5170 0x55ce131fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce132181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1b1930 con 0x55ce131f5a90 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1af200 con 0x55ce131f5170 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0673d9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1ad070 con 0x55ce13200350 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60846/0 (socket says 172.21.15.87:60846) 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 -- 172.21.15.87:0/259901072 learned_addr learned my addr 172.21.15.87:0/259901072 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 -- 172.21.15.87:0/259901072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 msgr2=0x55ce132181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f066a59c640 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5170 0x55ce131fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce132181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 -- 172.21.15.87:0/259901072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5170 msgr2=0x55ce131fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5170 0x55ce131fbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.916+0000 7f0669d9b640 1 -- 172.21.15.87:0/259901072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce131f5880 con 0x55ce131f5a90 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0669d9b640 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f066000fc90 tx=0x7f066000ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21d2a8cc937ec2f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0653fff640 1 -- 172.21.15.87:0/259901072 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066001d040 con 0x55ce131f5a90 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0653fff640 1 -- 172.21.15.87:0/259901072 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f066001e040 con 0x55ce131f5a90 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0653fff640 1 -- 172.21.15.87:0/259901072 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06600048c0 con 0x55ce131f5a90 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 -- 172.21.15.87:0/259901072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 msgr2=0x55ce131fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f066000fc90 tx=0x7f066000ab10 comp rx=0 tx=0).stop 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 -- 172.21.15.87:0/259901072 shutdown_connections 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce13200350 0x55ce132181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce131fc0e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 --2- 172.21.15.87:0/259901072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce131f5170 0x55ce131fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 -- 172.21.15.87:0/259901072 >> 172.21.15.87:0/259901072 conn(0x55ce131f1680 msgr2=0x55ce131f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 -- 172.21.15.87:0/259901072 shutdown_connections 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.917+0000 7f0673d9a740 1 -- 172.21.15.87:0/259901072 wait complete. 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 -- start start 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 Processor -- start 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce13203910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 0x55ce13279160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1ad6c0 con 0x55ce131f5a90 2024-06-06T07:59:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1aeb30 con 0x55ce13200350 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1add90 con 0x55ce131f5170 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0673d9a740 10 client.0 _ll_register_callbacks cb 0x7f065c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60854/0 (socket says 172.21.15.87:60854) 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 -- 172.21.15.87:0/3583978458 learned_addr learned my addr 172.21.15.87:0/3583978458 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f066a59c640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce13203910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f066ad9d640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 0x55ce13279160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 -- 172.21.15.87:0/3583978458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 msgr2=0x55ce13203910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce13203910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 -- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 msgr2=0x55ce13279160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 0x55ce13279160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f0669d9b640 1 -- 172.21.15.87:0/3583978458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce132796a0 con 0x55ce131f5a90 2024-06-06T07:59:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f066a59c640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce13203910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.918+0000 7f066ad9d640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 0x55ce13279160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0669d9b640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f06600054e0 tx=0x7f0660005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f81335eb8be073 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066003d060 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066001e040 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9671+0+0 (secure 0 0 0) 0x7f0660018540 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f066001d040 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 10 client.17235 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ce1327a920 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f066001e040 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f0660024430 con 0x55ce131f5a90 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 10 client.17235 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 20 client.17235 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce1327ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 4 client.17235 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.919+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce13166c20 con 0x55ce132104a0 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.920+0000 7f066a59c640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce1327ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.920+0000 7f066a59c640 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce1327ce80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a33f495694548c server_cookie=26a74bc3c8219bf5 in_seq=0 out_seq=0 2024-06-06T07:59:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.920+0000 7f0651ffb640 10 client.17235 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0651ffb640 1 -- 172.21.15.87:0/3583978458 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ce13166c20 con 0x55ce132104a0 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0651ffb640 10 client.17235 handle_command_reply: tid=0 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 client.17235 shutdown 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 msgr2=0x55ce13203e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f06600054e0 tx=0x7f0660005510 comp rx=0 tx=0).stop 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 shutdown_connections 2024-06-06T07:59:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ce132104a0 0x55ce1327ce80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce13200350 0x55ce13279160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0651ffb640 0 client.17235 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce131f5a90 0x55ce13203e50 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 --2- 172.21.15.87:0/3583978458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce131f5170 0x55ce13203910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 >> 172.21.15.87:0/3583978458 conn(0x55ce131f1680 msgr2=0x55ce131fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 shutdown_connections 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 1 -- 172.21.15.87:0/3583978458 wait complete. 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.948+0000 7f0673d9a740 20 client.17235 trim_cache size 0 max 16384 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:59:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:59:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:59:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 msgr2=0x7f06540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 0x7f06540a5260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0664005fd0 tx=0x7f0664009500 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 msgr2=0x7f066c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f065c009230 tx=0x7f065c004270 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 shutdown_connections 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f06540a2da0 0x7f06540a5260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066c106890 0x7f066c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f066c101290 0x7f066c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 --2- 172.21.15.87:0/1496133287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f066c1008d0 0x7f066c1982f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 >> 172.21.15.87:0/1496133287 conn(0x7f066c077a20 msgr2=0x7f066c069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 shutdown_connections 2024-06-06T07:59:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:18.949+0000 7f0670fa9640 1 -- 172.21.15.87:0/1496133287 wait complete. 2024-06-06T07:59:19.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.11 MB/sec execute 443 sec latency 24602.818 ms 2024-06-06T07:59:19.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:59:20.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:19 smithi195 ceph-mon[37040]: pgmap v354: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1000 KiB/s wr, 42 op/s 2024-06-06T07:59:20.029 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:59:20.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:19 smithi181 ceph-mon[36790]: pgmap v354: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1000 KiB/s wr, 42 op/s 2024-06-06T07:59:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:19 smithi087 ceph-mon[32276]: pgmap v354: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1000 KiB/s wr, 42 op/s 2024-06-06T07:59:20.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.09 MB/sec execute 444 sec latency 25602.888 ms 2024-06-06T07:59:21.058 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:20 smithi181 ceph-mon[36790]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:59:21.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:20 smithi087 ceph-mon[32276]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:59:21.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:20 smithi195 ceph-mon[37040]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:59:21.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.08 MB/sec execute 445 sec latency 26602.917 ms 2024-06-06T07:59:21.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.717+0000 7f2691cfe700 1 -- 172.21.15.87:0/726444508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c107400 msgr2=0x7f268c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:21.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.717+0000 7f2691cfe700 1 --2- 172.21.15.87:0/726444508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c107400 0x7f268c1077e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f267c00b0a0 tx=0x7f267c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.717+0000 7f2691cfe700 1 -- 172.21.15.87:0/726444508 shutdown_connections 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.717+0000 7f2691cfe700 1 --2- 172.21.15.87:0/726444508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101d50 0x7f268c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 --2- 172.21.15.87:0/726444508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c107400 0x7f268c1077e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 --2- 172.21.15.87:0/726444508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c101390 0x7f268c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 -- 172.21.15.87:0/726444508 >> 172.21.15.87:0/726444508 conn(0x7f268c078340 msgr2=0x7f268c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 -- 172.21.15.87:0/726444508 shutdown_connections 2024-06-06T07:59:21.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 -- 172.21.15.87:0/726444508 wait complete. 2024-06-06T07:59:21.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.718+0000 7f2691cfe700 1 Processor -- start 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 -- start start 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101390 0x7f268c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c107400 0x7f268c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1116a0 con 0x7f268c101d50 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f268c111520 con 0x7f268c101390 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f2691cfe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f268c111820 con 0x7f268c107400 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60868/0 (socket says 172.21.15.87:60868) 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 -- 172.21.15.87:0/1554143570 learned_addr learned my addr 172.21.15.87:0/1554143570 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268bfff700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c107400 0x7f268c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:21.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268b7fe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101390 0x7f268c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 -- 172.21.15.87:0/1554143570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c107400 msgr2=0x7f268c103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c107400 0x7f268c103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 -- 172.21.15.87:0/1554143570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101390 msgr2=0x7f268c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101390 0x7f268c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 -- 172.21.15.87:0/1554143570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1a71b0 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.719+0000 7f268affd700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f267400d980 tx=0x7f267400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.720+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.720+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674005020 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.720+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1a74a0 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.720+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674009bc0 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.720+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a7990 con 0x7f268c101d50 2024-06-06T07:59:21.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.721+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c069180 con 0x7f268c101d50 2024-06-06T07:59:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.722+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f26740051d0 con 0x7f268c101d50 2024-06-06T07:59:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.722+0000 7f2688ff9700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.723+0000 7f268b7fe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:21.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.723+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7f26740cfba0 con 0x7f268c101d50 2024-06-06T07:59:21.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.723+0000 7f268b7fe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 0x7f26780a5250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f267c006130 tx=0x7f267c0060a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:21.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.726+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2674020020 con 0x7f268c101d50 2024-06-06T07:59:21.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.967+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f268c1a7bf0 con 0x7f268c101d50 2024-06-06T07:59:21.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.968+0000 7f2688ff9700 1 -- 172.21.15.87:0/1554143570 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11877 (secure 0 0 0) 0x7f267409c3a0 con 0x7f268c101d50 2024-06-06T07:59:21.968 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T07:59:21.968 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T07:58:53.675707+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26416":{"gid":26416,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1032843361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1032843361},{"type":"v1","addr":"172.21.15.195:6837","nonce":1032843361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 msgr2=0x7f26780a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 0x7f26780a5250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f267c006130 tx=0x7f267c0060a0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 msgr2=0x7f268c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f267400d980 tx=0x7f267400dc90 comp rx=0 tx=0).stop 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 shutdown_connections 2024-06-06T07:59:21.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268c101390 0x7f268c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f268c107400 0x7f268c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 --2- 172.21.15.87:0/1554143570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c101d50 0x7f268c1035b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 >> 172.21.15.87:0/1554143570 conn(0x7f268c078340 msgr2=0x7f268c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 shutdown_connections 2024-06-06T07:59:21.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:21.970+0000 7f2691cfe700 1 -- 172.21.15.87:0/1554143570 wait complete. 2024-06-06T07:59:21.972 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 38 2024-06-06T07:59:22.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:21 smithi181 ceph-mon[36790]: pgmap v356: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1013 KiB/s wr, 42 op/s 2024-06-06T07:59:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:21 smithi087 ceph-mon[32276]: pgmap v356: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1013 KiB/s wr, 42 op/s 2024-06-06T07:59:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:21 smithi195 ceph-mon[37040]: pgmap v356: 129 pgs: 129 active+clean; 84 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1013 KiB/s wr, 42 op/s 2024-06-06T07:59:22.680 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 1494725a --release --await' 2024-06-06T07:59:22.681 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 1494725a --release --await 2024-06-06T07:59:22.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.07 MB/sec execute 446 sec latency 27602.948 ms 2024-06-06T07:59:22.921 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:59:23.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:22 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1554143570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:59:23.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:22 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1554143570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:59:23.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:22 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1554143570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:59:23.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.05 MB/sec execute 447 sec latency 28602.975 ms 2024-06-06T07:59:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:23 smithi087 ceph-mon[32276]: pgmap v357: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:23 smithi195 ceph-mon[37040]: pgmap v357: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:23 smithi181 ceph-mon[36790]: pgmap v357: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:24.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 623085 6.04 MB/sec execute 448 sec latency 29603.050 ms 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 -- 172.21.15.87:0/2572512045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 msgr2=0x7f24b810e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 --2- 172.21.15.87:0/2572512045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b810e270 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009990 tx=0x7f24b0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 -- 172.21.15.87:0/2572512045 shutdown_connections 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 --2- 172.21.15.87:0/2572512045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 --2- 172.21.15.87:0/2572512045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b810e270 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 --2- 172.21.15.87:0/2572512045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 0x7f24b8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 -- 172.21.15.87:0/2572512045 >> 172.21.15.87:0/2572512045 conn(0x7f24b80783d0 msgr2=0x7f24b80787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 -- 172.21.15.87:0/2572512045 shutdown_connections 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.015+0000 7f24c0769700 1 -- 172.21.15.87:0/2572512045 wait complete. 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 Processor -- start 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 -- start start 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 0x7f24b819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81116a0 con 0x7f24b8101d80 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8111520 con 0x7f24b8107430 2024-06-06T07:59:25.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24c0769700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8111820 con 0x7f24b81013c0 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60878/0 (socket says 172.21.15.87:60878) 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bdd04700 1 -- 172.21.15.87:0/121924666 learned_addr learned my addr 172.21.15.87:0/121924666 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bed06700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24be505700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 0x7f24b819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bdd04700 1 -- 172.21.15.87:0/121924666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 msgr2=0x7f24b819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.016+0000 7f24bdd04700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 0x7f24b819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24bdd04700 1 -- 172.21.15.87:0/121924666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 msgr2=0x7f24b8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24bdd04700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24bdd04700 1 -- 172.21.15.87:0/121924666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b81990b0 con 0x7f24b8101d80 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24bed06700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24bdd04700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c920 tx=0x7f24a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24af7fe700 1 -- 172.21.15.87:0/121924666 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8007b00 con 0x7f24b8101d80 2024-06-06T07:59:25.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b81993a0 con 0x7f24b8101d80 2024-06-06T07:59:25.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24af7fe700 1 -- 172.21.15.87:0/121924666 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8004d10 con 0x7f24b8101d80 2024-06-06T07:59:25.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24af7fe700 1 -- 172.21.15.87:0/121924666 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80056e0 con 0x7f24b8101d80 2024-06-06T07:59:25.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.017+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a5660 con 0x7f24b8101d80 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.019+0000 7f24af7fe700 1 -- 172.21.15.87:0/121924666 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f24a8007ca0 con 0x7f24b8101d80 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.020+0000 7f24af7fe700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 0x7f24a40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.020+0000 7f24be505700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 0x7f24a40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.020+0000 7f24af7fe700 1 -- 172.21.15.87:0/121924666 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7f24a80cf500 con 0x7f24b8101d80 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.020+0000 7f24be505700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 0x7f24a40a5250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24b8199a90 tx=0x7f24b4009040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 Processor -- start 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 -- start start 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 0x56119f1ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56119f138910 con 0x56119f1ed560 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56119f0e0820 con 0x56119f1ecc40 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24cafecb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56119f1348d0 con 0x56119f1e73d0 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60888/0 (socket says 172.21.15.87:60888) 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24bdd04700 1 -- 172.21.15.87:0/505864742 learned_addr learned my addr 172.21.15.87:0/505864742 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.021+0000 7f24bed06700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24be505700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 0x56119f1ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 -- 172.21.15.87:0/505864742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 msgr2=0x56119f1e7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1e7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 -- 172.21.15.87:0/505864742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 msgr2=0x56119f1ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 0x56119f1ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 -- 172.21.15.87:0/505864742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119f0d7050 con 0x56119f1ed560 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24be505700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 0x56119f1ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24bdd04700 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097a60 tx=0x7f24a80979d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6284b73b6421d55a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24acff9700 1 -- 172.21.15.87:0/505864742 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a801d040 con 0x56119f1ed560 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24acff9700 1 -- 172.21.15.87:0/505864742 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a809b600 con 0x56119f1ed560 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.022+0000 7f24acff9700 1 -- 172.21.15.87:0/505864742 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8097040 con 0x56119f1ed560 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- 172.21.15.87:0/505864742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 msgr2=0x56119f1e6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097a60 tx=0x7f24a80979d0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- 172.21.15.87:0/505864742 shutdown_connections 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ecc40 0x56119f1ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 --2- 172.21.15.87:0/505864742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ed560 0x56119f1e6e90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- 172.21.15.87:0/505864742 >> 172.21.15.87:0/505864742 conn(0x56119f111dc0 msgr2=0x56119f1e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- 172.21.15.87:0/505864742 shutdown_connections 2024-06-06T07:59:25.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- 172.21.15.87:0/505864742 wait complete. 2024-06-06T07:59:25.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 -- start start 2024-06-06T07:59:25.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.023+0000 7f24cafecb80 1 Processor -- start 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1ecc40 0x56119f1f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56119f1f3fd0 con 0x56119f1e73d0 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56119f0d8cf0 con 0x56119f1ed560 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56119f1f4150 con 0x56119f1ecc40 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 10 client.0 _ll_register_callbacks cb 0x56119f0d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60896/0 (socket says 172.21.15.87:60896) 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 -- 172.21.15.87:0/1691235391 learned_addr learned my addr 172.21.15.87:0/1691235391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1ecc40 0x56119f1f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24bed06700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 -- 172.21.15.87:0/1691235391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1ecc40 msgr2=0x56119f1f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1ecc40 0x56119f1f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 -- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 msgr2=0x56119f1f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 -- 172.21.15.87:0/1691235391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119f0879d0 con 0x56119f1e73d0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24be505700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f24b40159e0 tx=0x7f24b4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bb7927ed32e1ab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4013870 con 0x56119f1e73d0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7f24b4013a10 con 0x56119f1e73d0 2024-06-06T07:59:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.024+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a809b600 con 0x56119f1e73d0 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24b402e050 con 0x56119f1e73d0 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 10 client.17253 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56119f1fe8c0 con 0x56119f1e73d0 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a809b600 con 0x56119f1e73d0 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f24b4014070 con 0x56119f1e73d0 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 10 client.17253 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 10 client.17253 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 20 client.17253 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f203330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 4 client.17253 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56119f145e10 con 0x56119f200f00 2024-06-06T07:59:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.025+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f203330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.026+0000 7f24a1ffb700 10 client.17253 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.026+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f203330 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bcd5bd33f61a41 server_cookie=e163afbda9c27e9b in_seq=0 out_seq=0 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.026+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1691235391 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56119f145e10 con 0x56119f200f00 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.026+0000 7f24a1ffb700 10 client.17253 handle_command_reply: tid=0 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 client.17253 shutdown 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 msgr2=0x56119f1f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f24b40159e0 tx=0x7f24b4004b40 comp rx=0 tx=0).stop 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 shutdown_connections 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f203330 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1e73d0 0x56119f1f3330 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24a1ffb700 0 client.17253 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:25.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1691235391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1ecc40 0x56119f1f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 >> 172.21.15.87:0/1691235391 conn(0x56119f111dc0 msgr2=0x56119f1e5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 shutdown_connections 2024-06-06T07:59:25.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 1 -- 172.21.15.87:0/1691235391 wait complete. 2024-06-06T07:59:25.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.027+0000 7f24cafecb80 20 client.17253 trim_cache size 0 max 16384 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 Processor -- start 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 -- start start 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81b1040 con 0x56119f1ecc40 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ae8d0 con 0x56119f1ed560 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24cafecb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ac740 con 0x56119f1e73d0 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60902/0 (socket says 172.21.15.87:60902) 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 -- 172.21.15.87:0/504574363 learned_addr learned my addr 172.21.15.87:0/504574363 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bed06700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24be505700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 -- 172.21.15.87:0/504574363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 msgr2=0x56119f1db9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1db9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 -- 172.21.15.87:0/504574363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 msgr2=0x56119f1f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bdd04700 1 -- 172.21.15.87:0/504574363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119f1edff0 con 0x56119f1ecc40 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.047+0000 7f24bed06700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24bdd04700 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c8f0 tx=0x7f24a8086130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49454fba25d68b85 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24a3fff700 1 -- 172.21.15.87:0/504574363 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b9070 con 0x56119f1ecc40 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24a3fff700 1 -- 172.21.15.87:0/504574363 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8097c70 con 0x56119f1ecc40 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24a3fff700 1 -- 172.21.15.87:0/504574363 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b4070 con 0x56119f1ecc40 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 -- 172.21.15.87:0/504574363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 msgr2=0x56119f1e1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c8f0 tx=0x7f24a8086130 comp rx=0 tx=0).stop 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 -- 172.21.15.87:0/504574363 shutdown_connections 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f1db9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 --2- 172.21.15.87:0/504574363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f1e1f40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 -- 172.21.15.87:0/504574363 >> 172.21.15.87:0/504574363 conn(0x56119f111dc0 msgr2=0x56119f1e1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 -- 172.21.15.87:0/504574363 shutdown_connections 2024-06-06T07:59:25.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.048+0000 7f24cafecb80 1 -- 172.21.15.87:0/504574363 wait complete. 2024-06-06T07:59:25.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 -- start start 2024-06-06T07:59:25.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 Processor -- start 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f207bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f208670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81acd90 con 0x56119f1ecc40 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ae200 con 0x56119f1ed560 2024-06-06T07:59:25.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ad460 con 0x56119f1e73d0 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24cafecb80 10 client.0 _ll_register_callbacks cb 0x56119f0d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60918/0 (socket says 172.21.15.87:60918) 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 -- 172.21.15.87:0/1223884077 learned_addr learned my addr 172.21.15.87:0/1223884077 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24be505700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f207bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bed06700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f208670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 -- 172.21.15.87:0/1223884077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 msgr2=0x56119f207bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f207bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 -- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 msgr2=0x56119f208670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f208670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bdd04700 1 -- 172.21.15.87:0/1223884077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119f26fc10 con 0x56119f1ecc40 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.049+0000 7f24bed06700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f208670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:25.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24bdd04700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097a60 tx=0x7f24a8004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827f78c973aa1680 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b9070 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7f24a809ba60 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a8097c70 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24a80b4040 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 10 client.17259 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56119f270eb0 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8097c70 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f24a809aa50 con 0x56119f1ecc40 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 10 client.17259 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 10 client.17259 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 20 client.17259 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f2730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 4 client.17259 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1494725a", "release": true, "await": true}] 2024-06-06T07:59:25.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.050+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1494725a", "release": true, "await": true}) -- 0x56119f140390 con 0x56119f200f00 2024-06-06T07:59:25.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.051+0000 7f24be505700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f2730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:25.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.051+0000 7f24a1ffb700 10 client.17259 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:25.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.051+0000 7f24be505700 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f2730e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513bae7c858b7f69 server_cookie=1fab26b03c45e2a8 in_seq=0 out_seq=0 2024-06-06T07:59:25.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24a1ffb700 1 -- 172.21.15.87:0/1223884077 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56119f140390 con 0x56119f200f00 2024-06-06T07:59:25.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24a1ffb700 10 client.17259 handle_command_reply: tid=0 2024-06-06T07:59:25.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 client.17259 shutdown 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 msgr2=0x56119f208130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097a60 tx=0x7f24a8004fe0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 shutdown_connections 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56119f200f00 0x56119f2730e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56119f1ed560 0x56119f208670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56119f1e73d0 0x56119f207bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24a1ffb700 0 client.17259 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 --2- 172.21.15.87:0/1223884077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56119f1ecc40 0x56119f208130 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 >> 172.21.15.87:0/1223884077 conn(0x56119f111dc0 msgr2=0x56119f1e20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 shutdown_connections 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 1 -- 172.21.15.87:0/1223884077 wait complete. 2024-06-06T07:59:25.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.063+0000 7f24cafecb80 20 client.17259 trim_cache size 0 max 16384 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 38, 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 36, 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "1494725a": { 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 36, 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:59:25.065 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:59:25.066 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 msgr2=0x7f24a40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 0x7f24a40a5250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24b8199a90 tx=0x7f24b4009040 comp rx=0 tx=0).stop 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 msgr2=0x7f24b819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c920 tx=0x7f24a800cce0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 shutdown_connections 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24b8107430 0x7f24b8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f24a40a2d90 0x7f24a40a5250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81013c0 0x7f24b819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 --2- 172.21.15.87:0/121924666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8101d80 0x7f24b819ecd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 >> 172.21.15.87:0/121924666 conn(0x7f24b80783d0 msgr2=0x7f24b81005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 shutdown_connections 2024-06-06T07:59:25.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T07:59:25.064+0000 7f24c0769700 1 -- 172.21.15.87:0/121924666 wait complete. 2024-06-06T07:59:25.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 624712 6.04 MB/sec execute 449 sec latency 29985.684 ms 2024-06-06T07:59:25.790 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1494725a', total seconds elapsed: 36.1 2024-06-06T07:59:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:25 smithi087 ceph-mon[32276]: pgmap v358: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:25 smithi195 ceph-mon[37040]: pgmap v358: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:25 smithi181 ceph-mon[36790]: pgmap v358: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 600 KiB/s wr, 59 op/s 2024-06-06T07:59:26.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 626479 6.05 MB/sec execute 450 sec latency 16.483 ms 2024-06-06T07:59:27.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 628758 6.07 MB/sec execute 451 sec latency 20.641 ms 2024-06-06T07:59:28.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:27 smithi087 ceph-mon[32276]: pgmap v359: 129 pgs: 129 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 439 KiB/s wr, 50 op/s 2024-06-06T07:59:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:27 smithi195 ceph-mon[37040]: pgmap v359: 129 pgs: 129 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 439 KiB/s wr, 50 op/s 2024-06-06T07:59:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:27 smithi181 ceph-mon[36790]: pgmap v359: 129 pgs: 129 active+clean; 75 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 439 KiB/s wr, 50 op/s 2024-06-06T07:59:28.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 630266 6.07 MB/sec execute 452 sec latency 33.009 ms 2024-06-06T07:59:29.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:59:29.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:59:29.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:59:29.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:59:29.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:59:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:59:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:59:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:59:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:59:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:59:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:59:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T07:59:29.684 INFO:tasks.workunit.client.0.smithi087.stdout: 1 632697 6.09 MB/sec execute 453 sec latency 12.803 ms 2024-06-06T07:59:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:29 smithi087 ceph-mon[32276]: pgmap v360: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 365 KiB/s wr, 58 op/s 2024-06-06T07:59:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:29 smithi195 ceph-mon[37040]: pgmap v360: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 365 KiB/s wr, 58 op/s 2024-06-06T07:59:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:29 smithi181 ceph-mon[36790]: pgmap v360: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 365 KiB/s wr, 58 op/s 2024-06-06T07:59:30.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 634737 6.09 MB/sec execute 454 sec latency 17.067 ms 2024-06-06T07:59:31.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.10 MB/sec execute 455 sec latency 725.280 ms 2024-06-06T07:59:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 -- 172.21.15.87:0/951368811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988105850 msgr2=0x7ff988105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 --2- 172.21.15.87:0/951368811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988105850 0x7ff988105c30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff978009900 tx=0x7ff97802a410 comp rx=0 tx=0).stop 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 -- 172.21.15.87:0/951368811 shutdown_connections 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 --2- 172.21.15.87:0/951368811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff98810d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 --2- 172.21.15.87:0/951368811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988100bf0 0x7ff988101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 --2- 172.21.15.87:0/951368811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988105850 0x7ff988105c30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 -- 172.21.15.87:0/951368811 >> 172.21.15.87:0/951368811 conn(0x7ff9880766a0 msgr2=0x7ff988076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 -- 172.21.15.87:0/951368811 shutdown_connections 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.046+0000 7ff98d91b640 1 -- 172.21.15.87:0/951368811 wait complete. 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 Processor -- start 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 -- start start 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff9881994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988105850 0x7ff9881a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff988110c90 con 0x7ff988100bf0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff988110b10 con 0x7ff9881015b0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff98d91b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff988110e10 con 0x7ff988105850 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36180/0 (socket says 172.21.15.87:36180) 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.047+0000 7ff986ffd640 1 -- 172.21.15.87:0/298696699 learned_addr learned my addr 172.21.15.87:0/298696699 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff9867fc640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff9881994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff9877fe640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988105850 0x7ff9881a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 -- 172.21.15.87:0/298696699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988105850 msgr2=0x7ff9881a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988105850 0x7ff9881a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 -- 172.21.15.87:0/298696699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 msgr2=0x7ff9881994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff9881994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 -- 172.21.15.87:0/298696699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9881a4570 con 0x7ff988100bf0 2024-06-06T07:59:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff9867fc640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff9881994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff986ffd640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff978003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff98c919640 1 -- 172.21.15.87:0/298696699 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97803f070 con 0x7ff988100bf0 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9881a4740 con 0x7ff988100bf0 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff98c919640 1 -- 172.21.15.87:0/298696699 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff978031c30 con 0x7ff988100bf0 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff98c919640 1 -- 172.21.15.87:0/298696699 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978035030 con 0x7ff988100bf0 2024-06-06T07:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.048+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9881a4bb0 con 0x7ff988100bf0 2024-06-06T07:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff98c919640 1 -- 172.21.15.87:0/298696699 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff978042070 con 0x7ff988100bf0 2024-06-06T07:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 Processor -- start 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 -- start start 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 0x55add91f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 0x55add91f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55add914a4f0 con 0x55add91ea580 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55add90e8870 con 0x55add91f5760 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.051+0000 7ff99062e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55add91464b0 con 0x55add91eaea0 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36196/0 (socket says 172.21.15.87:36196) 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 -- 172.21.15.87:0/3210835781 learned_addr learned my addr 172.21.15.87:0/3210835781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff98c919640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 0x7ff9700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff986ffd640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 0x55add91f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9867fc640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 0x55add91f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 -- 172.21.15.87:0/3210835781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 msgr2=0x55add91f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 0x55add91f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 -- 172.21.15.87:0/3210835781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 msgr2=0x55add91f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 0x55add91f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 -- 172.21.15.87:0/3210835781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add91049b0 con 0x55add91ea580 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9867fc640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 0x55add91f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff986ffd640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 0x55add91f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff98c919640 1 -- 172.21.15.87:0/298696699 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7ff9780efe40 con 0x7ff988100bf0 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 0x7ff9700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff98000f3e0 tx=0x7ff98000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3747e065c60c69 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff96dffb640 1 -- 172.21.15.87:0/3210835781 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98001c9d0 con 0x55add91ea580 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff96dffb640 1 -- 172.21.15.87:0/3210835781 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98001cb70 con 0x55add91ea580 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.052+0000 7ff96dffb640 1 -- 172.21.15.87:0/3210835781 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98001a3d0 con 0x55add91ea580 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 -- 172.21.15.87:0/3210835781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 msgr2=0x55add91ea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff98000f3e0 tx=0x7ff98000f8a0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff9877fe640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 0x7ff9700a5260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff980020000 tx=0x7ff98001d000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 -- 172.21.15.87:0/3210835781 shutdown_connections 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91f5760 0x55add91f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91eaea0 0x55add91f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 --2- 172.21.15.87:0/3210835781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91ea580 0x55add91ea960 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 -- 172.21.15.87:0/3210835781 >> 172.21.15.87:0/3210835781 conn(0x55add91e6a90 msgr2=0x55add91e6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 -- 172.21.15.87:0/3210835781 shutdown_connections 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.053+0000 7ff99062e740 1 -- 172.21.15.87:0/3210835781 wait complete. 2024-06-06T07:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 -- start start 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 Processor -- start 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add91f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55add91f8390 con 0x55add91f5760 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55add90d3ae0 con 0x55add91eaea0 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55add91f8510 con 0x55add91ea580 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff99062e740 10 client.0 _ll_register_callbacks cb 0x55add910c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36206/0 (socket says 172.21.15.87:36206) 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff9877fe640 1 -- 172.21.15.87:0/3855645524 learned_addr learned my addr 172.21.15.87:0/3855645524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff9867fc640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add91f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.054+0000 7ff986ffd640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 -- 172.21.15.87:0/3855645524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 msgr2=0x55add91f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 -- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 msgr2=0x55add91f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add91f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 -- 172.21.15.87:0/3855645524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add8e5e210 con 0x55add91f5760 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9867fc640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add91f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff9877fe640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff980012fa0 tx=0x7ff98000ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2205d8a1538ba7bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980035060 con 0x55add91f5760 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7ff98001bcc0 con 0x55add91f5760 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98001cb70 con 0x55add91f5760 2024-06-06T07:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff98002c040 con 0x55add91f5760 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff99062e740 10 client.17271 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55add8f2a620 con 0x55add91f5760 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98001cb70 con 0x55add91f5760 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.055+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7ff9800266b0 con 0x55add91f5760 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 10 client.17271 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 20 client.17271 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add92056f0 0x55add9207ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 4 client.17271 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add915df10 con 0x55add92056f0 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff986ffd640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add92056f0 0x55add9207ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff986ffd640 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add92056f0 0x55add9207ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66feafcb9995a537 server_cookie=368786819d01e12a in_seq=0 out_seq=0 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.056+0000 7ff966ffd640 10 client.17271 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.057+0000 7ff966ffd640 1 -- 172.21.15.87:0/3855645524 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55add915df10 con 0x55add92056f0 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.057+0000 7ff966ffd640 10 client.17271 handle_command_reply: tid=0 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.057+0000 7ff99062e740 1 client.17271 shutdown 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 msgr2=0x55add91f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff980012fa0 tx=0x7ff98000ffb0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 shutdown_connections 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add92056f0 0x55add9207ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91f5760 0x55add91f3110 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add91f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 --2- 172.21.15.87:0/3855645524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff966ffd640 0 client.17271 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 >> 172.21.15.87:0/3855645524 conn(0x55add91e6a90 msgr2=0x55add91e17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 shutdown_connections 2024-06-06T07:59:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 1 -- 172.21.15.87:0/3855645524 wait complete. 2024-06-06T07:59:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.058+0000 7ff99062e740 20 client.17271 trim_cache size 0 max 16384 2024-06-06T07:59:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 Processor -- start 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 -- start start 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91e6a90 0x55add91f1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add920d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881af730 con 0x55add91eaea0 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881ad000 con 0x55add91e6a90 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff99062e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881aae70 con 0x55add91ea580 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36220/0 (socket says 172.21.15.87:36220) 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.073+0000 7ff9877fe640 1 -- 172.21.15.87:0/2501908112 learned_addr learned my addr 172.21.15.87:0/2501908112 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff986ffd640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91e6a90 0x55add91f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9867fc640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add920d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 -- 172.21.15.87:0/2501908112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 msgr2=0x55add920d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add920d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 -- 172.21.15.87:0/2501908112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91e6a90 msgr2=0x55add91f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91e6a90 0x55add91f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 -- 172.21.15.87:0/2501908112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add92015f0 con 0x55add91eaea0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9867fc640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add920d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff9877fe640 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff98001cfd0 tx=0x7ff9800092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224a4c5fee8f9893 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff96d7fa640 1 -- 172.21.15.87:0/2501908112 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980045070 con 0x55add91eaea0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff96d7fa640 1 -- 172.21.15.87:0/2501908112 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9800098a0 con 0x55add91eaea0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff96d7fa640 1 -- 172.21.15.87:0/2501908112 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980035040 con 0x55add91eaea0 2024-06-06T07:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff99062e740 1 -- 172.21.15.87:0/2501908112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 msgr2=0x55add920d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.074+0000 7ff99062e740 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff98001cfd0 tx=0x7ff9800092a0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- 172.21.15.87:0/2501908112 shutdown_connections 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91eaea0 0x55add920d7d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add920d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- 172.21.15.87:0/2501908112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91e6a90 0x55add91f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- 172.21.15.87:0/2501908112 >> 172.21.15.87:0/2501908112 conn(0x55add9207e50 msgr2=0x55add91f0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- 172.21.15.87:0/2501908112 shutdown_connections 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- 172.21.15.87:0/2501908112 wait complete. 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- start start 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 Processor -- start 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add926e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881ab4c0 con 0x55add91e6a90 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881ac930 con 0x55add91eaea0 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881abb90 con 0x55add91ea580 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.075+0000 7ff99062e740 10 client.0 _ll_register_callbacks cb 0x7ff980029040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36236/0 (socket says 172.21.15.87:36236) 2024-06-06T07:59:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 -- 172.21.15.87:0/1069185996 learned_addr learned my addr 172.21.15.87:0/1069185996 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff9877fe640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add926e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff9867fc640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 -- 172.21.15.87:0/1069185996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 msgr2=0x55add91f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 -- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 msgr2=0x55add926e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add926e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 -- 172.21.15.87:0/1069185996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add926e790 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff9867fc640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff986ffd640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff9780bbed0 tx=0x7ff97808af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60fff59904e086f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978049040 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7ff9780b85b0 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9800098a0 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff978087090 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff99062e740 10 client.17277 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55add926f900 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.076+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9800098a0 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7ff9780f1040 con 0x55add91e6a90 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 10 client.17277 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 20 client.17277 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add91f5760 0x55add9272030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 4 client.17277 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55add9157c20 con 0x55add91f5760 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff9867fc640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add91f5760 0x55add9272030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff966ffd640 10 client.17277 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.077+0000 7ff9867fc640 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add91f5760 0x55add9272030 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f452b938f4ccd84 server_cookie=c7be16df02aa16d6 in_seq=0 out_seq=0 2024-06-06T07:59:32.108 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:31 smithi181 ceph-mon[36790]: pgmap v361: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 338 KiB/s wr, 54 op/s 2024-06-06T07:59:32.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:31 smithi087 ceph-mon[32276]: pgmap v361: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 338 KiB/s wr, 54 op/s 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff966ffd640 1 -- 172.21.15.87:0/1069185996 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55add9157c20 con 0x55add91f5760 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff966ffd640 10 client.17277 handle_command_reply: tid=0 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 client.17277 shutdown 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 msgr2=0x55add91f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff9780bbed0 tx=0x7ff97808af30 comp rx=0 tx=0).stop 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 shutdown_connections 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55add91f5760 0x55add9272030 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add91eaea0 0x55add926e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55add91ea580 0x55add91f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff966ffd640 0 client.17277 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 --2- 172.21.15.87:0/1069185996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55add91e6a90 0x55add91f8900 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.247+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 >> 172.21.15.87:0/1069185996 conn(0x55add9207e50 msgr2=0x55add91e0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 shutdown_connections 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff99062e740 1 -- 172.21.15.87:0/1069185996 wait complete. 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff99062e740 20 client.17277 trim_cache size 0 max 16384 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:59:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 msgr2=0x7ff9700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 0x7ff9700a5260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff980020000 tx=0x7ff98001d000 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 msgr2=0x7ff988198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff978003c10 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 shutdown_connections 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9700a2da0 0x7ff9700a5260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff988105850 0x7ff9881a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9881015b0 0x7ff9881994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 --2- 172.21.15.87:0/298696699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff988100bf0 0x7ff988198fb0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.248+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 >> 172.21.15.87:0/298696699 conn(0x7ff9880766a0 msgr2=0x7ff9880fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.249+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 shutdown_connections 2024-06-06T07:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:32.249+0000 7ff98d91b640 1 -- 172.21.15.87:0/298696699 wait complete. 2024-06-06T07:59:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:31 smithi195 ceph-mon[37040]: pgmap v361: 129 pgs: 129 active+clean; 77 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 338 KiB/s wr, 54 op/s 2024-06-06T07:59:32.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.08 MB/sec execute 456 sec latency 1725.323 ms 2024-06-06T07:59:33.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.07 MB/sec execute 457 sec latency 2725.372 ms 2024-06-06T07:59:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:33 smithi087 ceph-mon[32276]: pgmap v362: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1003 KiB/s wr, 112 op/s 2024-06-06T07:59:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:33 smithi195 ceph-mon[37040]: pgmap v362: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1003 KiB/s wr, 112 op/s 2024-06-06T07:59:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:33 smithi181 ceph-mon[36790]: pgmap v362: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1003 KiB/s wr, 112 op/s 2024-06-06T07:59:34.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.06 MB/sec execute 458 sec latency 3725.470 ms 2024-06-06T07:59:35.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.04 MB/sec execute 459 sec latency 4725.546 ms 2024-06-06T07:59:36.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:35 smithi087 ceph-mon[32276]: pgmap v363: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 89 op/s 2024-06-06T07:59:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:35 smithi195 ceph-mon[37040]: pgmap v363: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 89 op/s 2024-06-06T07:59:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:35 smithi181 ceph-mon[36790]: pgmap v363: 129 pgs: 129 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 89 op/s 2024-06-06T07:59:36.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.03 MB/sec execute 460 sec latency 5725.589 ms 2024-06-06T07:59:37.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.02 MB/sec execute 461 sec latency 6725.708 ms 2024-06-06T07:59:38.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:37 smithi087 ceph-mon[32276]: pgmap v364: 129 pgs: 129 active+clean; 81 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-06-06T07:59:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:37 smithi195 ceph-mon[37040]: pgmap v364: 129 pgs: 129 active+clean; 81 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-06-06T07:59:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:37 smithi181 ceph-mon[36790]: pgmap v364: 129 pgs: 129 active+clean; 81 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-06-06T07:59:38.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 6.01 MB/sec execute 462 sec latency 7725.827 ms 2024-06-06T07:59:39.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 5.99 MB/sec execute 463 sec latency 8725.880 ms 2024-06-06T07:59:40.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:39 smithi087 ceph-mon[32276]: pgmap v365: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-06-06T07:59:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:39 smithi195 ceph-mon[37040]: pgmap v365: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-06-06T07:59:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:39 smithi181 ceph-mon[36790]: pgmap v365: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-06-06T07:59:40.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 5.98 MB/sec execute 464 sec latency 9725.962 ms 2024-06-06T07:59:41.685 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 5.97 MB/sec execute 465 sec latency 10726.049 ms 2024-06-06T07:59:42.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:41 smithi087 ceph-mon[32276]: pgmap v366: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 823 KiB/s wr, 135 op/s 2024-06-06T07:59:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:41 smithi195 ceph-mon[37040]: pgmap v366: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 823 KiB/s wr, 135 op/s 2024-06-06T07:59:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:41 smithi181 ceph-mon[36790]: pgmap v366: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 823 KiB/s wr, 135 op/s 2024-06-06T07:59:42.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 635653 5.95 MB/sec execute 466 sec latency 11726.094 ms 2024-06-06T07:59:43.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 636860 5.95 MB/sec execute 467 sec latency 11903.467 ms 2024-06-06T07:59:44.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:43 smithi087 ceph-mon[32276]: pgmap v367: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 140 op/s 2024-06-06T07:59:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:43 smithi195 ceph-mon[37040]: pgmap v367: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 140 op/s 2024-06-06T07:59:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:43 smithi181 ceph-mon[36790]: pgmap v367: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 140 op/s 2024-06-06T07:59:44.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 639394 5.97 MB/sec execute 468 sec latency 15.443 ms 2024-06-06T07:59:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.342+0000 7fb03375a640 1 -- 172.21.15.87:0/2705054347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 msgr2=0x7fb02c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.342+0000 7fb03375a640 1 --2- 172.21.15.87:0/2705054347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c100d50 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009930 tx=0x7fb01c02a420 comp rx=0 tx=0).stop 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 -- 172.21.15.87:0/2705054347 shutdown_connections 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 --2- 172.21.15.87:0/2705054347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c101290 0x7fb02c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 --2- 172.21.15.87:0/2705054347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c100d50 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 --2- 172.21.15.87:0/2705054347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c106890 0x7fb02c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 -- 172.21.15.87:0/2705054347 >> 172.21.15.87:0/2705054347 conn(0x7fb02c077a20 msgr2=0x7fb02c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 -- 172.21.15.87:0/2705054347 shutdown_connections 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 -- 172.21.15.87:0/2705054347 wait complete. 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 Processor -- start 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 -- start start 2024-06-06T07:59:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.343+0000 7fb03375a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c101290 0x7fb02c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c106890 0x7fb02c1a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c110ca0 con 0x7fb02c1008d0 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c110b20 con 0x7fb02c101290 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c110e20 con 0x7fb02c106890 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55300/0 (socket says 172.21.15.87:55300) 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 -- 172.21.15.87:0/1951632227 learned_addr learned my addr 172.21.15.87:0/1951632227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb030cf5640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c101290 0x7fb02c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb031cf7640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c106890 0x7fb02c1a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 -- 172.21.15.87:0/1951632227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c106890 msgr2=0x7fb02c1a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c106890 0x7fb02c1a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 -- 172.21.15.87:0/1951632227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c101290 msgr2=0x7fb02c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c101290 0x7fb02c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 -- 172.21.15.87:0/1951632227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c1a4600 con 0x7fb02c1008d0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0314f6640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9f0 tx=0x7fb02800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0227fc640 1 -- 172.21.15.87:0/1951632227 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02800cd80 con 0x7fb02c1008d0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c1a47d0 con 0x7fb02c1008d0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c1a4d40 con 0x7fb02c1008d0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0227fc640 1 -- 172.21.15.87:0/1951632227 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb028010910 con 0x7fb02c1008d0 2024-06-06T07:59:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.344+0000 7fb0227fc640 1 -- 172.21.15.87:0/1951632227 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028018980 con 0x7fb02c1008d0 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb0227fc640 1 -- 172.21.15.87:0/1951632227 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb028010ac0 con 0x7fb02c1008d0 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 Processor -- start 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 -- start start 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56047694e4f0 con 0x5604769eaa90 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5604768ec870 con 0x5604769ea170 2024-06-06T07:59:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb036469740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56047694a4b0 con 0x5604769f5350 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb031cf7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb0314f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb031cf7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:46008/0 (socket says 172.21.15.87:46008) 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.347+0000 7fb031cf7640 1 -- 172.21.15.87:0/2367957634 learned_addr learned my addr 172.21.15.87:0/2367957634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 -- 172.21.15.87:0/2367957634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 msgr2=0x5604769f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 -- 172.21.15.87:0/2367957634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 msgr2=0x5604769f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 -- 172.21.15.87:0/2367957634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604769089b0 con 0x5604769ea170 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb0227fc640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 0x7fb0180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 0x7fb0180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb0227fc640 1 -- 172.21.15.87:0/1951632227 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7fb0280cfae0 con 0x7fb02c1008d0 2024-06-06T07:59:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb031cf7640 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb024010370 tx=0x7fb024010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=419a9db11564720e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb0177fe640 1 -- 172.21.15.87:0/2367957634 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024022070 con 0x5604769ea170 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb0177fe640 1 -- 172.21.15.87:0/2367957634 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240113e0 con 0x5604769ea170 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.348+0000 7fb0177fe640 1 -- 172.21.15.87:0/2367957634 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024016070 con 0x5604769ea170 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- 172.21.15.87:0/2367957634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 msgr2=0x5604769ea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb031cf7640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 0x7fb0180a5260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb02401f000 tx=0x7fb024022000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb024010370 tx=0x7fb024010830 comp rx=0 tx=0).stop 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- 172.21.15.87:0/2367957634 shutdown_connections 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 --2- 172.21.15.87:0/2367957634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769ea550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- 172.21.15.87:0/2367957634 >> 172.21.15.87:0/2367957634 conn(0x5604769e6680 msgr2=0x5604769e6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- 172.21.15.87:0/2367957634 shutdown_connections 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- 172.21.15.87:0/2367957634 wait complete. 2024-06-06T07:59:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.349+0000 7fb036469740 1 -- start start 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 Processor -- start 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5604769f7f80 con 0x5604769eaa90 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5604768d7ae0 con 0x5604769ea170 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5604769f8100 con 0x5604769f5350 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb036469740 10 client.0 _ll_register_callbacks cb 0x5604769109b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55312/0 (socket says 172.21.15.87:55312) 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 -- 172.21.15.87:0/3636809335 learned_addr learned my addr 172.21.15.87:0/3636809335 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb0314f6640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb031cf7640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 -- 172.21.15.87:0/3636809335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 msgr2=0x5604769f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 -- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 msgr2=0x5604769f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb030cf5640 1 -- 172.21.15.87:0/3636809335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560476662210 con 0x5604769eaa90 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.350+0000 7fb0314f6640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb030cf5640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb01c000c00 tx=0x7fb01c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862dcdb83987434 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c035040 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7fb01c004250 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0240113e0 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb01c051050 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 10 client.17289 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56047672e620 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240113e0 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb01c003800 con 0x5604769eaa90 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 10 client.17289 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 20 client.17289 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a05510 0x560476a079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 4 client.17289 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.351+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560476961f10 con 0x560476a05510 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.352+0000 7fb0314f6640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a05510 0x560476a079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.352+0000 7fb0314f6640 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a05510 0x560476a079a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885bd2d467700657 server_cookie=7b6bc7eb5477136b in_seq=0 out_seq=0 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.352+0000 7fb014ff9640 10 client.17289 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb014ff9640 1 -- 172.21.15.87:0/3636809335 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560476961f10 con 0x560476a05510 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb014ff9640 10 client.17289 handle_command_reply: tid=0 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 client.17289 shutdown 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 msgr2=0x5604769f27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb01c000c00 tx=0x7fb01c031920 comp rx=0 tx=0).stop 2024-06-06T07:59:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 shutdown_connections 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a05510 0x560476a079a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x5604769f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f27f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb014ff9640 0 client.17289 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 --2- 172.21.15.87:0/3636809335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x5604769f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 >> 172.21.15.87:0/3636809335 conn(0x5604769e6680 msgr2=0x5604769ec400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 shutdown_connections 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 1 -- 172.21.15.87:0/3636809335 wait complete. 2024-06-06T07:59:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.353+0000 7fb036469740 20 client.17289 trim_cache size 0 max 16384 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 Processor -- start 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 -- start start 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x560476a018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x560476a0d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100129c0 con 0x5604769eaa90 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb010010290 con 0x5604769ea170 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb036469740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb01000e100 con 0x5604769f5350 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55316/0 (socket says 172.21.15.87:55316) 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.369+0000 7fb030cf5640 1 -- 172.21.15.87:0/574056026 learned_addr learned my addr 172.21.15.87:0/574056026 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb0314f6640 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x560476a018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb031cf7640 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x560476a0d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 -- 172.21.15.87:0/574056026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 msgr2=0x560476a0d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x560476a0d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 -- 172.21.15.87:0/574056026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 msgr2=0x560476a018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x560476a018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 -- 172.21.15.87:0/574056026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560476a08020 con 0x5604769eaa90 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb030cf5640 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb01c003bd0 tx=0x7fb01c02ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4df7205402e5d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb016ffd640 1 -- 172.21.15.87:0/574056026 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c035070 con 0x5604769eaa90 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb016ffd640 1 -- 172.21.15.87:0/574056026 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01c003e40 con 0x5604769eaa90 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.370+0000 7fb016ffd640 1 -- 172.21.15.87:0/574056026 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c005280 con 0x5604769eaa90 2024-06-06T07:59:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- 172.21.15.87:0/574056026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 msgr2=0x5604769f41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb01c003bd0 tx=0x7fb01c02ab60 comp rx=0 tx=0).stop 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- 172.21.15.87:0/574056026 shutdown_connections 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769f5350 0x560476a0d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f41f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 --2- 172.21.15.87:0/574056026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769ea170 0x560476a018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- 172.21.15.87:0/574056026 >> 172.21.15.87:0/574056026 conn(0x5604769e6680 msgr2=0x5604769f3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- 172.21.15.87:0/574056026 shutdown_connections 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- 172.21.15.87:0/574056026 wait complete. 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 -- start start 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.371+0000 7fb036469740 1 Processor -- start 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769ea170 0x5604769fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 0x5604769f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb01000e750 con 0x5604769eaa90 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb01000fbc0 con 0x5604769f5350 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb01000ee20 con 0x5604769ea170 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb036469740 10 client.0 _ll_register_callbacks cb 0x7fb01c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55318/0 (socket says 172.21.15.87:55318) 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 -- 172.21.15.87:0/2565154435 learned_addr learned my addr 172.21.15.87:0/2565154435 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb0314f6640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769ea170 0x5604769fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb031cf7640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 0x5604769f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 -- 172.21.15.87:0/2565154435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769ea170 msgr2=0x5604769fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769ea170 0x5604769fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 -- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 msgr2=0x5604769f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 0x5604769f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 -- 172.21.15.87:0/2565154435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604769f96e0 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb031cf7640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 0x5604769f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb030cf5640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009850 tx=0x7fb01c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8202a61af065e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.372+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c035040 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9583+0+0 (secure 0 0 0) 0x7fb01c005230 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01c003e40 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb01c066050 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 10 client.17295 fetch_fsmap learned FSMap version 38 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560476a70000 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01c003e40 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb01c005430 con 0x5604769eaa90 2024-06-06T07:59:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 10 client.17295 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 20 client.17295 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a70680 0x560476a72aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 4 client.17295 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56047695bc20 con 0x560476a70680 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.373+0000 7fb0314f6640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a70680 0x560476a72aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.374+0000 7fb014ff9640 10 client.17295 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.374+0000 7fb0314f6640 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a70680 0x560476a72aa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e690a9472c740229 server_cookie=d114516c558d75d1 in_seq=0 out_seq=0 2024-06-06T07:59:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.497+0000 7fb014ff9640 1 -- 172.21.15.87:0/2565154435 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56047695bc20 con 0x560476a70680 2024-06-06T07:59:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.497+0000 7fb014ff9640 10 client.17295 handle_command_reply: tid=0 2024-06-06T07:59:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.497+0000 7fb036469740 1 client.17295 shutdown 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 msgr2=0x5604769f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009850 tx=0x7fb01c0038a0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 shutdown_connections 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560476a70680 0x560476a72aa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5604769f5350 0x5604769f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5604769eaa90 0x5604769f8be0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 --2- 172.21.15.87:0/2565154435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5604769ea170 0x5604769fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb014ff9640 0 client.17295 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 >> 172.21.15.87:0/2565154435 conn(0x5604769e6680 msgr2=0x5604769f2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 shutdown_connections 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 1 -- 172.21.15.87:0/2565154435 wait complete. 2024-06-06T07:59:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb036469740 20 client.17295 trim_cache size 0 max 16384 2024-06-06T07:59:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:59:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:59:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:59:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 msgr2=0x7fb0180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 0x7fb0180a5260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb02401f000 tx=0x7fb024022000 comp rx=0 tx=0).stop 2024-06-06T07:59:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 msgr2=0x7fb02c199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.498+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9f0 tx=0x7fb02800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 shutdown_connections 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0180a2da0 0x7fb0180a5260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb02c106890 0x7fb02c1a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb02c101290 0x7fb02c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 --2- 172.21.15.87:0/1951632227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb02c1008d0 0x7fb02c199040 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 >> 172.21.15.87:0/1951632227 conn(0x7fb02c077a20 msgr2=0x7fb02c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 shutdown_connections 2024-06-06T07:59:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:45.499+0000 7fb03375a640 1 -- 172.21.15.87:0/1951632227 wait complete. 2024-06-06T07:59:45.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 640815 5.97 MB/sec execute 469 sec latency 50.770 ms 2024-06-06T07:59:46.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:45 smithi087 ceph-mon[32276]: pgmap v368: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 142 KiB/s wr, 76 op/s 2024-06-06T07:59:46.200 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:45 smithi195 ceph-mon[37040]: pgmap v368: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 142 KiB/s wr, 76 op/s 2024-06-06T07:59:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:45 smithi181 ceph-mon[36790]: pgmap v368: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 142 KiB/s wr, 76 op/s 2024-06-06T07:59:46.509 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:59:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T07:59:46.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:59:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T07:59:46.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:59:46 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/2910544800 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557881497a40 con 0x55788146a800 2024-06-06T07:59:46.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 07:59:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T07:59:46.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 643127 5.99 MB/sec execute 470 sec latency 45.926 ms 2024-06-06T07:59:47.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:46 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T07:59:47.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:46 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T07:59:47.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:46 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T07:59:47.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 645000 5.99 MB/sec execute 471 sec latency 20.085 ms 2024-06-06T07:59:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: pgmap v369: 129 pgs: 129 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 688 KiB/s wr, 162 op/s 2024-06-06T07:59:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:59:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/643847133,v1:172.21.15.195:6837/643847133] up:boot 2024-06-06T07:59:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:59:48.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:59:48.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:47 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: pgmap v369: 129 pgs: 129 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 688 KiB/s wr, 162 op/s 2024-06-06T07:59:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:59:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/643847133,v1:172.21.15.195:6837/643847133] up:boot 2024-06-06T07:59:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:59:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:59:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:47 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: pgmap v369: 129 pgs: 129 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 688 KiB/s wr, 162 op/s 2024-06-06T07:59:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:59:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/643847133,v1:172.21.15.195:6837/643847133] up:boot 2024-06-06T07:59:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T07:59:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:59:48.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:47 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:48.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 646290 6.00 MB/sec execute 472 sec latency 584.077 ms 2024-06-06T07:59:49.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:48 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:49.285 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:48 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:49.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:48 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T07:59:49.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 646290 5.99 MB/sec execute 473 sec latency 1584.122 ms 2024-06-06T07:59:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:49 smithi195 ceph-mon[37040]: pgmap v371: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:50.265 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:49 smithi087 ceph-mon[32276]: pgmap v371: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:49 smithi181 ceph-mon[36790]: pgmap v371: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:50.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 646290 5.97 MB/sec execute 474 sec latency 2584.187 ms 2024-06-06T07:59:51.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 646290 5.96 MB/sec execute 475 sec latency 3584.269 ms 2024-06-06T07:59:52.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:51 smithi195 ceph-mon[37040]: pgmap v372: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:51 smithi181 ceph-mon[36790]: pgmap v372: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:51 smithi087 ceph-mon[32276]: pgmap v372: 129 pgs: 129 active+clean; 80 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 765 KiB/s wr, 139 op/s 2024-06-06T07:59:52.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 646290 5.95 MB/sec execute 476 sec latency 4584.384 ms 2024-06-06T07:59:53.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 647273 5.94 MB/sec execute 477 sec latency 4737.700 ms 2024-06-06T07:59:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:53 smithi195 ceph-mon[37040]: pgmap v373: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:53 smithi181 ceph-mon[36790]: pgmap v373: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:53 smithi087 ceph-mon[32276]: pgmap v373: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:54.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 649837 5.96 MB/sec execute 478 sec latency 20.061 ms 2024-06-06T07:59:55.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 649837 5.95 MB/sec execute 479 sec latency 1020.098 ms 2024-06-06T07:59:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:55 smithi195 ceph-mon[37040]: pgmap v374: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:55 smithi181 ceph-mon[36790]: pgmap v374: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:55 smithi087 ceph-mon[32276]: pgmap v374: 129 pgs: 129 active+clean; 82 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-06-06T07:59:56.686 INFO:tasks.workunit.client.0.smithi087.stdout: 1 649964 5.94 MB/sec execute 480 sec latency 1467.915 ms 2024-06-06T07:59:57.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 649964 5.93 MB/sec execute 481 sec latency 1481.277 ms 2024-06-06T07:59:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:57 smithi195 ceph-mon[37040]: pgmap v375: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 100 op/s 2024-06-06T07:59:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:57 smithi181 ceph-mon[36790]: pgmap v375: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 100 op/s 2024-06-06T07:59:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:57 smithi087 ceph-mon[32276]: pgmap v375: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 900 KiB/s wr, 100 op/s 2024-06-06T07:59:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 -- 172.21.15.87:0/3052457792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9410bf70 msgr2=0x7f6c9410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/3052457792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9410bf70 0x7f6c9410e410 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009ca0 tx=0x7f6c8c02a9e0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 -- 172.21.15.87:0/3052457792 shutdown_connections 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/3052457792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c9410bf70 0x7f6c9410e410 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/3052457792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c9410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/3052457792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c94102c30 0x7f6c94103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 -- 172.21.15.87:0/3052457792 >> 172.21.15.87:0/3052457792 conn(0x7f6c94077a20 msgr2=0x7f6c94077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 -- 172.21.15.87:0/3052457792 shutdown_connections 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.594+0000 7f6c98b9b640 1 -- 172.21.15.87:0/3052457792 wait complete. 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 Processor -- start 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 -- start start 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c941986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 0x7f6c9419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94111d70 con 0x7f6c94102c30 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94111bf0 con 0x7f6c94103550 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c98b9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94111ef0 con 0x7f6c9410bf70 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39744/0 (socket says 172.21.15.87:39744) 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.595+0000 7f6c9259c640 1 -- 172.21.15.87:0/205203618 learned_addr learned my addr 172.21.15.87:0/205203618 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 0x7f6c9419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c941986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 -- 172.21.15.87:0/205203618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 msgr2=0x7f6c9419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 0x7f6c9419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 -- 172.21.15.87:0/205203618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 msgr2=0x7f6c941986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c941986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 -- 172.21.15.87:0/205203618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9419d160 con 0x7f6c94102c30 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 0x7f6c9419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c941986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c9259c640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400d950 tx=0x7f6c8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c8b7fe640 1 -- 172.21.15.87:0/205203618 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017070 con 0x7f6c94102c30 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c8b7fe640 1 -- 172.21.15.87:0/205203618 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c84009c30 con 0x7f6c94102c30 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c8b7fe640 1 -- 172.21.15.87:0/205203618 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400f6b0 con 0x7f6c94102c30 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9419d450 con 0x7f6c94102c30 2024-06-06T07:59:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.596+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c941a4ce0 con 0x7f6c94102c30 2024-06-06T07:59:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c8b7fe640 1 -- 172.21.15.87:0/205203618 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6c840109d0 con 0x7f6c94102c30 2024-06-06T07:59:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 Processor -- start 2024-06-06T07:59:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 -- start start 2024-06-06T07:59:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c281303430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 0x55c281312220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.599+0000 7f6c9b983740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c2812674f0 con 0x55c281312760 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c9b983740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c281205870 con 0x55c281303970 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c9b983740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2812634b0 con 0x55c281303050 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8b7fe640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 0x7f6c800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39756/0 (socket says 172.21.15.87:39756) 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 -- 172.21.15.87:0/2432960715 learned_addr learned my addr 172.21.15.87:0/2432960715 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c9259c640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 0x55c281312220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c281303430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 0x7f6c800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8b7fe640 1 -- 172.21.15.87:0/205203618 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9495+0+0 (secure 0 0 0) 0x7f6c840cf460 con 0x7f6c94102c30 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 -- 172.21.15.87:0/2432960715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 msgr2=0x55c281303430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c281303430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 -- 172.21.15.87:0/2432960715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 msgr2=0x55c281312220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 0x55c281312220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 -- 172.21.15.87:0/2432960715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2812219b0 con 0x55c281312760 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c281303430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c9259c640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 0x55c281312220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009bd0 tx=0x7f6c7c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6756218cf2ca6a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 0x7f6c800a52e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005fa0 tx=0x7f6c8c0095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c88ff9640 1 -- 172.21.15.87:0/2432960715 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c016070 con 0x55c281312760 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.600+0000 7f6c88ff9640 1 -- 172.21.15.87:0/2432960715 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c011950 con 0x55c281312760 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c88ff9640 1 -- 172.21.15.87:0/2432960715 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0109a0 con 0x55c281312760 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 -- 172.21.15.87:0/2432960715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 msgr2=0x55c281314b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009bd0 tx=0x7f6c7c00e6c0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 -- 172.21.15.87:0/2432960715 shutdown_connections 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c281314b50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303970 0x55c281312220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 --2- 172.21.15.87:0/2432960715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c281303430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 -- 172.21.15.87:0/2432960715 >> 172.21.15.87:0/2432960715 conn(0x55c2812ff680 msgr2=0x55c2812ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 -- 172.21.15.87:0/2432960715 shutdown_connections 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.601+0000 7f6c9b983740 1 -- 172.21.15.87:0/2432960715 wait complete. 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 -- start start 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 Processor -- start 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c28130f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303970 0x55c28130fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c281315390 con 0x55c281312760 2024-06-06T07:59:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c2811f0ae0 con 0x55c281303050 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c281315510 con 0x55c281303970 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9b983740 10 client.0 _ll_register_callbacks cb 0x55c2812299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39758/0 (socket says 172.21.15.87:39758) 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c92d9d640 1 -- 172.21.15.87:0/884336947 learned_addr learned my addr 172.21.15.87:0/884336947 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303970 0x55c28130fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.602+0000 7f6c9259c640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c28130f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 -- 172.21.15.87:0/884336947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303970 msgr2=0x55c28130fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303970 0x55c28130fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 -- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 msgr2=0x55c28130f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c28130f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 -- 172.21.15.87:0/884336947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c280f7b210 con 0x55c281312760 2024-06-06T07:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c003b80 tx=0x7f6c8c02a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48196727d759ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c033040 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9495+0+0 (secure 0 0 0) 0x7f6c8c004250 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c011950 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6c8c004450 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c9b983740 10 client.17310 fetch_fsmap learned FSMap version 41 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c281047620 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c011950 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f6c8c04f020 con 0x55c281312760 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c9b983740 10 client.17310 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.603+0000 7f6c9b983740 10 client.17310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9b983740 20 client.17310 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c2813228a0 0x55c281324cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9b983740 4 client.17310 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28127af10 con 0x55c2813228a0 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9259c640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c2813228a0 0x55c281324cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c79ffb640 10 client.17310 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.604+0000 7f6c9259c640 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c2813228a0 0x55c281324cf0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276706061bdd7b73 server_cookie=dea6fee7d0b9dbf0 in_seq=0 out_seq=0 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c79ffb640 1 -- 172.21.15.87:0/884336947 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c28127af10 con 0x55c2813228a0 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c79ffb640 10 client.17310 handle_command_reply: tid=0 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 client.17310 shutdown 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 msgr2=0x55c2813100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c003b80 tx=0x7f6c8c02a9e0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 shutdown_connections 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c2813228a0 0x55c281324cf0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281312760 0x55c2813100a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c79ffb640 0 client.17310 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303970 0x55c28130fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 --2- 172.21.15.87:0/884336947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c28130f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 >> 172.21.15.87:0/884336947 conn(0x55c2812ff680 msgr2=0x55c2813054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 shutdown_connections 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 1 -- 172.21.15.87:0/884336947 wait complete. 2024-06-06T07:59:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.605+0000 7f6c9b983740 20 client.17310 trim_cache size 0 max 16384 2024-06-06T07:59:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.620+0000 7f6c9b983740 1 Processor -- start 2024-06-06T07:59:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.620+0000 7f6c9b983740 1 -- start start 2024-06-06T07:59:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c2812fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 0x55c28132ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941af5a0 con 0x55c281303970 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ace70 con 0x55c281312760 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c9b983740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941aacc0 con 0x55c281303050 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39772/0 (socket says 172.21.15.87:39772) 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 -- 172.21.15.87:0/4009415399 learned_addr learned my addr 172.21.15.87:0/4009415399 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 0x55c28132ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 -- 172.21.15.87:0/4009415399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 msgr2=0x55c2812fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c2812fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 -- 172.21.15.87:0/4009415399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 msgr2=0x55c28132ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 0x55c28132ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 -- 172.21.15.87:0/4009415399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28130a3f0 con 0x55c281303970 2024-06-06T07:59:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 0x55c28132ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009bd0 tx=0x7f6c7c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe57528e349a449 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c7bfff640 1 -- 172.21.15.87:0/4009415399 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c02a070 con 0x55c281303970 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c7bfff640 1 -- 172.21.15.87:0/4009415399 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c005520 con 0x55c281303970 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.621+0000 7f6c7bfff640 1 -- 172.21.15.87:0/4009415399 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c016040 con 0x55c281303970 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- 172.21.15.87:0/4009415399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 msgr2=0x55c28132a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009bd0 tx=0x7f6c7c005f30 comp rx=0 tx=0).stop 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- 172.21.15.87:0/4009415399 shutdown_connections 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281312760 0x55c28132ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c28132a600 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- 172.21.15.87:0/4009415399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281303050 0x55c2812fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- 172.21.15.87:0/4009415399 >> 172.21.15.87:0/4009415399 conn(0x55c2812ff680 msgr2=0x55c2812fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- 172.21.15.87:0/4009415399 shutdown_connections 2024-06-06T07:59:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- 172.21.15.87:0/4009415399 wait complete. 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- start start 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 Processor -- start 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c281316650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 0x55c2813170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ab330 con 0x55c281303970 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ac7a0 con 0x55c281303050 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.622+0000 7f6c9b983740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941aba00 con 0x55c281312760 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c9b983740 10 client.0 _ll_register_callbacks cb 0x7f6c8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39788/0 (socket says 172.21.15.87:39788) 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 -- 172.21.15.87:0/757420524 learned_addr learned my addr 172.21.15.87:0/757420524 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c9259c640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c281316650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 0x55c2813170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 -- 172.21.15.87:0/757420524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 msgr2=0x55c2813170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 0x55c2813170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 -- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 msgr2=0x55c281316650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c281316650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 -- 172.21.15.87:0/757420524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28138c680 con 0x55c281303970 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c92d9d640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 0x55c2813170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c8bfff640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c004ae0 tx=0x7f6c7c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a854de0a163655e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c032070 con 0x55c281303970 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9495+0+0 (secure 0 0 0) 0x7f6c7c004020 con 0x55c281303970 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c005520 con 0x55c281303970 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6c7c03b050 con 0x55c281303970 2024-06-06T07:59:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c9b983740 10 client.17316 fetch_fsmap learned FSMap version 41 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.623+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c28138d8c0 con 0x55c281303970 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c005520 con 0x55c281303970 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f6c7c035020 con 0x55c281303970 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 10 client.17316 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 20 client.17316 populate_metadata read hostname 'smithi087' 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c28138ded0 0x55c2813902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 4 client.17316 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c281274c20 con 0x55c28138ded0 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9259c640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c28138ded0 0x55c2813902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c9259c640 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c28138ded0 0x55c2813902f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d070f35c8d321489 server_cookie=e70511e45fa29702 in_seq=0 out_seq=0 2024-06-06T07:59:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.624+0000 7f6c79ffb640 10 client.17316 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:58.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 650031 5.92 MB/sec execute 482 sec latency 1637.226 ms 2024-06-06T07:59:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c79ffb640 1 -- 172.21.15.87:0/757420524 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c281274c20 con 0x55c28138ded0 2024-06-06T07:59:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c79ffb640 10 client.17316 handle_command_reply: tid=0 2024-06-06T07:59:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 client.17316 shutdown 2024-06-06T07:59:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 msgr2=0x55c281316b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c004ae0 tx=0x7f6c7c0048b0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 shutdown_connections 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c28138ded0 0x55c2813902f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c281312760 0x55c2813170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c281303970 0x55c281316b90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c79ffb640 0 client.17316 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 --2- 172.21.15.87:0/757420524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c281303050 0x55c281316650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 >> 172.21.15.87:0/757420524 conn(0x55c2812ff680 msgr2=0x55c2812fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 shutdown_connections 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 1 -- 172.21.15.87:0/757420524 wait complete. 2024-06-06T07:59:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c9b983740 20 client.17316 trim_cache size 0 max 16384 2024-06-06T07:59:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T07:59:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T07:59:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.759+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 msgr2=0x7f6c800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 0x7f6c800a52e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005fa0 tx=0x7f6c8c0095c0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 msgr2=0x7f6c94198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400d950 tx=0x7f6c8400de10 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 shutdown_connections 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c800a2e20 0x7f6c800a52e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c9410bf70 0x7f6c9419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c94103550 0x7f6c941986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 --2- 172.21.15.87:0/205203618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c94102c30 0x7f6c94198170 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:59:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 >> 172.21.15.87:0/205203618 conn(0x7f6c94077a20 msgr2=0x7f6c940fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:59:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 shutdown_connections 2024-06-06T07:59:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T07:59:58.760+0000 7f6c98b9b640 1 -- 172.21.15.87:0/205203618 wait complete. 2024-06-06T07:59:59.687 INFO:tasks.workunit.client.0.smithi087.stdout: 1 651397 5.92 MB/sec execute 483 sec latency 805.589 ms 2024-06-06T07:59:59.999 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 07:59:59 smithi087 ceph-mon[32276]: pgmap v376: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 889 KiB/s wr, 64 op/s 2024-06-06T08:00:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 07:59:59 smithi195 ceph-mon[37040]: pgmap v376: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 889 KiB/s wr, 64 op/s 2024-06-06T08:00:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 07:59:59 smithi181 ceph-mon[36790]: pgmap v376: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 889 KiB/s wr, 64 op/s 2024-06-06T08:00:00.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T07:59:59.999+0000 7f61096e0700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:00:00.329 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T07:59:59.999+0000 7f61096e0700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T08:00:00.329 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T07:59:59.999+0000 7f61096e0700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-06-06T08:00:00.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653133 5.92 MB/sec execute 484 sec latency 70.945 ms 2024-06-06T08:00:01.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:00 smithi195 ceph-mon[37040]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:00:01.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:00 smithi195 ceph-mon[37040]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T08:00:01.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:00 smithi195 ceph-mon[37040]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:00:01.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:00 smithi181 ceph-mon[36790]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:00:01.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:00 smithi181 ceph-mon[36790]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T08:00:01.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:00 smithi181 ceph-mon[36790]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:00:01.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-mon[32276]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:00:01.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-mon[32276]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T08:00:01.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:00 smithi087 ceph-mon[32276]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:00:01.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653146 5.91 MB/sec execute 485 sec latency 599.170 ms 2024-06-06T08:00:02.161 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:01 smithi195 ceph-mon[37040]: pgmap v377: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 871 KiB/s wr, 62 op/s 2024-06-06T08:00:02.161 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:01 smithi195 ceph-mon[37040]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:00:02.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:01 smithi181 ceph-mon[36790]: pgmap v377: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 871 KiB/s wr, 62 op/s 2024-06-06T08:00:02.207 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:01 smithi181 ceph-mon[36790]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:00:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:01 smithi087 ceph-mon[32276]: pgmap v377: 129 pgs: 129 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 871 KiB/s wr, 62 op/s 2024-06-06T08:00:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:01 smithi087 ceph-mon[32276]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:00:02.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653148 5.90 MB/sec execute 486 sec latency 672.429 ms 2024-06-06T08:00:03.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653509 5.89 MB/sec execute 487 sec latency 278.580 ms 2024-06-06T08:00:04.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:03 smithi195 ceph-mon[37040]: pgmap v379: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:03 smithi181 ceph-mon[36790]: pgmap v379: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:03 smithi087 ceph-mon[32276]: pgmap v379: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:04.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653509 5.88 MB/sec execute 488 sec latency 1131.981 ms 2024-06-06T08:00:05.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653509 5.87 MB/sec execute 489 sec latency 2132.044 ms 2024-06-06T08:00:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:05 smithi195 ceph-mon[37040]: pgmap v380: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:05 smithi181 ceph-mon[36790]: pgmap v380: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:05 smithi087 ceph-mon[32276]: pgmap v380: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-06-06T08:00:06.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653509 5.86 MB/sec execute 490 sec latency 3132.178 ms 2024-06-06T08:00:07.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653509 5.85 MB/sec execute 491 sec latency 4132.221 ms 2024-06-06T08:00:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:07 smithi195 ceph-mon[37040]: pgmap v381: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 91 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-06-06T08:00:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:07 smithi181 ceph-mon[36790]: pgmap v381: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 91 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-06-06T08:00:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:07 smithi087 ceph-mon[32276]: pgmap v381: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 91 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-06-06T08:00:08.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653576 5.83 MB/sec execute 492 sec latency 4286.224 ms 2024-06-06T08:00:09.688 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653576 5.82 MB/sec execute 493 sec latency 1742.391 ms 2024-06-06T08:00:10.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:09 smithi195 ceph-mon[37040]: pgmap v382: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-06-06T08:00:10.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:09 smithi195 ceph-mon[37040]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T08:00:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:09 smithi181 ceph-mon[36790]: pgmap v382: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-06-06T08:00:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:09 smithi181 ceph-mon[36790]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T08:00:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:09 smithi087 ceph-mon[32276]: pgmap v382: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-06-06T08:00:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:09 smithi087 ceph-mon[32276]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T08:00:10.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653576 5.81 MB/sec execute 494 sec latency 2742.477 ms 2024-06-06T08:00:11.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653576 5.80 MB/sec execute 495 sec latency 3742.628 ms 2024-06-06T08:00:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 -- 172.21.15.87:0/746698381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff768105370 msgr2=0x7ff76810cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 --2- 172.21.15.87:0/746698381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff768105370 0x7ff76810cf40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff760009ca0 tx=0x7ff76002a8c0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 -- 172.21.15.87:0/746698381 shutdown_connections 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 --2- 172.21.15.87:0/746698381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff768105370 0x7ff76810cf40 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 --2- 172.21.15.87:0/746698381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff768100c50 0x7ff768104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 --2- 172.21.15.87:0/746698381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7681002a0 0x7ff768100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 -- 172.21.15.87:0/746698381 >> 172.21.15.87:0/746698381 conn(0x7ff768076e00 msgr2=0x7ff7680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 -- 172.21.15.87:0/746698381 shutdown_connections 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.857+0000 7ff76f631640 1 -- 172.21.15.87:0/746698381 wait complete. 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 Processor -- start 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 -- start start 2024-06-06T08:00:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 0x7ff768198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 0x7ff76819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff768110ca0 con 0x7ff7681002a0 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff768110b20 con 0x7ff768105370 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76f631640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff768110e20 con 0x7ff76819c9f0 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59724/0 (socket says 172.21.15.87:59724) 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.858+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2629341923 learned_addr learned my addr 172.21.15.87:0/2629341923 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 0x7ff76819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 0x7ff768198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2629341923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 msgr2=0x7ff76819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 0x7ff76819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2629341923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 msgr2=0x7ff768198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 0x7ff768198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2629341923 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76819d4e0 con 0x7ff7681002a0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 0x7ff76819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 0x7ff768198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c920 tx=0x7ff76400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff75e7fc640 1 -- 172.21.15.87:0/2629341923 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764007c70 con 0x7ff7681002a0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff75e7fc640 1 -- 172.21.15.87:0/2629341923 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff764007e10 con 0x7ff7681002a0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76819d7d0 con 0x7ff7681002a0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff75e7fc640 1 -- 172.21.15.87:0/2629341923 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764018890 con 0x7ff7681002a0 2024-06-06T08:00:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.859+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681a4fc0 con 0x7ff7681002a0 2024-06-06T08:00:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff75e7fc640 1 -- 172.21.15.87:0/2629341923 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff764018a70 con 0x7ff7681002a0 2024-06-06T08:00:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 Processor -- start 2024-06-06T08:00:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 -- start start 2024-06-06T08:00:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251a90 0x55bfcb25be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb25e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.862+0000 7ff772418740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb1b54f0 con 0x55bfcb251170 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff772418740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb153870 con 0x55bfcb25c350 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff772418740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb1b14b0 con 0x55bfcb251a90 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff75e7fc640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 0x7ff7540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59726/0 (socket says 172.21.15.87:59726) 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 -- 172.21.15.87:0/1446383478 learned_addr learned my addr 172.21.15.87:0/1446383478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251a90 0x55bfcb25be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 0x7ff7540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb25e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff75e7fc640 1 -- 172.21.15.87:0/2629341923 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff7640cfa70 con 0x7ff7681002a0 2024-06-06T08:00:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 -- 172.21.15.87:0/1446383478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251a90 msgr2=0x55bfcb25be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251a90 0x55bfcb25be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 -- 172.21.15.87:0/1446383478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 msgr2=0x55bfcb25e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb25e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 -- 172.21.15.87:0/1446383478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfcb16f9b0 con 0x55bfcb251170 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff760035000 tx=0x7ff760031780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d0999224e2d099 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 0x7ff7540a5260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff760009a60 tx=0x7ff760048000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.863+0000 7ff7537fe640 1 -- 172.21.15.87:0/1446383478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76004e070 con 0x55bfcb251170 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff7537fe640 1 -- 172.21.15.87:0/1446383478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff760036040 con 0x55bfcb251170 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff7537fe640 1 -- 172.21.15.87:0/1446383478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760030a60 con 0x55bfcb251170 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 -- 172.21.15.87:0/1446383478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 msgr2=0x55bfcb251550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff760035000 tx=0x7ff760031780 comp rx=0 tx=0).stop 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 -- 172.21.15.87:0/1446383478 shutdown_connections 2024-06-06T08:00:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb25e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251a90 0x55bfcb25be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 --2- 172.21.15.87:0/1446383478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb251550 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 -- 172.21.15.87:0/1446383478 >> 172.21.15.87:0/1446383478 conn(0x55bfcb24d680 msgr2=0x55bfcb24da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 -- 172.21.15.87:0/1446383478 shutdown_connections 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.864+0000 7ff772418740 1 -- 172.21.15.87:0/1446383478 wait complete. 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 -- start start 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 Processor -- start 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb259860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb259da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb25ef20 con 0x55bfcb251170 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb13eae0 con 0x55bfcb251a90 2024-06-06T08:00:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bfcb25f0a0 con 0x55bfcb25c350 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff772418740 10 client.0 _ll_register_callbacks cb 0x55bfcb1779b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59732/0 (socket says 172.21.15.87:59732) 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff76d3cd640 1 -- 172.21.15.87:0/860179111 learned_addr learned my addr 172.21.15.87:0/860179111 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.865+0000 7ff76dbce640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb259da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb259860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 -- 172.21.15.87:0/860179111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 msgr2=0x55bfcb259da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb259da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 -- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 msgr2=0x55bfcb259860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb259860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 -- 172.21.15.87:0/860179111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfcaec9210 con 0x55bfcb251170 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff764097fd0 tx=0x7ff76406aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfd55dba42ce295 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76400de20 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff7640981e0 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff760036040 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff7640b9050 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff772418740 10 client.17328 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfcaf95620 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff760036040 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7ff76409bdd0 con 0x55bfcb251170 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff772418740 10 client.17328 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.866+0000 7ff772418740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff772418740 20 client.17328 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb26c5b0 0x55bfcb26ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff772418740 4 client.17328 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfcb1c8f10 con 0x55bfcb26c5b0 2024-06-06T08:00:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb26c5b0 0x55bfcb26ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff750ff9640 10 client.17328 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.867+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb26c5b0 0x55bfcb26ea20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5074dd720f1ab214 server_cookie=47cedf919e3564d0 in_seq=0 out_seq=0 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff750ff9640 1 -- 172.21.15.87:0/860179111 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bfcb1c8f10 con 0x55bfcb26c5b0 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff750ff9640 10 client.17328 handle_command_reply: tid=0 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 client.17328 shutdown 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 msgr2=0x55bfcb259320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff764097fd0 tx=0x7ff76406aaa0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 shutdown_connections 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb26c5b0 0x55bfcb26ea20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb259da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb259860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 --2- 172.21.15.87:0/860179111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb259320 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff750ff9640 0 client.17328 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 >> 172.21.15.87:0/860179111 conn(0x55bfcb24d680 msgr2=0x55bfcb2533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 shutdown_connections 2024-06-06T08:00:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 1 -- 172.21.15.87:0/860179111 wait complete. 2024-06-06T08:00:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.868+0000 7ff772418740 20 client.17328 trim_cache size 0 max 16384 2024-06-06T08:00:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.883+0000 7ff772418740 1 Processor -- start 2024-06-06T08:00:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.883+0000 7ff772418740 1 -- start start 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb2646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb264c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b1870 con 0x55bfcb251170 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681af140 con 0x55bfcb251a90 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff772418740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681acfb0 con 0x55bfcb25c350 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59738/0 (socket says 172.21.15.87:59738) 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2197638123 learned_addr learned my addr 172.21.15.87:0/2197638123 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2197638123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 msgr2=0x55bfcb264c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb264c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb264c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb2646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2197638123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 msgr2=0x55bfcb2646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb2646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 -- 172.21.15.87:0/2197638123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfcb2582d0 con 0x55bfcb251170 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76dbce640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb264c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb2646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff76409bb00 tx=0x7ff764000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2917ed6b224159 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff752ffd640 1 -- 172.21.15.87:0/2197638123 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764064070 con 0x55bfcb251170 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff752ffd640 1 -- 172.21.15.87:0/2197638123 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff76409bb70 con 0x55bfcb251170 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.884+0000 7ff752ffd640 1 -- 172.21.15.87:0/2197638123 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b3030 con 0x55bfcb251170 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- 172.21.15.87:0/2197638123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 msgr2=0x55bfcb25b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff76409bb00 tx=0x7ff764000bc0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- 172.21.15.87:0/2197638123 shutdown_connections 2024-06-06T08:00:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb25c350 0x55bfcb264c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb251a90 0x55bfcb2646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 --2- 172.21.15.87:0/2197638123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251170 0x55bfcb25b3e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- 172.21.15.87:0/2197638123 >> 172.21.15.87:0/2197638123 conn(0x55bfcb24d680 msgr2=0x55bfcb24da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- 172.21.15.87:0/2197638123 shutdown_connections 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- 172.21.15.87:0/2197638123 wait complete. 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 -- start start 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.885+0000 7ff772418740 1 Processor -- start 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251170 0x55bfcb2cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb2da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681ad600 con 0x55bfcb251a90 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681aea70 con 0x55bfcb25c350 2024-06-06T08:00:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681adcd0 con 0x55bfcb251170 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 10 client.0 _ll_register_callbacks cb 0x7ff76400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59754/0 (socket says 172.21.15.87:59754) 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 -- 172.21.15.87:0/1520989247 learned_addr learned my addr 172.21.15.87:0/1520989247 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251170 0x55bfcb2cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76dbce640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb2da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 -- 172.21.15.87:0/1520989247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251170 msgr2=0x55bfcb2cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251170 0x55bfcb2cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 -- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 msgr2=0x55bfcb2da2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb2da2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 -- 172.21.15.87:0/1520989247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfcb25eab0 con 0x55bfcb251a90 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76dbce640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb2da2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff76cbcc640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff758009bd0 tx=0x7ff758012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1237aa0fc01dacf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75801b4c0 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff76409bb70 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.886+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff75801b660 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff75802c050 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 10 client.17334 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfcb2dba20 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff76409bb70 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7ff75800f040 con 0x55bfcb251a90 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 10 client.17334 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 20 client.17334 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb2dc220 0x55bfcb2de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 4 client.17334 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfcb1c2c20 con 0x55bfcb2dc220 2024-06-06T08:00:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.887+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb2dc220 0x55bfcb2de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.888+0000 7ff750ff9640 10 client.17334 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.888+0000 7ff76d3cd640 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb2dc220 0x55bfcb2de640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215a6c7169a1d12c server_cookie=b545506c5e338bfe in_seq=0 out_seq=0 2024-06-06T08:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.978+0000 7ff750ff9640 1 -- 172.21.15.87:0/1520989247 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bfcb1c2c20 con 0x55bfcb2dc220 2024-06-06T08:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.978+0000 7ff750ff9640 10 client.17334 handle_command_reply: tid=0 2024-06-06T08:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.978+0000 7ff772418740 1 client.17334 shutdown 2024-06-06T08:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 msgr2=0x55bfcb2cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff758009bd0 tx=0x7ff758012820 comp rx=0 tx=0).stop 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 shutdown_connections 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bfcb2dc220 0x55bfcb2de640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfcb25c350 0x55bfcb2da2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfcb251a90 0x55bfcb2cf9a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff750ff9640 0 client.17334 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 --2- 172.21.15.87:0/1520989247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bfcb251170 0x55bfcb2cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 >> 172.21.15.87:0/1520989247 conn(0x55bfcb24d680 msgr2=0x55bfcb24c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 shutdown_connections 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 1 -- 172.21.15.87:0/1520989247 wait complete. 2024-06-06T08:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.979+0000 7ff772418740 20 client.17334 trim_cache size 0 max 16384 2024-06-06T08:00:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:00:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:00:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 msgr2=0x7ff7540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 0x7ff7540a5260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff760009a60 tx=0x7ff760048000 comp rx=0 tx=0).stop 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 msgr2=0x7ff768198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c920 tx=0x7ff76400cde0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 shutdown_connections 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff7540a2da0 0x7ff7540a5260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff76819c9f0 0x7ff76819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff768105370 0x7ff768198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 --2- 172.21.15.87:0/2629341923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7681002a0 0x7ff768198130 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 >> 172.21.15.87:0/2629341923 conn(0x7ff768076e00 msgr2=0x7ff76810da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 shutdown_connections 2024-06-06T08:00:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:11.980+0000 7ff76f631640 1 -- 172.21.15.87:0/2629341923 wait complete. 2024-06-06T08:00:12.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:11 smithi195 ceph-mon[37040]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-06T08:00:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:11 smithi181 ceph-mon[36790]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-06T08:00:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:11 smithi087 ceph-mon[32276]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-06T08:00:12.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 653576 5.79 MB/sec execute 496 sec latency 4742.728 ms 2024-06-06T08:00:13.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 654628 5.78 MB/sec execute 497 sec latency 4893.178 ms 2024-06-06T08:00:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:13 smithi195 ceph-mon[37040]: pgmap v385: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:13 smithi181 ceph-mon[36790]: pgmap v385: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:13 smithi087 ceph-mon[32276]: pgmap v385: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:14.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 656990 5.80 MB/sec execute 498 sec latency 21.155 ms 2024-06-06T08:00:15.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 656990 5.79 MB/sec execute 499 sec latency 1021.255 ms 2024-06-06T08:00:16.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:15 smithi195 ceph-mon[37040]: pgmap v386: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:15 smithi181 ceph-mon[36790]: pgmap v386: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:15 smithi087 ceph-mon[32276]: pgmap v386: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 54 op/s 2024-06-06T08:00:16.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 656990 5.78 MB/sec execute 500 sec latency 2021.367 ms 2024-06-06T08:00:17.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 656990 5.77 MB/sec execute 501 sec latency 3021.483 ms 2024-06-06T08:00:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:17 smithi195 ceph-mon[37040]: pgmap v387: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 79 op/s 2024-06-06T08:00:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:17 smithi181 ceph-mon[36790]: pgmap v387: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 79 op/s 2024-06-06T08:00:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:17 smithi087 ceph-mon[32276]: pgmap v387: 129 pgs: 129 active+clean; 82 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 79 op/s 2024-06-06T08:00:18.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657054 5.76 MB/sec execute 502 sec latency 3201.973 ms 2024-06-06T08:00:19.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657054 5.74 MB/sec execute 503 sec latency 1802.019 ms 2024-06-06T08:00:20.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:19 smithi087 ceph-mon[32276]: pgmap v388: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 772 KiB/s wr, 78 op/s 2024-06-06T08:00:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:19 smithi195 ceph-mon[37040]: pgmap v388: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 772 KiB/s wr, 78 op/s 2024-06-06T08:00:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:19 smithi181 ceph-mon[36790]: pgmap v388: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 772 KiB/s wr, 78 op/s 2024-06-06T08:00:20.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657054 5.73 MB/sec execute 504 sec latency 2802.122 ms 2024-06-06T08:00:21.689 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657054 5.72 MB/sec execute 505 sec latency 3802.165 ms 2024-06-06T08:00:22.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:21 smithi195 ceph-mon[37040]: pgmap v389: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 715 KiB/s wr, 72 op/s 2024-06-06T08:00:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:21 smithi181 ceph-mon[36790]: pgmap v389: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 715 KiB/s wr, 72 op/s 2024-06-06T08:00:22.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:21 smithi087 ceph-mon[32276]: pgmap v389: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 715 KiB/s wr, 72 op/s 2024-06-06T08:00:22.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657054 5.71 MB/sec execute 506 sec latency 4802.236 ms 2024-06-06T08:00:23.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657121 5.70 MB/sec execute 507 sec latency 4963.259 ms 2024-06-06T08:00:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:23 smithi195 ceph-mon[37040]: pgmap v390: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 856 KiB/s wr, 90 op/s 2024-06-06T08:00:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:23 smithi181 ceph-mon[36790]: pgmap v390: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 856 KiB/s wr, 90 op/s 2024-06-06T08:00:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:23 smithi087 ceph-mon[32276]: pgmap v390: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 856 KiB/s wr, 90 op/s 2024-06-06T08:00:24.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657121 5.69 MB/sec execute 508 sec latency 1736.573 ms 2024-06-06T08:00:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 -- 172.21.15.87:0/82831356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 msgr2=0x7fc604101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 --2- 172.21.15.87:0/82831356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc604101e30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009930 tx=0x7fc5ec02a420 comp rx=0 tx=0).stop 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 -- 172.21.15.87:0/82831356 shutdown_connections 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 --2- 172.21.15.87:0/82831356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc604102370 0x7fc60410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 --2- 172.21.15.87:0/82831356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc604101e30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 --2- 172.21.15.87:0/82831356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6041079c0 0x7fc604107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.076+0000 7fc60997e640 1 -- 172.21.15.87:0/82831356 >> 172.21.15.87:0/82831356 conn(0x7fc604078e30 msgr2=0x7fc604079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- 172.21.15.87:0/82831356 shutdown_connections 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- 172.21.15.87:0/82831356 wait complete. 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 Processor -- start 2024-06-06T08:00:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- start start 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 0x7fc604198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 0x7fc60419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc604111dd0 con 0x7fc6041019b0 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc604111c50 con 0x7fc604102370 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.077+0000 7fc60997e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc604111f50 con 0x7fc6041079c0 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33844/0 (socket says 172.21.15.87:33844) 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 -- 172.21.15.87:0/1677721230 learned_addr learned my addr 172.21.15.87:0/1677721230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 0x7fc604198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 0x7fc60419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 -- 172.21.15.87:0/1677721230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 msgr2=0x7fc60419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 0x7fc60419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 -- 172.21.15.87:0/1677721230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 msgr2=0x7fc604198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 0x7fc604198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 -- 172.21.15.87:0/1677721230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60419d1e0 con 0x7fc6041019b0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 0x7fc604198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 0x7fc60419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc60897c640 1 -- 172.21.15.87:0/1677721230 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4017070 con 0x7fc6041019b0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc60897c640 1 -- 172.21.15.87:0/1677721230 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4009c30 con 0x7fc6041019b0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60419d4d0 con 0x7fc6041019b0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc60897c640 1 -- 172.21.15.87:0/1677721230 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400f6b0 con 0x7fc6041019b0 2024-06-06T08:00:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.078+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041a4d60 con 0x7fc6041019b0 2024-06-06T08:00:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60897c640 1 -- 172.21.15.87:0/1677721230 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc5f40109d0 con 0x7fc6041019b0 2024-06-06T08:00:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 Processor -- start 2024-06-06T08:00:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 -- start start 2024-06-06T08:00:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab1b170 0x55d89ab1b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab25e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d89aa7f4f0 con 0x55d89ab26350 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d89aa1d870 con 0x55d89ab1b170 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60c76a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d89aa7b4b0 con 0x55d89ab1ba90 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc60897c640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 0x7fc5f00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33860/0 (socket says 172.21.15.87:33860) 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc5fbfff640 1 -- 172.21.15.87:0/638355236 learned_addr learned my addr 172.21.15.87:0/638355236 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.081+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 0x7fc5f00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc60897c640 1 -- 172.21.15.87:0/1677721230 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fc5f40cf440 con 0x7fc6041019b0 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc602ffd640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab25e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc6037fe640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab1b170 0x55d89ab1b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 -- 172.21.15.87:0/638355236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 msgr2=0x55d89ab25e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab25e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 -- 172.21.15.87:0/638355236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab1b170 msgr2=0x55d89ab1b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab1b170 0x55d89ab1b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 -- 172.21.15.87:0/638355236 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89aa399b0 con 0x55d89ab26350 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc602ffd640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab25e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 0x7fc5f00a52e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005fd0 tx=0x7fc5fc009630 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009930 tx=0x7fc5ec032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd90531a4641b6dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5f9ffb640 1 -- 172.21.15.87:0/638355236 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec03f070 con 0x55d89ab26350 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5f9ffb640 1 -- 172.21.15.87:0/638355236 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0317b0 con 0x55d89ab26350 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.082+0000 7fc5f9ffb640 1 -- 172.21.15.87:0/638355236 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec035070 con 0x55d89ab26350 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- 172.21.15.87:0/638355236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 msgr2=0x55d89ab28740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009930 tx=0x7fc5ec032cb0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- 172.21.15.87:0/638355236 shutdown_connections 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab26350 0x55d89ab28740 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab25e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 --2- 172.21.15.87:0/638355236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab1b170 0x55d89ab1b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- 172.21.15.87:0/638355236 >> 172.21.15.87:0/638355236 conn(0x55d89ab17680 msgr2=0x55d89ab17a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- 172.21.15.87:0/638355236 shutdown_connections 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- 172.21.15.87:0/638355236 wait complete. 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.083+0000 7fc60c76a740 1 -- start start 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 Processor -- start 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1b170 0x55d89ab232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d89ab28f20 con 0x55d89ab1ba90 2024-06-06T08:00:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d89aa08ae0 con 0x55d89ab26350 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d89ab290a0 con 0x55d89ab1b170 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc60c76a740 10 client.0 _ll_register_callbacks cb 0x55d89aa419b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33862/0 (socket says 172.21.15.87:33862) 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 -- 172.21.15.87:0/1038156347 learned_addr learned my addr 172.21.15.87:0/1038156347 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1b170 0x55d89ab232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 -- 172.21.15.87:0/1038156347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1b170 msgr2=0x55d89ab232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1b170 0x55d89ab232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 -- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 msgr2=0x55d89ab23d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab23d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 -- 172.21.15.87:0/1038156347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89a793210 con 0x55d89ab1ba90 2024-06-06T08:00:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc6037fe640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab23d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005400 tx=0x7fc5ec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98665bd7ee987c4e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.084+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec035070 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fc5ec0048d0 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ec0317b0 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5ec055050 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 10 client.17346 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d89a85f620 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0317b0 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fc5ec03bb50 con 0x55d89ab1ba90 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 10 client.17346 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 20 client.17346 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89ab362e0 0x55d89ab386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 4 client.17346 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d89aa92f10 con 0x55d89ab362e0 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.085+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89ab362e0 0x55d89ab386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.086+0000 7fc602ffd640 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89ab362e0 0x55d89ab386d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160dc8c26db7e7ff server_cookie=c1ceffc172b90813 in_seq=0 out_seq=0 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.086+0000 7fc5e2ffd640 10 client.17346 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/1038156347 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d89aa92f10 con 0x55d89ab362e0 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc5e2ffd640 10 client.17346 handle_command_reply: tid=0 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 client.17346 shutdown 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 msgr2=0x55d89ab23800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005400 tx=0x7fc5ec031400 comp rx=0 tx=0).stop 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 shutdown_connections 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89ab362e0 0x55d89ab386d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab23d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1ba90 0x55d89ab23800 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 --2- 172.21.15.87:0/1038156347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1b170 0x55d89ab232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc5e2ffd640 0 client.17346 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 >> 172.21.15.87:0/1038156347 conn(0x55d89ab17680 msgr2=0x55d89ab1d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 shutdown_connections 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 1 -- 172.21.15.87:0/1038156347 wait complete. 2024-06-06T08:00:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.087+0000 7fc60c76a740 20 client.17346 trim_cache size 0 max 16384 2024-06-06T08:00:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 Processor -- start 2024-06-06T08:00:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 -- start start 2024-06-06T08:00:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab20520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041b1880 con 0x55d89ab1b170 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041af150 con 0x55d89ab26350 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc60c76a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041acfc0 con 0x55d89ab1ba90 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33870/0 (socket says 172.21.15.87:33870) 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 -- 172.21.15.87:0/3285911523 learned_addr learned my addr 172.21.15.87:0/3285911523 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 -- 172.21.15.87:0/3285911523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 msgr2=0x55d89ab20520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab20520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 -- 172.21.15.87:0/3285911523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 msgr2=0x55d89ab20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.102+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc602ffd640 1 -- 172.21.15.87:0/3285911523 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89ab1c200 con 0x55d89ab1b170 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009610 tx=0x7fc5f4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45211897eb27efcc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc5f97fa640 1 -- 172.21.15.87:0/3285911523 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020040 con 0x55d89ab1b170 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc5f97fa640 1 -- 172.21.15.87:0/3285911523 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f40bb040 con 0x55d89ab1b170 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc5f97fa640 1 -- 172.21.15.87:0/3285911523 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40ccb80 con 0x55d89ab1b170 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 -- 172.21.15.87:0/3285911523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 msgr2=0x55d89ab1db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009610 tx=0x7fc5f4015040 comp rx=0 tx=0).stop 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 -- 172.21.15.87:0/3285911523 shutdown_connections 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab20520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3285911523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab1db30 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 -- 172.21.15.87:0/3285911523 >> 172.21.15.87:0/3285911523 conn(0x55d89ab17680 msgr2=0x55d89ab1e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 -- 172.21.15.87:0/3285911523 shutdown_connections 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.103+0000 7fc60c76a740 1 -- 172.21.15.87:0/3285911523 wait complete. 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 -- start start 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 Processor -- start 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab38ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041ad610 con 0x55d89ab1b170 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041aea80 con 0x55d89ab26350 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041adce0 con 0x55d89ab1ba90 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc60c76a740 10 client.0 _ll_register_callbacks cb 0x7fc5ec005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc602ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33876/0 (socket says 172.21.15.87:33876) 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc602ffd640 1 -- 172.21.15.87:0/3216962689 learned_addr learned my addr 172.21.15.87:0/3216962689 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.104+0000 7fc6037fe640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab38ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 -- 172.21.15.87:0/3216962689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 msgr2=0x55d89ab3bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 -- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 msgr2=0x55d89ab38ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab38ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 -- 172.21.15.87:0/3216962689 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89ab397f0 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc602ffd640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b1f0 tx=0x7fc5f4005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d89dffe177c6835 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b4060 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fc5f409a990 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f40bb040 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5f4020070 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 10 client.17352 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d89aba0840 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f40bb040 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fc5f4097290 con 0x55d89ab1b170 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 10 client.17352 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 20 client.17352 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89aba0e90 0x55d89aba32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 4 client.17352 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:00:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.105+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d89aa8cc20 con 0x55d89aba0e90 2024-06-06T08:00:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.106+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89aba0e90 0x55d89aba32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.106+0000 7fc5fbfff640 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89aba0e90 0x55d89aba32b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9a06ffa01f5fa server_cookie=854b324f827d8dc0 in_seq=0 out_seq=0 2024-06-06T08:00:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.106+0000 7fc5e2ffd640 10 client.17352 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc5e2ffd640 1 -- 172.21.15.87:0/3216962689 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d89aa8cc20 con 0x55d89aba0e90 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc5e2ffd640 10 client.17352 handle_command_reply: tid=0 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 client.17352 shutdown 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 msgr2=0x55d89ab3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b1f0 tx=0x7fc5f4005bf0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 shutdown_connections 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d89aba0e90 0x55d89aba32b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d89ab26350 0x55d89ab38ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc5e2ffd640 0 client.17352 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d89ab1ba90 0x55d89ab3bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 --2- 172.21.15.87:0/3216962689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d89ab1b170 0x55d89ab3ba60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 >> 172.21.15.87:0/3216962689 conn(0x55d89ab17680 msgr2=0x55d89ab23260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 shutdown_connections 2024-06-06T08:00:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 1 -- 172.21.15.87:0/3216962689 wait complete. 2024-06-06T08:00:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.191+0000 7fc60c76a740 20 client.17352 trim_cache size 0 max 16384 2024-06-06T08:00:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:00:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:00:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 msgr2=0x7fc5f00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 0x7fc5f00a52e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005fd0 tx=0x7fc5fc009630 comp rx=0 tx=0).stop 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 msgr2=0x7fc6041981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).stop 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 shutdown_connections 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5f00a2e20 0x7fc5f00a52e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6041079c0 0x7fc60419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc604102370 0x7fc604198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 --2- 172.21.15.87:0/1677721230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6041019b0 0x7fc6041981f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 >> 172.21.15.87:0/1677721230 conn(0x7fc604078e30 msgr2=0x7fc6040ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 shutdown_connections 2024-06-06T08:00:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:25.192+0000 7fc60997e640 1 -- 172.21.15.87:0/1677721230 wait complete. 2024-06-06T08:00:25.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657121 5.68 MB/sec execute 509 sec latency 2736.660 ms 2024-06-06T08:00:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:25 smithi195 ceph-mon[37040]: pgmap v391: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 840 KiB/s wr, 85 op/s 2024-06-06T08:00:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:25 smithi181 ceph-mon[36790]: pgmap v391: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 840 KiB/s wr, 85 op/s 2024-06-06T08:00:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:25 smithi087 ceph-mon[32276]: pgmap v391: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 840 KiB/s wr, 85 op/s 2024-06-06T08:00:26.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657121 5.67 MB/sec execute 510 sec latency 3736.747 ms 2024-06-06T08:00:27.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 657121 5.66 MB/sec execute 511 sec latency 4736.856 ms 2024-06-06T08:00:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:27 smithi195 ceph-mon[37040]: pgmap v392: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 907 KiB/s wr, 107 op/s 2024-06-06T08:00:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:27 smithi181 ceph-mon[36790]: pgmap v392: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 907 KiB/s wr, 107 op/s 2024-06-06T08:00:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:27 smithi087 ceph-mon[32276]: pgmap v392: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 907 KiB/s wr, 107 op/s 2024-06-06T08:00:28.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 658221 5.65 MB/sec execute 512 sec latency 4894.890 ms 2024-06-06T08:00:29.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:28 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:00:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:28 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:00:29.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:28 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:00:29.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 660370 5.66 MB/sec execute 513 sec latency 38.865 ms 2024-06-06T08:00:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:29 smithi195 ceph-mon[37040]: pgmap v393: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 790 KiB/s wr, 92 op/s 2024-06-06T08:00:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:29 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:00:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:29 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:00:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:29 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:00:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:29 smithi181 ceph-mon[36790]: pgmap v393: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 790 KiB/s wr, 92 op/s 2024-06-06T08:00:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:29 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:00:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:29 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:00:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:29 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:00:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:29 smithi087 ceph-mon[32276]: pgmap v393: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 790 KiB/s wr, 92 op/s 2024-06-06T08:00:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:29 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:00:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:29 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:00:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:29 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:00:30.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 660599 5.66 MB/sec execute 514 sec latency 907.151 ms 2024-06-06T08:00:31.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 660599 5.65 MB/sec execute 515 sec latency 1907.251 ms 2024-06-06T08:00:32.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:31 smithi195 ceph-mon[37040]: pgmap v394: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 496 KiB/s wr, 68 op/s 2024-06-06T08:00:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:31 smithi181 ceph-mon[36790]: pgmap v394: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 496 KiB/s wr, 68 op/s 2024-06-06T08:00:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:31 smithi087 ceph-mon[32276]: pgmap v394: 129 pgs: 129 active+clean; 81 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 496 KiB/s wr, 68 op/s 2024-06-06T08:00:32.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 660666 5.64 MB/sec execute 516 sec latency 2180.229 ms 2024-06-06T08:00:33.690 INFO:tasks.workunit.client.0.smithi087.stdout: 1 661958 5.64 MB/sec execute 517 sec latency 781.229 ms 2024-06-06T08:00:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:33 smithi195 ceph-mon[37040]: pgmap v395: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-06-06T08:00:34.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:33 smithi181 ceph-mon[36790]: pgmap v395: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-06-06T08:00:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:33 smithi087 ceph-mon[32276]: pgmap v395: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-06-06T08:00:34.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664146 5.65 MB/sec execute 518 sec latency 111.755 ms 2024-06-06T08:00:35.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664146 5.64 MB/sec execute 519 sec latency 1111.851 ms 2024-06-06T08:00:36.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:35 smithi195 ceph-mon[37040]: pgmap v396: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 815 KiB/s wr, 85 op/s 2024-06-06T08:00:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:35 smithi181 ceph-mon[36790]: pgmap v396: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 815 KiB/s wr, 85 op/s 2024-06-06T08:00:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:35 smithi087 ceph-mon[32276]: pgmap v396: 129 pgs: 129 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 815 KiB/s wr, 85 op/s 2024-06-06T08:00:36.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664146 5.63 MB/sec execute 520 sec latency 2111.948 ms 2024-06-06T08:00:37.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664146 5.62 MB/sec execute 521 sec latency 3112.034 ms 2024-06-06T08:00:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:37 smithi195 ceph-mon[37040]: pgmap v397: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- 172.21.15.87:0/802569939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 msgr2=0x7fbaac10d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 --2- 172.21.15.87:0/802569939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac10d780 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009cd0 tx=0x7fbaa402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- 172.21.15.87:0/802569939 shutdown_connections 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 --2- 172.21.15.87:0/802569939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac10d780 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 --2- 172.21.15.87:0/802569939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac100900 0x7fbaac100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 --2- 172.21.15.87:0/802569939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac1068c0 0x7fbaac106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- 172.21.15.87:0/802569939 >> 172.21.15.87:0/802569939 conn(0x7fbaac077ab0 msgr2=0x7fbaac077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- 172.21.15.87:0/802569939 shutdown_connections 2024-06-06T08:00:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- 172.21.15.87:0/802569939 wait complete. 2024-06-06T08:00:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 Processor -- start 2024-06-06T08:00:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.284+0000 7fbab2542640 1 -- start start 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac100900 0x7fbaac198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 0x7fbaac19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111dd0 con 0x7fbaac1012c0 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111c50 con 0x7fbaac1068c0 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111f50 con 0x7fbaac100900 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53974/0 (socket says 172.21.15.87:53974) 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 -- 172.21.15.87:0/3787449605 learned_addr learned my addr 172.21.15.87:0/3787449605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab0adf640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 0x7fbaac19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaabfff640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac100900 0x7fbaac198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 -- 172.21.15.87:0/3787449605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac100900 msgr2=0x7fbaac198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac100900 0x7fbaac198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 -- 172.21.15.87:0/3787449605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 msgr2=0x7fbaac19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 0x7fbaac19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 -- 172.21.15.87:0/3787449605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac19d120 con 0x7fbaac1012c0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab0adf640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 0x7fbaac19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000c970 tx=0x7fbaa000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaa97fa640 1 -- 172.21.15.87:0/3787449605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0007bb0 con 0x7fbaac1012c0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac19d410 con 0x7fbaac1012c0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaa97fa640 1 -- 172.21.15.87:0/3787449605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa0007d50 con 0x7fbaac1012c0 2024-06-06T08:00:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbaa97fa640 1 -- 172.21.15.87:0/3787449605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa000f450 con 0x7fbaac1012c0 2024-06-06T08:00:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.285+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a4ca0 con 0x7fbaac1012c0 2024-06-06T08:00:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbaa97fa640 1 -- 172.21.15.87:0/3787449605 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbaa0005020 con 0x7fbaac1012c0 2024-06-06T08:00:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 Processor -- start 2024-06-06T08:00:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 -- start start 2024-06-06T08:00:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbaa97fa640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 0x7fba940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eeaee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeb1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f59ee084f0 con 0x55f59eea4170 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f59eda6870 con 0x55f59eea4a90 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab5251740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f59ee044b0 con 0x55f59eeaf350 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbab0adf640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 0x7fba940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.288+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaa97fa640 1 -- 172.21.15.87:0/3787449605 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fbaa00cf710 con 0x7fbaac1012c0 2024-06-06T08:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53988/0 (socket says 172.21.15.87:53988) 2024-06-06T08:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 -- 172.21.15.87:0/4146183125 learned_addr learned my addr 172.21.15.87:0/4146183125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eeaee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 -- 172.21.15.87:0/4146183125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 msgr2=0x55f59eeb1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeb1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 -- 172.21.15.87:0/4146183125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 msgr2=0x55f59eeaee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eeaee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 -- 172.21.15.87:0/4146183125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59edc29b0 con 0x55f59eea4170 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eeaee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbab0adf640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 0x7fba940a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006010 tx=0x7fbaa4031610 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fbaabfff640 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bc20 tx=0x7fba9c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2871406124aeebd2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fba927fc640 1 -- 172.21.15.87:0/4146183125 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c015070 con 0x55f59eea4170 2024-06-06T08:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fba927fc640 1 -- 172.21.15.87:0/4146183125 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c005500 con 0x55f59eea4170 2024-06-06T08:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.289+0000 7fba927fc640 1 -- 172.21.15.87:0/4146183125 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0116a0 con 0x55f59eea4170 2024-06-06T08:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- 172.21.15.87:0/4146183125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 msgr2=0x55f59eea4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bc20 tx=0x7fba9c00f730 comp rx=0 tx=0).stop 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- 172.21.15.87:0/4146183125 shutdown_connections 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeb1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eeaee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 --2- 172.21.15.87:0/4146183125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eea4550 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- 172.21.15.87:0/4146183125 >> 172.21.15.87:0/4146183125 conn(0x55f59eea0680 msgr2=0x55f59eea0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- 172.21.15.87:0/4146183125 shutdown_connections 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- 172.21.15.87:0/4146183125 wait complete. 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 -- start start 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.290+0000 7fbab5251740 1 Processor -- start 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4170 0x55f59eeac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeacce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f59eeb1f80 con 0x55f59eea4a90 2024-06-06T08:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f59ed91ae0 con 0x55f59eea4170 2024-06-06T08:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f59eeb2100 con 0x55f59eeaf350 2024-06-06T08:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 10 client.0 _ll_register_callbacks cb 0x55f59edca9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4170 0x55f59eeac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53990/0 (socket says 172.21.15.87:53990) 2024-06-06T08:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 -- 172.21.15.87:0/1527193505 learned_addr learned my addr 172.21.15.87:0/1527193505 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab0adf640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeacce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 -- 172.21.15.87:0/1527193505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 msgr2=0x55f59eeacce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeacce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 -- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4170 msgr2=0x55f59eeac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4170 0x55f59eeac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 -- 172.21.15.87:0/1527193505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59eb1c210 con 0x55f59eea4a90 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab0adf640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeacce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00058c0 tx=0x7fbaa009b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8e8d9487aed58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fba8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa001d040 con 0x55f59eea4a90 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fbaa0065830 con 0x55f59eea4a90 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.291+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c005500 con 0x55f59eea4a90 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbaa009bf10 con 0x55f59eea4a90 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 10 client.17364 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f59ebe8620 con 0x55f59eea4a90 2024-06-06T08:00:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c005500 con 0x55f59eea4a90 2024-06-06T08:00:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fbaa0097e80 con 0x55f59eea4a90 2024-06-06T08:00:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 10 client.17364 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 20 client.17364 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59eebf4d0 0x55f59eec18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 4 client.17364 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:00:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f59ee1bf10 con 0x55f59eebf4d0 2024-06-06T08:00:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.292+0000 7fbaabfff640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59eebf4d0 0x55f59eec18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.293+0000 7fbaabfff640 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59eebf4d0 0x55f59eec18f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc74b10dd058ba42 server_cookie=409a1798869f2472 in_seq=0 out_seq=0 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.293+0000 7fba8b7fe640 10 client.17364 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.293+0000 7fba8b7fe640 1 -- 172.21.15.87:0/1527193505 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f59ee1bf10 con 0x55f59eebf4d0 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.293+0000 7fba8b7fe640 10 client.17364 handle_command_reply: tid=0 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.293+0000 7fbab5251740 1 client.17364 shutdown 2024-06-06T08:00:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 msgr2=0x55f59eeac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00058c0 tx=0x7fbaa009b440 comp rx=0 tx=0).stop 2024-06-06T08:00:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 shutdown_connections 2024-06-06T08:00:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59eebf4d0 0x55f59eec18f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eeacce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fba8b7fe640 0 client.17364 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4a90 0x55f59eeac7a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 --2- 172.21.15.87:0/1527193505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4170 0x55f59eeac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 >> 172.21.15.87:0/1527193505 conn(0x55f59eea0680 msgr2=0x55f59eea6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 shutdown_connections 2024-06-06T08:00:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 1 -- 172.21.15.87:0/1527193505 wait complete. 2024-06-06T08:00:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.294+0000 7fbab5251740 20 client.17364 trim_cache size 0 max 16384 2024-06-06T08:00:38.320 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:37 smithi181 ceph-mon[36790]: pgmap v397: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T08:00:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 Processor -- start 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 -- start start 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eea4a90 0x55f59eec7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eeaf350 0x55f59eec7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1b1840 con 0x55f59eea4170 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1af110 con 0x55f59eeaf350 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab5251740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1acf80 con 0x55f59eea4a90 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54004/0 (socket says 172.21.15.87:54004) 2024-06-06T08:00:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 -- 172.21.15.87:0/2432822210 learned_addr learned my addr 172.21.15.87:0/2432822210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbab0adf640 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eeaf350 0x55f59eec7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eea4a90 0x55f59eec7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 -- 172.21.15.87:0/2432822210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eea4a90 msgr2=0x55f59eec7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eea4a90 0x55f59eec7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 -- 172.21.15.87:0/2432822210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eeaf350 msgr2=0x55f59eec7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eeaf350 0x55f59eec7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 -- 172.21.15.87:0/2432822210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59eeaa7b0 con 0x55f59eea4170 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.309+0000 7fbaabfff640 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bc20 tx=0x7fba9c00b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cc9b883ebeebc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fba91ffb640 1 -- 172.21.15.87:0/2432822210 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c015040 con 0x55f59eea4170 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fba91ffb640 1 -- 172.21.15.87:0/2432822210 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c004680 con 0x55f59eea4170 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fba91ffb640 1 -- 172.21.15.87:0/2432822210 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0116a0 con 0x55f59eea4170 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- 172.21.15.87:0/2432822210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 msgr2=0x55f59eeac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bc20 tx=0x7fba9c00b550 comp rx=0 tx=0).stop 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- 172.21.15.87:0/2432822210 shutdown_connections 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eeaf350 0x55f59eec7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eea4a90 0x55f59eec7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 --2- 172.21.15.87:0/2432822210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeac930 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- 172.21.15.87:0/2432822210 >> 172.21.15.87:0/2432822210 conn(0x55f59eea0680 msgr2=0x55f59eeac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- 172.21.15.87:0/2432822210 shutdown_connections 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- 172.21.15.87:0/2432822210 wait complete. 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.310+0000 7fbab5251740 1 -- start start 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 Processor -- start 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eea8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eea9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1ad5d0 con 0x55f59eea4170 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1aea40 con 0x55f59eea4a90 2024-06-06T08:00:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1adca0 con 0x55f59eeaf350 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab5251740 10 client.0 _ll_register_callbacks cb 0x7fbaa0007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54012/0 (socket says 172.21.15.87:54012) 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 -- 172.21.15.87:0/726636517 learned_addr learned my addr 172.21.15.87:0/726636517 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab0adf640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eea9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eea8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 -- 172.21.15.87:0/726636517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 msgr2=0x55f59eea9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eea9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 -- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 msgr2=0x55f59eea8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eea8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbaabfff640 1 -- 172.21.15.87:0/726636517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59eea9cf0 con 0x55f59eea4170 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.311+0000 7fbab0adf640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eea9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbaabfff640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005570 tx=0x7fba9c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99951af7b1e91cc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c004e60 con 0x55f59eea4170 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fba9c005000 con 0x55f59eea4170 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c004680 con 0x55f59eea4170 2024-06-06T08:00:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fba9c03f050 con 0x55f59eea4170 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 10 client.17370 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f59ef2a1f0 con 0x55f59eea4170 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c004680 con 0x55f59eea4170 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fba9c0394a0 con 0x55f59eea4170 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 10 client.17370 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 20 client.17370 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59ef2a710 0x55f59ef2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 4 client.17370 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:00:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.312+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f59ee15c20 con 0x55f59ef2a710 2024-06-06T08:00:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.313+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59ef2a710 0x55f59ef2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.313+0000 7fba8b7fe640 10 client.17370 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.313+0000 7fbaab7fe640 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59ef2a710 0x55f59ef2cb00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bb5be4c62f7cb0 server_cookie=8abb953097934b5e in_seq=0 out_seq=0 2024-06-06T08:00:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:37 smithi087 ceph-mon[32276]: pgmap v397: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T08:00:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fba8b7fe640 1 -- 172.21.15.87:0/726636517 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f59ee15c20 con 0x55f59ef2a710 2024-06-06T08:00:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fba8b7fe640 10 client.17370 handle_command_reply: tid=0 2024-06-06T08:00:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fbab5251740 1 client.17370 shutdown 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 msgr2=0x55f59eeb7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005570 tx=0x7fba9c00b420 comp rx=0 tx=0).stop 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 shutdown_connections 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.440+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f59ef2a710 0x55f59ef2cb00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fba8b7fe640 0 client.17370 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f59eeaf350 0x55f59eea9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f59eea4a90 0x55f59eea8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 --2- 172.21.15.87:0/726636517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f59eea4170 0x55f59eeb7ab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 >> 172.21.15.87:0/726636517 conn(0x55f59eea0680 msgr2=0x55f59eeacab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 shutdown_connections 2024-06-06T08:00:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 1 -- 172.21.15.87:0/726636517 wait complete. 2024-06-06T08:00:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab5251740 20 client.17370 trim_cache size 0 max 16384 2024-06-06T08:00:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:00:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:00:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 msgr2=0x7fba940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 0x7fba940a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006010 tx=0x7fbaa4031610 comp rx=0 tx=0).stop 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 msgr2=0x7fbaac198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000c970 tx=0x7fbaa000ce30 comp rx=0 tx=0).stop 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 shutdown_connections 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fba940a2da0 0x7fba940a5260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac1068c0 0x7fbaac19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac1012c0 0x7fbaac198670 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 --2- 172.21.15.87:0/3787449605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaac100900 0x7fbaac198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 >> 172.21.15.87:0/3787449605 conn(0x7fbaac077ab0 msgr2=0x7fbaac103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 shutdown_connections 2024-06-06T08:00:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:38.441+0000 7fbab2542640 1 -- 172.21.15.87:0/3787449605 wait complete. 2024-06-06T08:00:38.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664213 5.61 MB/sec execute 522 sec latency 3258.868 ms 2024-06-06T08:00:39.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664213 5.60 MB/sec execute 523 sec latency 1762.136 ms 2024-06-06T08:00:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:39 smithi195 ceph-mon[37040]: pgmap v398: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-06-06T08:00:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:39 smithi181 ceph-mon[36790]: pgmap v398: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-06-06T08:00:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:39 smithi087 ceph-mon[32276]: pgmap v398: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-06-06T08:00:40.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664213 5.59 MB/sec execute 524 sec latency 2762.267 ms 2024-06-06T08:00:41.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664213 5.58 MB/sec execute 525 sec latency 3762.356 ms 2024-06-06T08:00:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:41 smithi195 ceph-mon[37040]: pgmap v399: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-06-06T08:00:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:41 smithi181 ceph-mon[36790]: pgmap v399: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-06-06T08:00:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:41 smithi087 ceph-mon[32276]: pgmap v399: 129 pgs: 129 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-06-06T08:00:42.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 664213 5.57 MB/sec execute 526 sec latency 4762.455 ms 2024-06-06T08:00:43.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 665458 5.57 MB/sec execute 527 sec latency 4919.085 ms 2024-06-06T08:00:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:43 smithi195 ceph-mon[37040]: pgmap v400: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-06T08:00:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:43 smithi181 ceph-mon[36790]: pgmap v400: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-06T08:00:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:43 smithi087 ceph-mon[32276]: pgmap v400: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-06T08:00:44.691 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667690 5.58 MB/sec execute 528 sec latency 26.503 ms 2024-06-06T08:00:45.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667690 5.57 MB/sec execute 529 sec latency 1012.216 ms 2024-06-06T08:00:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:45 smithi195 ceph-mon[37040]: pgmap v401: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 677 KiB/s wr, 90 op/s 2024-06-06T08:00:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:45 smithi181 ceph-mon[36790]: pgmap v401: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 677 KiB/s wr, 90 op/s 2024-06-06T08:00:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:45 smithi087 ceph-mon[32276]: pgmap v401: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 677 KiB/s wr, 90 op/s 2024-06-06T08:00:46.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667690 5.56 MB/sec execute 530 sec latency 2012.312 ms 2024-06-06T08:00:47.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667690 5.55 MB/sec execute 531 sec latency 3012.410 ms 2024-06-06T08:00:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:47 smithi195 ceph-mon[37040]: pgmap v402: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 694 KiB/s wr, 93 op/s 2024-06-06T08:00:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:47 smithi181 ceph-mon[36790]: pgmap v402: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 694 KiB/s wr, 93 op/s 2024-06-06T08:00:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:47 smithi087 ceph-mon[32276]: pgmap v402: 129 pgs: 129 active+clean; 79 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 694 KiB/s wr, 93 op/s 2024-06-06T08:00:48.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667757 5.54 MB/sec execute 532 sec latency 3169.626 ms 2024-06-06T08:00:49.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667757 5.53 MB/sec execute 533 sec latency 1752.828 ms 2024-06-06T08:00:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:49 smithi195 ceph-mon[37040]: pgmap v403: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 795 KiB/s wr, 102 op/s 2024-06-06T08:00:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:49 smithi181 ceph-mon[36790]: pgmap v403: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 795 KiB/s wr, 102 op/s 2024-06-06T08:00:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:49 smithi087 ceph-mon[32276]: pgmap v403: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 795 KiB/s wr, 102 op/s 2024-06-06T08:00:50.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667757 5.52 MB/sec execute 534 sec latency 2752.933 ms 2024-06-06T08:00:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 -- 172.21.15.87:0/2652570281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30102c30 msgr2=0x7f6d30103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 --2- 172.21.15.87:0/2652570281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30102c30 0x7f6d30103010 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009900 tx=0x7f6d2002a510 comp rx=0 tx=0).stop 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 -- 172.21.15.87:0/2652570281 shutdown_connections 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 --2- 172.21.15.87:0/2652570281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3010bf70 0x7f6d3010e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 --2- 172.21.15.87:0/2652570281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d30103550 0x7f6d3010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 --2- 172.21.15.87:0/2652570281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30102c30 0x7f6d30103010 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 -- 172.21.15.87:0/2652570281 >> 172.21.15.87:0/2652570281 conn(0x7f6d30077a20 msgr2=0x7f6d30077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 -- 172.21.15.87:0/2652570281 shutdown_connections 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.538+0000 7f6d3597e640 1 -- 172.21.15.87:0/2652570281 wait complete. 2024-06-06T08:00:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 Processor -- start 2024-06-06T08:00:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 -- start start 2024-06-06T08:00:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 0x7f6d30198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 0x7f6d3019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30111d70 con 0x7f6d30103550 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30111bf0 con 0x7f6d30102c30 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d3597e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30111ef0 con 0x7f6d3010bf70 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d2e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d2e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60786/0 (socket says 172.21.15.87:60786) 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.539+0000 7f6d2e7fc640 1 -- 172.21.15.87:0/3715768776 learned_addr learned my addr 172.21.15.87:0/3715768776 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 0x7f6d3019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2effd640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 0x7f6d30198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 -- 172.21.15.87:0/3715768776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 msgr2=0x7f6d3019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 0x7f6d3019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 -- 172.21.15.87:0/3715768776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 msgr2=0x7f6d30198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 0x7f6d30198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 -- 172.21.15.87:0/3715768776 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3019d1b0 con 0x7f6d30103550 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2effd640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 0x7f6d30198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 0x7f6d3019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d2e7fc640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c920 tx=0x7f6d2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d3497c640 1 -- 172.21.15.87:0/3715768776 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24007c70 con 0x7f6d30103550 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d3497c640 1 -- 172.21.15.87:0/3715768776 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d24007e10 con 0x7f6d30103550 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3019d4a0 con 0x7f6d30103550 2024-06-06T08:00:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d3497c640 1 -- 172.21.15.87:0/3715768776 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24018890 con 0x7f6d30103550 2024-06-06T08:00:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.540+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d301a4d30 con 0x7f6d30103550 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3497c640 1 -- 172.21.15.87:0/3715768776 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6d24018a70 con 0x7f6d30103550 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 Processor -- start 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 -- start start 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63d9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3497c640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 0x7f6d180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560fd633d4f0 con 0x560fd63e4350 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560fd62db870 con 0x560fd63d9170 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.543+0000 7f6d3876d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560fd63394b0 con 0x560fd63d9a90 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2e7fc640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 0x7f6d180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60790/0 (socket says 172.21.15.87:60790) 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/3464869627 learned_addr learned my addr 172.21.15.87:0/3464869627 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2effd640 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d3497c640 1 -- 172.21.15.87:0/3715768776 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6d240cfa70 con 0x7f6d30103550 2024-06-06T08:00:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/3464869627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 msgr2=0x560fd63e3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/3464869627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 msgr2=0x560fd63d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/3464869627 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd62f79b0 con 0x560fd63e4350 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2e7fc640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 0x7f6d180a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d24097fd0 tx=0x7f6d24070040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6d28014850 tx=0x7f6d28009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26764bea270b5a64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d15ffb640 1 -- 172.21.15.87:0/3464869627 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a030 con 0x560fd63e4350 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d15ffb640 1 -- 172.21.15.87:0/3464869627 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d28012890 con 0x560fd63e4350 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.544+0000 7f6d15ffb640 1 -- 172.21.15.87:0/3464869627 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28005390 con 0x560fd63e4350 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- 172.21.15.87:0/3464869627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 msgr2=0x560fd63e6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6d28014850 tx=0x7f6d28009940 comp rx=0 tx=0).stop 2024-06-06T08:00:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- 172.21.15.87:0/3464869627 shutdown_connections 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63e6740 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 --2- 172.21.15.87:0/3464869627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- 172.21.15.87:0/3464869627 >> 172.21.15.87:0/3464869627 conn(0x560fd63d5680 msgr2=0x560fd63d5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- 172.21.15.87:0/3464869627 shutdown_connections 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- 172.21.15.87:0/3464869627 wait complete. 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.545+0000 7f6d3876d740 1 -- start start 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 Processor -- start 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560fd63e6f80 con 0x560fd63d9170 2024-06-06T08:00:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560fd62c6ae0 con 0x560fd63e4350 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560fd63e7100 con 0x560fd63d9a90 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d3876d740 10 client.0 _ll_register_callbacks cb 0x560fd62ff9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60802/0 (socket says 172.21.15.87:60802) 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1949883614 learned_addr learned my addr 172.21.15.87:0/1949883614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2effd640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1949883614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 msgr2=0x560fd63e17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 msgr2=0x560fd63e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1949883614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd6051210 con 0x560fd63d9170 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6d280050c0 tx=0x7f6d28009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cff5d77eea7c8dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.546+0000 7f6d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a040 con 0x560fd63d9170 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6d28004a40 con 0x560fd63d9170 2024-06-06T08:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d28012890 con 0x560fd63d9170 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d28004c40 con 0x560fd63d9170 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 10 client.17382 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560fd611d620 con 0x560fd63d9170 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d28012890 con 0x560fd63d9170 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f6d28010050 con 0x560fd63d9170 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 10 client.17382 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 20 client.17382 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd63f4400 0x560fd63f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 4 client.17382 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.547+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd6350f10 con 0x560fd63f4400 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.548+0000 7f6d2effd640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd63f4400 0x560fd63f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.548+0000 7f6d2effd640 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd63f4400 0x560fd63f67f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcd1d9b9371c351 server_cookie=b2402c7cdd53972a in_seq=0 out_seq=0 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.548+0000 7f6d0effd640 10 client.17382 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d0effd640 1 -- 172.21.15.87:0/1949883614 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560fd6350f10 con 0x560fd63f4400 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d0effd640 10 client.17382 handle_command_reply: tid=0 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 client.17382 shutdown 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 msgr2=0x560fd63e1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6d280050c0 tx=0x7f6d28009620 comp rx=0 tx=0).stop 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 shutdown_connections 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd63f4400 0x560fd63f67f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63e17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d0effd640 0 client.17382 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1949883614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd63e1290 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 >> 172.21.15.87:0/1949883614 conn(0x560fd63d5680 msgr2=0x560fd63db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 shutdown_connections 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 1 -- 172.21.15.87:0/1949883614 wait complete. 2024-06-06T08:00:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.549+0000 7f6d3876d740 20 client.17382 trim_cache size 0 max 16384 2024-06-06T08:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 Processor -- start 2024-06-06T08:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 -- start start 2024-06-06T08:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301b17e0 con 0x560fd63e4350 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301af0b0 con 0x560fd63d9170 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d3876d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301acf20 con 0x560fd63d9a90 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60814/0 (socket says 172.21.15.87:60814) 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1907126881 learned_addr learned my addr 172.21.15.87:0/1907126881 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2effd640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1907126881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 msgr2=0x560fd63d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1907126881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 msgr2=0x560fd63dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63dbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/1907126881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd63e70a0 con 0x560fd63e4350 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.565+0000 7f6d2effd640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800a010 tx=0x7f6d28016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee09ce17d377e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d157fa640 1 -- 172.21.15.87:0/1907126881 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a070 con 0x560fd63e4350 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d157fa640 1 -- 172.21.15.87:0/1907126881 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d28024070 con 0x560fd63e4350 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d157fa640 1 -- 172.21.15.87:0/1907126881 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28012c50 con 0x560fd63e4350 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 -- 172.21.15.87:0/1907126881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 msgr2=0x560fd63d4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800a010 tx=0x7f6d28016040 comp rx=0 tx=0).stop 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 -- 172.21.15.87:0/1907126881 shutdown_connections 2024-06-06T08:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63e4350 0x560fd63d4ba0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd63d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 --2- 172.21.15.87:0/1907126881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63d9170 0x560fd63dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 -- 172.21.15.87:0/1907126881 >> 172.21.15.87:0/1907126881 conn(0x560fd63d5680 msgr2=0x560fd63d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 -- 172.21.15.87:0/1907126881 shutdown_connections 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.566+0000 7f6d3876d740 1 -- 172.21.15.87:0/1907126881 wait complete. 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 -- start start 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 Processor -- start 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd6457640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301ad570 con 0x560fd63d9170 2024-06-06T08:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301ae9e0 con 0x560fd63e4350 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d301adc40 con 0x560fd63d9a90 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d3876d740 10 client.0 _ll_register_callbacks cb 0x7f6d20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60816/0 (socket says 172.21.15.87:60816) 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/2400882058 learned_addr learned my addr 172.21.15.87:0/2400882058 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.567+0000 7f6d2effd640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd6457640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/2400882058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 msgr2=0x560fd6457640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd6457640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 msgr2=0x560fd63e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:00:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 -- 172.21.15.87:0/2400882058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd63e7fe0 con 0x560fd63d9170 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d2f7fe640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d2801f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9163121b38e96d7a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a070 con 0x560fd63d9170 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6d28005090 con 0x560fd63d9170 2024-06-06T08:00:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d28024070 con 0x560fd63d9170 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d28012860 con 0x560fd63d9170 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d3876d740 10 client.17388 fetch_fsmap learned FSMap version 41 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560fd645e7d0 con 0x560fd63d9170 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.568+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d28024070 con 0x560fd63d9170 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f6d2803a490 con 0x560fd63d9170 2024-06-06T08:00:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 10 client.17388 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 20 client.17388 populate_metadata read hostname 'smithi087' 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd645ee80 0x560fd64612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 4 client.17388 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fd634ac20 con 0x560fd645ee80 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d2effd640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd645ee80 0x560fd64612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:00:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d0effd640 10 client.17388 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.569+0000 7f6d2effd640 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd645ee80 0x560fd64612a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a7de493146ee40 server_cookie=c0f2444bbefd752 in_seq=0 out_seq=0 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d0effd640 1 -- 172.21.15.87:0/2400882058 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560fd634ac20 con 0x560fd645ee80 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d0effd640 10 client.17388 handle_command_reply: tid=0 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 client.17388 shutdown 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 msgr2=0x560fd6457100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d2801f460 comp rx=0 tx=0).stop 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 shutdown_connections 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560fd645ee80 0x560fd64612a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fd63e4350 0x560fd63e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560fd63d9a90 0x560fd6457640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d0effd640 0 client.17388 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.650+0000 7f6d3876d740 1 --2- 172.21.15.87:0/2400882058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560fd63d9170 0x560fd6457100 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 >> 172.21.15.87:0/2400882058 conn(0x560fd63d5680 msgr2=0x560fd63d47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 shutdown_connections 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3876d740 1 -- 172.21.15.87:0/2400882058 wait complete. 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3876d740 20 client.17388 trim_cache size 0 max 16384 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:00:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:00:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 msgr2=0x7f6d180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 0x7f6d180a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d24097fd0 tx=0x7f6d24070040 comp rx=0 tx=0).stop 2024-06-06T08:00:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 msgr2=0x7f6d30198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c920 tx=0x7f6d2400cde0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 shutdown_connections 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d180a2da0 0x7f6d180a5260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3010bf70 0x7f6d3019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d30103550 0x7f6d30198750 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 --2- 172.21.15.87:0/3715768776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d30102c30 0x7f6d30198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 >> 172.21.15.87:0/3715768776 conn(0x7f6d30077a20 msgr2=0x7f6d300feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 shutdown_connections 2024-06-06T08:00:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:00:51.651+0000 7f6d3597e640 1 -- 172.21.15.87:0/3715768776 wait complete. 2024-06-06T08:00:51.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667757 5.51 MB/sec execute 535 sec latency 3752.991 ms 2024-06-06T08:00:52.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:51 smithi195 ceph-mon[37040]: pgmap v404: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 425 KiB/s wr, 77 op/s 2024-06-06T08:00:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:51 smithi181 ceph-mon[36790]: pgmap v404: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 425 KiB/s wr, 77 op/s 2024-06-06T08:00:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:51 smithi087 ceph-mon[32276]: pgmap v404: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 425 KiB/s wr, 77 op/s 2024-06-06T08:00:52.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 667757 5.50 MB/sec execute 536 sec latency 4753.050 ms 2024-06-06T08:00:53.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 668844 5.49 MB/sec execute 537 sec latency 4903.221 ms 2024-06-06T08:00:54.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:53 smithi195 ceph-mon[37040]: pgmap v405: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 708 KiB/s wr, 106 op/s 2024-06-06T08:00:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:53 smithi181 ceph-mon[36790]: pgmap v405: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 708 KiB/s wr, 106 op/s 2024-06-06T08:00:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:53 smithi087 ceph-mon[32276]: pgmap v405: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 708 KiB/s wr, 106 op/s 2024-06-06T08:00:54.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671227 5.51 MB/sec execute 538 sec latency 32.052 ms 2024-06-06T08:00:55.692 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671236 5.50 MB/sec execute 539 sec latency 995.221 ms 2024-06-06T08:00:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:55 smithi195 ceph-mon[37040]: pgmap v406: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 650 KiB/s wr, 59 op/s 2024-06-06T08:00:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:55 smithi181 ceph-mon[36790]: pgmap v406: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 650 KiB/s wr, 59 op/s 2024-06-06T08:00:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:55 smithi087 ceph-mon[32276]: pgmap v406: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 650 KiB/s wr, 59 op/s 2024-06-06T08:00:56.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671236 5.49 MB/sec execute 540 sec latency 1995.305 ms 2024-06-06T08:00:57.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671236 5.48 MB/sec execute 541 sec latency 2995.404 ms 2024-06-06T08:00:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:57 smithi195 ceph-mon[37040]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 671 KiB/s wr, 74 op/s 2024-06-06T08:00:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:57 smithi181 ceph-mon[36790]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 671 KiB/s wr, 74 op/s 2024-06-06T08:00:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:57 smithi087 ceph-mon[32276]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 671 KiB/s wr, 74 op/s 2024-06-06T08:00:58.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671303 5.47 MB/sec execute 542 sec latency 3146.084 ms 2024-06-06T08:00:59.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671303 5.46 MB/sec execute 543 sec latency 1750.099 ms 2024-06-06T08:01:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:00:59 smithi195 ceph-mon[37040]: pgmap v408: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 101 op/s 2024-06-06T08:01:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:00:59 smithi181 ceph-mon[36790]: pgmap v408: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 101 op/s 2024-06-06T08:01:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:00:59 smithi087 ceph-mon[32276]: pgmap v408: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 101 op/s 2024-06-06T08:01:00.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671303 5.45 MB/sec execute 544 sec latency 2750.137 ms 2024-06-06T08:01:01.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671303 5.44 MB/sec execute 545 sec latency 3750.249 ms 2024-06-06T08:01:02.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:01 smithi195 ceph-mon[37040]: pgmap v409: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 583 KiB/s wr, 75 op/s 2024-06-06T08:01:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:01 smithi181 ceph-mon[36790]: pgmap v409: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 583 KiB/s wr, 75 op/s 2024-06-06T08:01:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:01 smithi087 ceph-mon[32276]: pgmap v409: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 583 KiB/s wr, 75 op/s 2024-06-06T08:01:02.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671303 5.43 MB/sec execute 546 sec latency 4750.354 ms 2024-06-06T08:01:03.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 671360 5.42 MB/sec execute 547 sec latency 4905.738 ms 2024-06-06T08:01:04.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:03 smithi195 ceph-mon[37040]: pgmap v410: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-06-06T08:01:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:03 smithi181 ceph-mon[36790]: pgmap v410: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-06-06T08:01:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:03 smithi087 ceph-mon[32276]: pgmap v410: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-06-06T08:01:04.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 672272 5.42 MB/sec execute 548 sec latency 104.651 ms 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- 172.21.15.87:0/2195026059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 msgr2=0x7f1ebc106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/2195026059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc106c70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009930 tx=0x7f1eb802a420 comp rx=0 tx=0).stop 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- 172.21.15.87:0/2195026059 shutdown_connections 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/2195026059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc101290 0x7f1ebc10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/2195026059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc1008d0 0x7f1ebc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/2195026059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc106c70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- 172.21.15.87:0/2195026059 >> 172.21.15.87:0/2195026059 conn(0x7f1ebc077a20 msgr2=0x7f1ebc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- 172.21.15.87:0/2195026059 shutdown_connections 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- 172.21.15.87:0/2195026059 wait complete. 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 Processor -- start 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.745+0000 7f1ec34bd640 1 -- start start 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc1008d0 0x7f1ebc196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc110ca0 con 0x7f1ebc106890 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc110b20 con 0x7f1ebc1008d0 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec34bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc110e20 con 0x7f1ebc101290 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec1a5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:33228/0 (socket says 172.21.15.87:33228) 2024-06-06T08:01:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec1a5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48032/0 (socket says 172.21.15.87:48032) 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 -- 172.21.15.87:0/532472360 learned_addr learned my addr 172.21.15.87:0/532472360 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec1259640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc1008d0 0x7f1ebc196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 -- 172.21.15.87:0/532472360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc1008d0 msgr2=0x7f1ebc196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc1008d0 0x7f1ebc196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 -- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 msgr2=0x7f1ebc197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 -- 172.21.15.87:0/532472360 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc1a6670 con 0x7f1ebc101290 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.746+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00ea20 tx=0x7f1eac00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.747+0000 7f1eb27fc640 1 -- 172.21.15.87:0/532472360 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00cd80 con 0x7f1ebc101290 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.747+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ebc1a6960 con 0x7f1ebc101290 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.747+0000 7f1eb27fc640 1 -- 172.21.15.87:0/532472360 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac010910 con 0x7f1ebc101290 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.747+0000 7f1eb27fc640 1 -- 172.21.15.87:0/532472360 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac018920 con 0x7f1ebc101290 2024-06-06T08:01:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.747+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ebc1a6f00 con 0x7f1ebc101290 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.749+0000 7f1eb27fc640 1 -- 172.21.15.87:0/532472360 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1eac010ac0 con 0x7f1ebc101290 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 Processor -- start 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 -- start start 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1eb27fc640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 0x7f1ea80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 0x564a1651b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16525e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16528740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a1647f4f0 con 0x564a1651ba90 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a1641d870 con 0x564a1651b170 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a1647b4b0 con 0x564a16526350 2024-06-06T08:01:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16525e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 0x7f1ea80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 0x564a1651b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 0x564a1651b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:51902/0 (socket says 172.21.15.87:51902) 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec1259640 1 -- 172.21.15.87:0/4276956087 learned_addr learned my addr 172.21.15.87:0/4276956087 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1eb27fc640 1 -- 172.21.15.87:0/532472360 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f1eac0cfc00 con 0x7f1ebc101290 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16528740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 -- 172.21.15.87:0/4276956087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 msgr2=0x564a16528740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16528740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 -- 172.21.15.87:0/4276956087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 msgr2=0x564a1651b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 0x564a1651b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.750+0000 7f1ec0a58640 1 -- 172.21.15.87:0/4276956087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a164399b0 con 0x564a1651ba90 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16525e10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0ccf30 tx=0x7f1eac09ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a687e3e5ae504f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 0x7f1ea80a5260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fd0 tx=0x7f1eb4013680 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ea77fe640 1 -- 172.21.15.87:0/4276956087 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac09bdb0 con 0x564a1651ba90 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ea77fe640 1 -- 172.21.15.87:0/4276956087 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac0a00b0 con 0x564a1651ba90 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ea77fe640 1 -- 172.21.15.87:0/4276956087 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0b5450 con 0x564a1651ba90 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ec61d1740 1 -- 172.21.15.87:0/4276956087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 msgr2=0x564a16525e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.751+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16525e10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0ccf30 tx=0x7f1eac09ba20 comp rx=0 tx=0).stop 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- 172.21.15.87:0/4276956087 shutdown_connections 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16528740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16525e10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/4276956087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651b170 0x564a1651b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- 172.21.15.87:0/4276956087 >> 172.21.15.87:0/4276956087 conn(0x564a16517680 msgr2=0x564a16517a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- 172.21.15.87:0/4276956087 shutdown_connections 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- 172.21.15.87:0/4276956087 wait complete. 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- start start 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 Processor -- start 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1651b170 0x564a165233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a16526350 0x564a16523e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564a16528f80 con 0x564a1651ba90 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564a16408ae0 con 0x564a16526350 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a16529100 con 0x564a1651b170 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.752+0000 7f1ec61d1740 10 client.0 _ll_register_callbacks cb 0x564a164419b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1651b170 0x564a165233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48050/0 (socket says 172.21.15.87:48050) 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 -- 172.21.15.87:0/2333887348 learned_addr learned my addr 172.21.15.87:0/2333887348 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a16526350 0x564a16523e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 -- 172.21.15.87:0/2333887348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1651b170 msgr2=0x564a165233c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1651b170 0x564a165233c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 -- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a16526350 msgr2=0x564a16523e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a16526350 0x564a16523e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 -- 172.21.15.87:0/2333887348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a16193210 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0a4440 tx=0x7f1eac0b90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27897df8c9353f2f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0b9540 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f1eac09b200 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eac0a00b0 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1eac09bdb0 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec61d1740 10 client.17400 fetch_fsmap learned FSMap version 41 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564a1625f620 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.753+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac0a00b0 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f1eac086040 con 0x564a1651ba90 2024-06-06T08:01:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 10 client.17400 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 20 client.17400 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a16538830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 4 client.17400 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a16492f10 con 0x564a16536440 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec1259640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a16538830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ec1259640 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a16538830 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed978a1db97ecadf server_cookie=fba3c3980a712631 in_seq=0 out_seq=0 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.754+0000 7f1ea4ff9640 10 client.17400 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/2333887348 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a16492f10 con 0x564a16536440 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ea4ff9640 10 client.17400 handle_command_reply: tid=0 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 client.17400 shutdown 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 msgr2=0x564a16523900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0a4440 tx=0x7f1eac0b90f0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 shutdown_connections 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a16538830 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ea4ff9640 0 client.17400 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a16526350 0x564a16523e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651ba90 0x564a16523900 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2333887348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a1651b170 0x564a165233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 >> 172.21.15.87:0/2333887348 conn(0x564a16517680 msgr2=0x564a1651d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 shutdown_connections 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2333887348 wait complete. 2024-06-06T08:01:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.755+0000 7f1ec61d1740 20 client.17400 trim_cache size 0 max 16384 2024-06-06T08:01:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 Processor -- start 2024-06-06T08:01:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 -- start start 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16523c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16531a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1b3a80 con 0x564a1651b170 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1b1350 con 0x564a1651ba90 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1af1c0 con 0x564a16526350 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48060/0 (socket says 172.21.15.87:48060) 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 -- 172.21.15.87:0/2187898665 learned_addr learned my addr 172.21.15.87:0/2187898665 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16531a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16523c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 -- 172.21.15.87:0/2187898665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 msgr2=0x564a16531a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16531a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 -- 172.21.15.87:0/2187898665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 msgr2=0x564a16523c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16523c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.775+0000 7f1ec1259640 1 -- 172.21.15.87:0/2187898665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a165174a0 con 0x564a1651b170 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.776+0000 7f1ec1259640 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1eb802a940 tx=0x7f1eb80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d03e3174069bfe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.776+0000 7f1ea6ffd640 1 -- 172.21.15.87:0/2187898665 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb804d020 con 0x564a1651b170 2024-06-06T08:01:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.776+0000 7f1ea6ffd640 1 -- 172.21.15.87:0/2187898665 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8031a60 con 0x564a1651b170 2024-06-06T08:01:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.776+0000 7f1ea6ffd640 1 -- 172.21.15.87:0/2187898665 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8035040 con 0x564a1651b170 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2187898665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 msgr2=0x564a16516f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1eb802a940 tx=0x7f1eb80319f0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2187898665 shutdown_connections 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a16531a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16523c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/2187898665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a16516f60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2187898665 >> 172.21.15.87:0/2187898665 conn(0x564a16517680 msgr2=0x564a16531350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2187898665 shutdown_connections 2024-06-06T08:01:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.780+0000 7f1ec61d1740 1 -- 172.21.15.87:0/2187898665 wait complete. 2024-06-06T08:01:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 -- start start 2024-06-06T08:01:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 Processor -- start 2024-06-06T08:01:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16538f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a1659ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1af810 con 0x564a1651b170 2024-06-06T08:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1b0c80 con 0x564a1651ba90 2024-06-06T08:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc1afee0 con 0x564a16526350 2024-06-06T08:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec61d1740 10 client.0 _ll_register_callbacks cb 0x7f1eac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec1259640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48072/0 (socket says 172.21.15.87:48072) 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec1259640 1 -- 172.21.15.87:0/828390796 learned_addr learned my addr 172.21.15.87:0/828390796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.781+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16538f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1a5a640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a1659ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 -- 172.21.15.87:0/828390796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 msgr2=0x564a1659ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a1659ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 -- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 msgr2=0x564a16538f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16538f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 -- 172.21.15.87:0/828390796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1659f2b0 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec1259640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8004ed0 tx=0x7f1eb802ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38961250a6c17bc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb804d030 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f1eb8005000 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.782+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a1648cc20 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.783+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1eb8065050 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.783+0000 7f1ec61d1740 10 client.17406 fetch_fsmap learned FSMap version 41 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.783+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564a165a0440 con 0x564a1651b170 2024-06-06T08:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.783+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564a1648cc20 con 0x564a1651b170 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f1eb8048b60 con 0x564a1651b170 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 10 client.17406 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 20 client.17406 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a165a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 4 client.17406 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.784+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a16486f70 con 0x564a16536440 2024-06-06T08:01:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.785+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a165a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.787+0000 7f1ea4ff9640 10 client.17406 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.787+0000 7f1ec0a58640 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a165a2af0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f32137ce47463d9 server_cookie=e73529d509c9d5de in_seq=0 out_seq=0 2024-06-06T08:01:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ea4ff9640 1 -- 172.21.15.87:0/828390796 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564a16486f70 con 0x564a16536440 2024-06-06T08:01:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ea4ff9640 10 client.17406 handle_command_reply: tid=0 2024-06-06T08:01:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 client.17406 shutdown 2024-06-06T08:01:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 msgr2=0x564a165389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8004ed0 tx=0x7f1eb802ab60 comp rx=0 tx=0).stop 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 shutdown_connections 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564a16536440 0x564a165a2af0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a16526350 0x564a1659ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a1651ba90 0x564a16538f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 --2- 172.21.15.87:0/828390796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564a1651b170 0x564a165389f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 >> 172.21.15.87:0/828390796 conn(0x564a16517680 msgr2=0x564a16531350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ea4ff9640 0 client.17406 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 shutdown_connections 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 1 -- 172.21.15.87:0/828390796 wait complete. 2024-06-06T08:01:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.910+0000 7f1ec61d1740 20 client.17406 trim_cache size 0 max 16384 2024-06-06T08:01:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:01:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:01:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:01:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 msgr2=0x7f1ea80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 0x7f1ea80a5260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fd0 tx=0x7f1eb4013680 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 msgr2=0x7f1ebc1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00ea20 tx=0x7f1eac00eee0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 shutdown_connections 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1ea80a2da0 0x7f1ea80a5260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ebc106890 0x7f1ebc197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ebc101290 0x7f1ebc1973d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 --2- 172.21.15.87:0/532472360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc1008d0 0x7f1ebc196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 >> 172.21.15.87:0/532472360 conn(0x7f1ebc077a20 msgr2=0x7f1ebc103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 shutdown_connections 2024-06-06T08:01:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:04.911+0000 7f1ec34bd640 1 -- 172.21.15.87:0/532472360 wait complete. 2024-06-06T08:01:05.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674392 5.43 MB/sec execute 549 sec latency 72.386 ms 2024-06-06T08:01:05.984 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T08:01:05.984 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T08:01:05.984 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/2052093015 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a63c9da40 con 0x560a63c70800 2024-06-06T08:01:05.984 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T08:01:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-mon[37040]: pgmap v411: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 328 KiB/s wr, 54 op/s 2024-06-06T08:01:06.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:05 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T08:01:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:05 smithi181 ceph-mon[36790]: pgmap v411: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 328 KiB/s wr, 54 op/s 2024-06-06T08:01:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:05 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T08:01:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:05 smithi087 ceph-mon[32276]: pgmap v411: 129 pgs: 129 active+clean; 85 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 328 KiB/s wr, 54 op/s 2024-06-06T08:01:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:05 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T08:01:06.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674835 5.43 MB/sec execute 550 sec latency 822.890 ms 2024-06-06T08:01:07.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:06 smithi195 ceph-mon[37040]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T08:01:07.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:06 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/2710213509,v1:172.21.15.195:6837/2710213509] up:boot 2024-06-06T08:01:07.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:06 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:07.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:06 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:07.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:06 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:06 smithi181 ceph-mon[36790]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T08:01:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:06 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/2710213509,v1:172.21.15.195:6837/2710213509] up:boot 2024-06-06T08:01:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:06 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:06 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:06 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:06 smithi087 ceph-mon[32276]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T08:01:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:06 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/2710213509,v1:172.21.15.195:6837/2710213509] up:boot 2024-06-06T08:01:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:06 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:06 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:06 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:07.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674835 5.42 MB/sec execute 551 sec latency 1822.932 ms 2024-06-06T08:01:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:07 smithi195 ceph-mon[37040]: pgmap v413: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 508 KiB/s wr, 71 op/s 2024-06-06T08:01:08.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:07 smithi181 ceph-mon[36790]: pgmap v413: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 508 KiB/s wr, 71 op/s 2024-06-06T08:01:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:07 smithi087 ceph-mon[32276]: pgmap v413: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 508 KiB/s wr, 71 op/s 2024-06-06T08:01:08.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674902 5.41 MB/sec execute 552 sec latency 1946.785 ms 2024-06-06T08:01:09.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674902 5.40 MB/sec execute 553 sec latency 1748.880 ms 2024-06-06T08:01:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:09 smithi195 ceph-mon[37040]: pgmap v414: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:09 smithi181 ceph-mon[36790]: pgmap v414: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:09 smithi087 ceph-mon[32276]: pgmap v414: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:10.693 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674902 5.39 MB/sec execute 554 sec latency 2748.928 ms 2024-06-06T08:01:11.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674902 5.38 MB/sec execute 555 sec latency 3749.070 ms 2024-06-06T08:01:12.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:12 smithi195 ceph-mon[37040]: pgmap v415: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:12 smithi181 ceph-mon[36790]: pgmap v415: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:12 smithi087 ceph-mon[32276]: pgmap v415: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 899 KiB/s wr, 143 op/s 2024-06-06T08:01:12.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 674902 5.37 MB/sec execute 556 sec latency 4749.125 ms 2024-06-06T08:01:13.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 676046 5.37 MB/sec execute 557 sec latency 4898.391 ms 2024-06-06T08:01:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:14 smithi195 ceph-mon[37040]: pgmap v416: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:14 smithi181 ceph-mon[36790]: pgmap v416: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:14 smithi087 ceph-mon[32276]: pgmap v416: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:14.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678381 5.38 MB/sec execute 558 sec latency 93.578 ms 2024-06-06T08:01:15.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678381 5.37 MB/sec execute 559 sec latency 1093.675 ms 2024-06-06T08:01:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:15 smithi087 ceph-mon[32276]: pgmap v417: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:15 smithi087 ceph-mon[32276]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T08:01:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:15 smithi195 ceph-mon[37040]: pgmap v417: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:16.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:15 smithi195 ceph-mon[37040]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T08:01:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:15 smithi181 ceph-mon[36790]: pgmap v417: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-06T08:01:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:15 smithi181 ceph-mon[36790]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T08:01:16.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678381 5.36 MB/sec execute 560 sec latency 2093.757 ms 2024-06-06T08:01:17.219 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.3 2024-06-06T08:01:17.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:01:17.473 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:01:17.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678381 5.36 MB/sec execute 561 sec latency 3093.881 ms 2024-06-06T08:01:17.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:17 smithi087 ceph-mon[32276]: pgmap v419: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 -- 172.21.15.87:0/3296012448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58103550 msgr2=0x7f8d5810ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/3296012448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58103550 0x7f8d5810ba30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009900 tx=0x7f8d4002a510 comp rx=0 tx=0).stop 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 -- 172.21.15.87:0/3296012448 shutdown_connections 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/3296012448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5810bf70 0x7f8d5810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/3296012448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58103550 0x7f8d5810ba30 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/3296012448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d58102c30 0x7f8d58103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 -- 172.21.15.87:0/3296012448 >> 172.21.15.87:0/3296012448 conn(0x7f8d58077a20 msgr2=0x7f8d58077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 -- 172.21.15.87:0/3296012448 shutdown_connections 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.025+0000 7f8d5e54f640 1 -- 172.21.15.87:0/3296012448 wait complete. 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 Processor -- start 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 -- start start 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d58103550 0x7f8d58198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 0x7f8d5819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58111dc0 con 0x7f8d58102c30 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58111c40 con 0x7f8d5810bf70 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5e54f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58111f40 con 0x7f8d58103550 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48264/0 (socket says 172.21.15.87:48264) 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 -- 172.21.15.87:0/1715365596 learned_addr learned my addr 172.21.15.87:0/1715365596 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d58103550 0x7f8d58198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 0x7f8d5819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 -- 172.21.15.87:0/1715365596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d58103550 msgr2=0x7f8d58198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d58103550 0x7f8d58198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 -- 172.21.15.87:0/1715365596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 msgr2=0x7f8d5819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 0x7f8d5819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d57fff640 1 -- 172.21.15.87:0/1715365596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5819cfd0 con 0x7f8d58102c30 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.026+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 0x7f8d5819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c890 tx=0x7f8d4800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d55ffb640 1 -- 172.21.15.87:0/1715365596 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004220 con 0x7f8d58102c30 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5819d2c0 con 0x7f8d58102c30 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d55ffb640 1 -- 172.21.15.87:0/1715365596 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d480043c0 con 0x7f8d58102c30 2024-06-06T08:01:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d55ffb640 1 -- 172.21.15.87:0/1715365596 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d480049e0 con 0x7f8d58102c30 2024-06-06T08:01:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.027+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d581a4b50 con 0x7f8d58102c30 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d55ffb640 1 -- 172.21.15.87:0/1715365596 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d480079b0 con 0x7f8d58102c30 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 Processor -- start 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 -- start start 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd656550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd660e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5606dd5ba4f0 con 0x5606dd661350 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5606dd558870 con 0x5606dd656a90 2024-06-06T08:01:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d6125a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606dd5b64b0 con 0x5606dd656170 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d55ffb640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 0x7f8d440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48274/0 (socket says 172.21.15.87:48274) 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d4ffff640 1 -- 172.21.15.87:0/1627791868 learned_addr learned my addr 172.21.15.87:0/1627791868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.030+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd660e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd656550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 0x7f8d440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 -- 172.21.15.87:0/1627791868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 msgr2=0x5606dd656550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d55ffb640 1 -- 172.21.15.87:0/1715365596 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8d480cf8d0 con 0x7f8d58102c30 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd656550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 -- 172.21.15.87:0/1627791868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 msgr2=0x5606dd660e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd660e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 -- 172.21.15.87:0/1627791868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dd5749b0 con 0x5606dd661350 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd656550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8d4002ab00 tx=0x7f8d400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5234c85ec8ace5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 0x7f8d440a5260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005c40 tx=0x7f8d50009280 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4e7fc640 1 -- 172.21.15.87:0/1627791868 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40035040 con 0x5606dd661350 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4e7fc640 1 -- 172.21.15.87:0/1627791868 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d400047a0 con 0x5606dd661350 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.031+0000 7f8d4e7fc640 1 -- 172.21.15.87:0/1627791868 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40003a10 con 0x5606dd661350 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- 172.21.15.87:0/1627791868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 msgr2=0x5606dd663740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8d4002ab00 tx=0x7f8d400040c0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- 172.21.15.87:0/1627791868 shutdown_connections 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd663740 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd660e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1627791868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd656550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- 172.21.15.87:0/1627791868 >> 172.21.15.87:0/1627791868 conn(0x5606dd652680 msgr2=0x5606dd652a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- 172.21.15.87:0/1627791868 shutdown_connections 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- 172.21.15.87:0/1627791868 wait complete. 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 -- start start 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.032+0000 7f8d6125a740 1 Processor -- start 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd65e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd661350 0x5606dd65ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5606dd663f20 con 0x5606dd656a90 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5606dd543ae0 con 0x5606dd661350 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606dd6640a0 con 0x5606dd656170 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 10 client.0 _ll_register_callbacks cb 0x5606dd57c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48286/0 (socket says 172.21.15.87:48286) 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 -- 172.21.15.87:0/757982422 learned_addr learned my addr 172.21.15.87:0/757982422 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d57fff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd65e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d5caec640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd661350 0x5606dd65ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 -- 172.21.15.87:0/757982422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 msgr2=0x5606dd65e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd65e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 -- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd661350 msgr2=0x5606dd65ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd661350 0x5606dd65ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 -- 172.21.15.87:0/757982422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dd2ce210 con 0x5606dd656a90 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8d400057c0 tx=0x7f8d40005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73ba6de5da48ee6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40035040 con 0x5606dd656a90 2024-06-06T08:01:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d400047a0 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.033+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8d40005000 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8d40005200 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 10 client.17424 fetch_fsmap learned FSMap version 43 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5606dd39a620 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d400047a0 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f8d4003c670 con 0x5606dd656a90 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 10 client.17424 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 20 client.17424 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd673670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 4 client.17424 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606dd5cdf10 con 0x5606dd671280 2024-06-06T08:01:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.034+0000 7f8d57fff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd673670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.035+0000 7f8d377fe640 10 client.17424 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.035+0000 7f8d57fff640 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd673670 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8589459fd8ed81 server_cookie=f1eafe7dae2b930f in_seq=0 out_seq=0 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.035+0000 7f8d377fe640 1 -- 172.21.15.87:0/757982422 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5606dd5cdf10 con 0x5606dd671280 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.035+0000 7f8d377fe640 10 client.17424 handle_command_reply: tid=0 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.035+0000 7f8d6125a740 1 client.17424 shutdown 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 msgr2=0x5606dd65e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8d400057c0 tx=0x7f8d40005a40 comp rx=0 tx=0).stop 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 shutdown_connections 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd673670 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd661350 0x5606dd65ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d377fe640 0 client.17424 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656a90 0x5606dd65e7a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 --2- 172.21.15.87:0/757982422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd65e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 >> 172.21.15.87:0/757982422 conn(0x5606dd652680 msgr2=0x5606dd6583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 shutdown_connections 2024-06-06T08:01:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 1 -- 172.21.15.87:0/757982422 wait complete. 2024-06-06T08:01:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.036+0000 7f8d6125a740 20 client.17424 trim_cache size 0 max 16384 2024-06-06T08:01:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 Processor -- start 2024-06-06T08:01:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 -- start start 2024-06-06T08:01:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd658b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd678c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b1690 con 0x5606dd661350 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581aef60 con 0x5606dd656a90 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d6125a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581acdd0 con 0x5606dd656170 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48296/0 (socket says 172.21.15.87:48296) 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 -- 172.21.15.87:0/1320717767 learned_addr learned my addr 172.21.15.87:0/1320717767 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd658b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd678c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 -- 172.21.15.87:0/1320717767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 msgr2=0x5606dd658b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd658b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 -- 172.21.15.87:0/1320717767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 msgr2=0x5606dd678c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd678c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 -- 172.21.15.87:0/1320717767 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dd6568c0 con 0x5606dd661350 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d57fff640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd658b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd678c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.051+0000 7f8d5caec640 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8d500107b0 tx=0x7f8d50009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46d4d31961b6b12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d4dffb640 1 -- 172.21.15.87:0/1320717767 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50010d30 con 0x5606dd661350 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d4dffb640 1 -- 172.21.15.87:0/1320717767 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d5001f8f0 con 0x5606dd661350 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d4dffb640 1 -- 172.21.15.87:0/1320717767 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50005470 con 0x5606dd661350 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 -- 172.21.15.87:0/1320717767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 msgr2=0x5606dd6791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8d500107b0 tx=0x7f8d50009700 comp rx=0 tx=0).stop 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 -- 172.21.15.87:0/1320717767 shutdown_connections 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd661350 0x5606dd6791b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd678c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 --2- 172.21.15.87:0/1320717767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd656170 0x5606dd658b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 -- 172.21.15.87:0/1320717767 >> 172.21.15.87:0/1320717767 conn(0x5606dd652680 msgr2=0x5606dd652a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 -- 172.21.15.87:0/1320717767 shutdown_connections 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.052+0000 7f8d6125a740 1 -- 172.21.15.87:0/1320717767 wait complete. 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 -- start start 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 Processor -- start 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd664f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd661350 0x5606dd665680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581ad420 con 0x5606dd656170 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581ae890 con 0x5606dd656a90 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581adaf0 con 0x5606dd661350 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d6125a740 10 client.0 _ll_register_callbacks cb 0x7f8d4809b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48298/0 (socket says 172.21.15.87:48298) 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 -- 172.21.15.87:0/2806088122 learned_addr learned my addr 172.21.15.87:0/2806088122 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd664f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d5caec640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd661350 0x5606dd665680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 -- 172.21.15.87:0/2806088122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd661350 msgr2=0x5606dd665680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd661350 0x5606dd665680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 -- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 msgr2=0x5606dd664f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd664f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.053+0000 7f8d57fff640 1 -- 172.21.15.87:0/2806088122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dd6dac40 con 0x5606dd656170 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d57fff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8d48097e30 tx=0x7f8d48005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3ffce694e9cd2d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4801d040 con 0x5606dd656170 2024-06-06T08:01:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8d480cce80 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5001f8f0 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8d4809fd80 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 10 client.17430 fetch_fsmap learned FSMap version 43 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5606dd6dbdc0 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d5001f8f0 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f8d48007ca0 con 0x5606dd656170 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 10 client.17430 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 20 client.17430 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd6de450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 4 client.17430 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606dd5c7c20 con 0x5606dd671280 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.054+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd6de450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.055+0000 7f8d4ffff640 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd6de450 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a0d5aeabd91c29 server_cookie=79a22490a19689ed in_seq=0 out_seq=0 2024-06-06T08:01:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.055+0000 7f8d377fe640 10 client.17430 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.155+0000 7f8d377fe640 1 -- 172.21.15.87:0/2806088122 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5606dd5c7c20 con 0x5606dd671280 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.155+0000 7f8d377fe640 10 client.17430 handle_command_reply: tid=0 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.155+0000 7f8d6125a740 1 client.17430 shutdown 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.155+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 msgr2=0x5606dd669bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.155+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8d48097e30 tx=0x7f8d48005300 comp rx=0 tx=0).stop 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 shutdown_connections 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5606dd671280 0x5606dd6de450 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606dd661350 0x5606dd665680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d377fe640 0 client.17430 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606dd656a90 0x5606dd664f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 --2- 172.21.15.87:0/2806088122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5606dd656170 0x5606dd669bd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 >> 172.21.15.87:0/2806088122 conn(0x5606dd652680 msgr2=0x5606dd65e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 shutdown_connections 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 1 -- 172.21.15.87:0/2806088122 wait complete. 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.156+0000 7f8d6125a740 20 client.17430 trim_cache size 0 max 16384 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:01:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 msgr2=0x7f8d440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 0x7f8d440a5260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005c40 tx=0x7f8d50009280 comp rx=0 tx=0).stop 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 msgr2=0x7f8d58197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c890 tx=0x7f8d4800cd50 comp rx=0 tx=0).stop 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 shutdown_connections 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8d440a2da0 0x7f8d440a5260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5810bf70 0x7f8d5819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d58103550 0x7f8d58198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 --2- 172.21.15.87:0/1715365596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d58102c30 0x7f8d58197fe0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 >> 172.21.15.87:0/1715365596 conn(0x7f8d58077a20 msgr2=0x7f8d580fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 shutdown_connections 2024-06-06T08:01:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:18.157+0000 7f8d5e54f640 1 -- 172.21.15.87:0/1715365596 wait complete. 2024-06-06T08:01:18.244 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:17 smithi195 ceph-mon[37040]: pgmap v419: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-06T08:01:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:17 smithi181 ceph-mon[36790]: pgmap v419: 129 pgs: 129 active+clean; 78 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-06-06T08:01:18.509 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T08:01:18.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T08:01:18.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:18 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/1976322306 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9a0ad5a40 con 0x55f9a0aa8800 2024-06-06T08:01:18.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:01:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T08:01:18.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678448 5.35 MB/sec execute 562 sec latency 3252.238 ms 2024-06-06T08:01:18.978 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:18 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T08:01:19.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:18 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T08:01:19.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:18 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T08:01:19.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 -- 172.21.15.87:0/4013508261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 msgr2=0x7fb01c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:19.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 --2- 172.21.15.87:0/4013508261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c102b20 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0099c0 tx=0x7fb00c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 -- 172.21.15.87:0/4013508261 shutdown_connections 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 --2- 172.21.15.87:0/4013508261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c107840 0x7fb01c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 --2- 172.21.15.87:0/4013508261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c102b20 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 --2- 172.21.15.87:0/4013508261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c1030f0 0x7fb01c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 -- 172.21.15.87:0/4013508261 >> 172.21.15.87:0/4013508261 conn(0x7fb01c0795f0 msgr2=0x7fb01c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 -- 172.21.15.87:0/4013508261 shutdown_connections 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.521+0000 7fb020df9700 1 -- 172.21.15.87:0/4013508261 wait complete. 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 Processor -- start 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 -- start start 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 0x7fb01c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c107840 0x7fb01c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:19.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c112760 con 0x7fb01c102740 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1125e0 con 0x7fb01c1030f0 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb020df9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1128e0 con 0x7fb01c107840 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb019d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 0x7fb01c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48320/0 (socket says 172.21.15.87:48320) 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb019d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 0x7fb01c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:51294/0 (socket says 172.21.15.87:51294) 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb01a59c700 1 -- 172.21.15.87:0/802458745 learned_addr learned my addr 172.21.15.87:0/802458745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.522+0000 7fb01ad9d700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c107840 0x7fb01c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 -- 172.21.15.87:0/802458745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c107840 msgr2=0x7fb01c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c107840 0x7fb01c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 -- 172.21.15.87:0/802458745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 msgr2=0x7fb01c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 0x7fb01c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 -- 172.21.15.87:0/802458745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c199140 con 0x7fb01c102740 2024-06-06T08:01:19.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb01a59c700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006010 tx=0x7fb00c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c026070 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c004cd0 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1993d0 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c003ba0 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.523+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a56f0 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.524+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c069180 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.525+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb00c029070 con 0x7fb01c102740 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.526+0000 7fb00b7fe700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 0x7fb0040a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:19.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.526+0000 7fb019d9b700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 0x7fb0040a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:19.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.526+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fb00c0d0500 con 0x7fb01c102740 2024-06-06T08:01:19.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.526+0000 7fb019d9b700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 0x7fb0040a5250 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb01c1071e0 tx=0x7fb01000b410 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:19.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.529+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb00c030020 con 0x7fb01c102740 2024-06-06T08:01:19.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678448 5.34 MB/sec execute 563 sec latency 1734.416 ms 2024-06-06T08:01:19.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.778+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb01c02dca0 con 0x7fb01c102740 2024-06-06T08:01:19.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.779+0000 7fb00b7fe700 1 -- 172.21.15.87:0/802458745 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11878 (secure 0 0 0) 0x7fb00c013070 con 0x7fb01c102740 2024-06-06T08:01:19.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:01:19.779 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:01:18.835086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26623":{"gid":26623,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1019415876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1019415876},{"type":"v1","addr":"172.21.15.195:6837","nonce":1019415876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.781+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 msgr2=0x7fb0040a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 0x7fb0040a5250 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb01c1071e0 tx=0x7fb01000b410 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 msgr2=0x7fb01c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006010 tx=0x7fb00c009f80 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 shutdown_connections 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb0040a2d90 0x7fb0040a5250 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c107840 0x7fb01c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb01c102740 0x7fb01c19e820 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 --2- 172.21.15.87:0/802458745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01c1030f0 0x7fb01c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 >> 172.21.15.87:0/802458745 conn(0x7fb01c0795f0 msgr2=0x7fb01c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 shutdown_connections 2024-06-06T08:01:19.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:19.782+0000 7fb020df9700 1 -- 172.21.15.87:0/802458745 wait complete. 2024-06-06T08:01:19.783 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 45 2024-06-06T08:01:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: pgmap v420: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1002 KiB/s wr, 62 op/s 2024-06-06T08:01:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T08:01:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/1019415876,v1:172.21.15.195:6837/1019415876] up:boot 2024-06-06T08:01:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:20.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:19 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/802458745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: pgmap v420: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1002 KiB/s wr, 62 op/s 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/1019415876,v1:172.21.15.195:6837/1019415876] up:boot 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:19 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/802458745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: pgmap v420: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1002 KiB/s wr, 62 op/s 2024-06-06T08:01:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T08:01:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/1019415876,v1:172.21.15.195:6837/1019415876] up:boot 2024-06-06T08:01:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:01:20.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:01:20.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:01:20.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:19 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/802458745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:20.501 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0' 2024-06-06T08:01:20.501 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 142.3 --await-for 45.0 2024-06-06T08:01:20.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678448 5.33 MB/sec execute 564 sec latency 2734.467 ms 2024-06-06T08:01:20.761 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:01:21.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678448 5.32 MB/sec execute 565 sec latency 3734.494 ms 2024-06-06T08:01:21.947 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:21 smithi087 ceph-mon[32276]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 725 KiB/s wr, 45 op/s 2024-06-06T08:01:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:21 smithi195 ceph-mon[37040]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 725 KiB/s wr, 45 op/s 2024-06-06T08:01:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:21 smithi181 ceph-mon[36790]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 725 KiB/s wr, 45 op/s 2024-06-06T08:01:22.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 -- 172.21.15.87:0/156584630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8100e50 msgr2=0x7fbdd81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 --2- 172.21.15.87:0/156584630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8100e50 0x7fbdd81012d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009990 tx=0x7fbdc8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 -- 172.21.15.87:0/156584630 shutdown_connections 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 --2- 172.21.15.87:0/156584630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 0x7fbdd8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 --2- 172.21.15.87:0/156584630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8101810 0x7fbdd810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 --2- 172.21.15.87:0/156584630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8100e50 0x7fbdd81012d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 -- 172.21.15.87:0/156584630 >> 172.21.15.87:0/156584630 conn(0x7fbdd80766b0 msgr2=0x7fbdd8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 -- 172.21.15.87:0/156584630 shutdown_connections 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.525+0000 7fbde05a4700 1 -- 172.21.15.87:0/156584630 wait complete. 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 Processor -- start 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 -- start start 2024-06-06T08:01:22.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8100e50 0x7fbdd819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 0x7fbdd8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81127d0 con 0x7fbdd8101810 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd8112650 con 0x7fbdd8107570 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbde05a4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd8112950 con 0x7fbdd8100e50 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48332/0 (socket says 172.21.15.87:48332) 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 -- 172.21.15.87:0/51353759 learned_addr learned my addr 172.21.15.87:0/51353759 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbddeb41700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 0x7fbdd8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdde340700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8100e50 0x7fbdd819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 -- 172.21.15.87:0/51353759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8100e50 msgr2=0x7fbdd819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8100e50 0x7fbdd819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 -- 172.21.15.87:0/51353759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 msgr2=0x7fbdd8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 0x7fbdd8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 -- 172.21.15.87:0/51353759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd81990f0 con 0x7fbdd8101810 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.526+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.527+0000 7fbdcf7fe700 1 -- 172.21.15.87:0/51353759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8027070 con 0x7fbdd8101810 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.527+0000 7fbdcf7fe700 1 -- 172.21.15.87:0/51353759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8004050 con 0x7fbdd8101810 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.527+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd8199380 con 0x7fbdd8101810 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.527+0000 7fbdcf7fe700 1 -- 172.21.15.87:0/51353759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800f460 con 0x7fbdd8101810 2024-06-06T08:01:22.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.527+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd81a5650 con 0x7fbdd8101810 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.529+0000 7fbdcf7fe700 1 -- 172.21.15.87:0/51353759 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbdc802a070 con 0x7fbdd8101810 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.530+0000 7fbdcf7fe700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 0x7fbdc40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.530+0000 7fbdcf7fe700 1 -- 172.21.15.87:0/51353759 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbdc80d0710 con 0x7fbdd8101810 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.530+0000 7fbdde340700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 0x7fbdc40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.530+0000 7fbdde340700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 0x7fbdc40a5370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005950 tx=0x7fbdd40058e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.530+0000 7fbdeae27b80 1 Processor -- start 2024-06-06T08:01:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 -- start start 2024-06-06T08:01:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d510970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d50a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562c9d3fd0f0 con 0x562c9d50ace0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562c9d45b9e0 con 0x562c9d510590 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562c9d3c35f0 con 0x562c9d510eb0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48338/0 (socket says 172.21.15.87:48338) 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 -- 172.21.15.87:0/3452256875 learned_addr learned my addr 172.21.15.87:0/3452256875 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdde340700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d510970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d50a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 -- 172.21.15.87:0/3452256875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 msgr2=0x562c9d50a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d50a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 -- 172.21.15.87:0/3452256875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 msgr2=0x562c9d510970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d510970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbddeb41700 1 -- 172.21.15.87:0/3452256875 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9d3aaa60 con 0x562c9d50ace0 2024-06-06T08:01:22.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.531+0000 7fbdde340700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d510970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbddeb41700 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009dc0 tx=0x7fbdd00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc33c41ae45f1ae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbdccff9700 1 -- 172.21.15.87:0/3452256875 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0014610 con 0x562c9d50ace0 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbdccff9700 1 -- 172.21.15.87:0/3452256875 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd0019070 con 0x562c9d50ace0 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbdccff9700 1 -- 172.21.15.87:0/3452256875 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00139e0 con 0x562c9d50ace0 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbdeae27b80 1 -- 172.21.15.87:0/3452256875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 msgr2=0x562c9d50b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.532+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009dc0 tx=0x7fbdd00077e0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 -- 172.21.15.87:0/3452256875 shutdown_connections 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d510970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50b100 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/3452256875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d50a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 -- 172.21.15.87:0/3452256875 >> 172.21.15.87:0/3452256875 conn(0x562c9d437210 msgr2=0x562c9d505a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 -- 172.21.15.87:0/3452256875 shutdown_connections 2024-06-06T08:01:22.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 -- 172.21.15.87:0/3452256875 wait complete. 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 -- start start 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 Processor -- start 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.533+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d513eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d5143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562c9d3fc150 con 0x562c9d50ace0 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562c9d4579a0 con 0x562c9d510590 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562c9d517890 con 0x562c9d510eb0 2024-06-06T08:01:22.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 10 client.0 _ll_register_callbacks cb 0x562c9d3f1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48344/0 (socket says 172.21.15.87:48344) 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 -- 172.21.15.87:0/573656628 learned_addr learned my addr 172.21.15.87:0/573656628 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d513eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbddeb41700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d5143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 -- 172.21.15.87:0/573656628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 msgr2=0x562c9d5143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d5143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 -- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 msgr2=0x562c9d513eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d513eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 -- 172.21.15.87:0/573656628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9d4218e0 con 0x562c9d50ace0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdde340700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4011770 tx=0x7fbdd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878457d065336a52 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4023070 con 0x562c9d50ace0 2024-06-06T08:01:22.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbdd401f830 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd0019070 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdd402f050 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.534+0000 7fbdeae27b80 10 client.17454 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562c9d521fa0 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd0019070 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fbdd400f040 con 0x562c9d50ace0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 10 client.17454 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 20 client.17454 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 4 client.17454 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c9d468ee0 con 0x562c9d524580 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.535+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.536+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5269b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45304713b4561c0 server_cookie=5c5cc0842954d924 in_seq=0 out_seq=0 2024-06-06T08:01:22.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.536+0000 7fbdc1ffb700 10 client.17454 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.536+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/573656628 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c9d468ee0 con 0x562c9d524580 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.536+0000 7fbdc1ffb700 10 client.17454 handle_command_reply: tid=0 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.536+0000 7fbdeae27b80 1 client.17454 shutdown 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 msgr2=0x562c9d516bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4011770 tx=0x7fbdd4004750 comp rx=0 tx=0).stop 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 shutdown_connections 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5269b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510eb0 0x562c9d5143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d516bb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdc1ffb700 0 client.17454 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/573656628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510590 0x562c9d513eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 >> 172.21.15.87:0/573656628 conn(0x562c9d437210 msgr2=0x562c9d504c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 shutdown_connections 2024-06-06T08:01:22.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 1 -- 172.21.15.87:0/573656628 wait complete. 2024-06-06T08:01:22.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.537+0000 7fbdeae27b80 20 client.17454 trim_cache size 0 max 16384 2024-06-06T08:01:22.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.556+0000 7fbdeae27b80 1 Processor -- start 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 -- start start 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510590 0x562c9d512ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d511a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81b0dc0 con 0x562c9d50ace0 2024-06-06T08:01:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ae650 con 0x562c9d510eb0 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ac4a0 con 0x562c9d510590 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48358/0 (socket says 172.21.15.87:48358) 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 -- 172.21.15.87:0/1541991358 learned_addr learned my addr 172.21.15.87:0/1541991358 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510590 0x562c9d512ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 -- 172.21.15.87:0/1541991358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510590 msgr2=0x562c9d512ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510590 0x562c9d512ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 -- 172.21.15.87:0/1541991358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 msgr2=0x562c9d511a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d511a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 -- 172.21.15.87:0/1541991358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9d4fd530 con 0x562c9d50ace0 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdde340700 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4010b70 tx=0x7fbdd401e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed35aa8b07e2821 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdc3fff700 1 -- 172.21.15.87:0/1541991358 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4023070 con 0x562c9d50ace0 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdc3fff700 1 -- 172.21.15.87:0/1541991358 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd400ee50 con 0x562c9d50ace0 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.557+0000 7fbdc3fff700 1 -- 172.21.15.87:0/1541991358 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4005270 con 0x562c9d50ace0 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- 172.21.15.87:0/1541991358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 msgr2=0x562c9d50a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4010b70 tx=0x7fbdd401e420 comp rx=0 tx=0).stop 2024-06-06T08:01:22.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- 172.21.15.87:0/1541991358 shutdown_connections 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d511a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d50ace0 0x562c9d50a7a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/1541991358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d510590 0x562c9d512ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- 172.21.15.87:0/1541991358 >> 172.21.15.87:0/1541991358 conn(0x562c9d437210 msgr2=0x562c9d50f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- 172.21.15.87:0/1541991358 shutdown_connections 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- 172.21.15.87:0/1541991358 wait complete. 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 -- start start 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.558+0000 7fbdeae27b80 1 Processor -- start 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d50ace0 0x562c9d52ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d593290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81acb10 con 0x562c9d510590 2024-06-06T08:01:22.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81adf80 con 0x562c9d510eb0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ad1e0 con 0x562c9d50ace0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 10 client.0 _ll_register_callbacks cb 0x562c9d3f1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48374/0 (socket says 172.21.15.87:48374) 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 -- 172.21.15.87:0/130304249 learned_addr learned my addr 172.21.15.87:0/130304249 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdde340700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d50ace0 0x562c9d52ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbddeb41700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d593290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 -- 172.21.15.87:0/130304249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d50ace0 msgr2=0x562c9d52ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d50ace0 0x562c9d52ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 -- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 msgr2=0x562c9d593290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d593290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 -- 172.21.15.87:0/130304249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9d593900 con 0x562c9d510590 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdddb3f700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80cdaf0 tx=0x7fbdc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183f83ec82de8263 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:22.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80094b0 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbdc8098f60 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd400ee50 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8022500 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 10 client.17460 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.559+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562c9d594c40 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd400ee50 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fbdc809cb50 con 0x562c9d510590 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 10 client.17460 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 10 client.17460 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 20 client.17460 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 4 client.17460 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}] 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.3, "await_for": 45.0}) -- 0x562c9d463460 con 0x562c9d524580 2024-06-06T08:01:22.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.560+0000 7fbdde340700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:22.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.561+0000 7fbdde340700 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5972c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360d4ac5ed1ba9fa server_cookie=b1e929b6bdc17795 in_seq=0 out_seq=0 2024-06-06T08:01:22.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.561+0000 7fbdc1ffb700 10 client.17460 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:22.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678449 5.31 MB/sec execute 566 sec latency 4677.969 ms 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.721+0000 7fbdc1ffb700 1 -- 172.21.15.87:0/130304249 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562c9d463460 con 0x562c9d524580 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.721+0000 7fbdc1ffb700 10 client.17460 handle_command_reply: tid=0 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.721+0000 7fbdeae27b80 1 client.17460 shutdown 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 msgr2=0x562c9d52bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80cdaf0 tx=0x7fbdc8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 shutdown_connections 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562c9d524580 0x562c9d5972c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c9d510eb0 0x562c9d593290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562c9d50ace0 0x562c9d52ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdc1ffb700 0 client.17460 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:22.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 --2- 172.21.15.87:0/130304249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562c9d510590 0x562c9d52bf90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 >> 172.21.15.87:0/130304249 conn(0x562c9d437210 msgr2=0x562c9d50ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 shutdown_connections 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 1 -- 172.21.15.87:0/130304249 wait complete. 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbdeae27b80 20 client.17460 trim_cache size 0 max 16384 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 45, 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 38, 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:01:22.723 INFO:teuthology.orchestra.run.smithi087.stdout: "da05dcbd": { 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 38, 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 142.3, 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:01:22.724 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:22.725 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 msgr2=0x7fbdc40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 0x7fbdc40a5370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005950 tx=0x7fbdd40058e0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 msgr2=0x7fbdd819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc80047b0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 shutdown_connections 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd8107570 0x7fbdd8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbdc40a2eb0 0x7fbdc40a5370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdd8100e50 0x7fbdd819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 --2- 172.21.15.87:0/51353759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd8101810 0x7fbdd819ecc0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:22.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.722+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 >> 172.21.15.87:0/51353759 conn(0x7fbdd80766b0 msgr2=0x7fbdd80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:22.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.723+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 shutdown_connections 2024-06-06T08:01:22.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:22.723+0000 7fbde05a4700 1 -- 172.21.15.87:0/51353759 wait complete. 2024-06-06T08:01:23.536 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'da05dcbd', quiesce took 6.3 seconds. Will release after: 16.0 2024-06-06T08:01:23.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.30 MB/sec execute 567 sec latency 961.802 ms 2024-06-06T08:01:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:23 smithi087 ceph-mon[32276]: pgmap v423: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T08:01:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:23 smithi195 ceph-mon[37040]: pgmap v423: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T08:01:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:23 smithi181 ceph-mon[36790]: pgmap v423: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-06T08:01:24.694 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.29 MB/sec execute 568 sec latency 1961.910 ms 2024-06-06T08:01:25.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.28 MB/sec execute 569 sec latency 2962.008 ms 2024-06-06T08:01:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:25 smithi087 ceph-mon[32276]: pgmap v424: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-06T08:01:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:25 smithi195 ceph-mon[37040]: pgmap v424: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-06T08:01:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:25 smithi181 ceph-mon[36790]: pgmap v424: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-06T08:01:26.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.27 MB/sec execute 570 sec latency 3962.128 ms 2024-06-06T08:01:27.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.26 MB/sec execute 571 sec latency 4962.220 ms 2024-06-06T08:01:28.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:27 smithi087 ceph-mon[32276]: pgmap v425: 129 pgs: 129 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T08:01:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:27 smithi195 ceph-mon[37040]: pgmap v425: 129 pgs: 129 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T08:01:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:27 smithi181 ceph-mon[36790]: pgmap v425: 129 pgs: 129 active+clean; 80 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T08:01:28.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.25 MB/sec execute 572 sec latency 5962.316 ms 2024-06-06T08:01:29.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.24 MB/sec execute 573 sec latency 6962.415 ms 2024-06-06T08:01:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:29 smithi087 ceph-mon[32276]: pgmap v426: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 812 KiB/s wr, 83 op/s 2024-06-06T08:01:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:29 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:01:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:29 smithi195 ceph-mon[37040]: pgmap v426: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 812 KiB/s wr, 83 op/s 2024-06-06T08:01:30.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:29 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:01:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:29 smithi181 ceph-mon[36790]: pgmap v426: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 812 KiB/s wr, 83 op/s 2024-06-06T08:01:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:29 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:01:30.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.23 MB/sec execute 574 sec latency 7962.491 ms 2024-06-06T08:01:31.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:30 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:01:31.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:30 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:01:31.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:30 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 -- 172.21.15.87:0/1298151481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b41012c0 msgr2=0x7fd3b410d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/1298151481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b41012c0 0x7fd3b410d780 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009cd0 tx=0x7fd3ac02a8e0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 -- 172.21.15.87:0/1298151481 shutdown_connections 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/1298151481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b41012c0 0x7fd3b410d780 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/1298151481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b4100900 0x7fd3b4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/1298151481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 0x7fd3b4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 -- 172.21.15.87:0/1298151481 >> 172.21.15.87:0/1298151481 conn(0x7fd3b4077ab0 msgr2=0x7fd3b4077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 -- 172.21.15.87:0/1298151481 shutdown_connections 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.244+0000 7fd3bb37e640 1 -- 172.21.15.87:0/1298151481 wait complete. 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 Processor -- start 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 -- start start 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b41012c0 0x7fd3b41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 0x7fd3b419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4110cd0 con 0x7fd3b4100900 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4110b50 con 0x7fd3b41068c0 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3bb37e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4110e50 con 0x7fd3b41012c0 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46786/0 (socket says 172.21.15.87:46786) 2024-06-06T08:01:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 -- 172.21.15.87:0/3800460451 learned_addr learned my addr 172.21.15.87:0/3800460451 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b8919640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b41012c0 0x7fd3b41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 -- 172.21.15.87:0/3800460451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b41012c0 msgr2=0x7fd3b41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b41012c0 0x7fd3b41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b991b640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 0x7fd3b419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.245+0000 7fd3b911a640 1 -- 172.21.15.87:0/3800460451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 msgr2=0x7fd3b419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3b911a640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 0x7fd3b419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3b911a640 1 -- 172.21.15.87:0/3800460451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b419d160 con 0x7fd3b4100900 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3b911a640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c890 tx=0x7fd3a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3a67fc640 1 -- 172.21.15.87:0/3800460451 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8004220 con 0x7fd3b4100900 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b419d450 con 0x7fd3b4100900 2024-06-06T08:01:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3a67fc640 1 -- 172.21.15.87:0/3800460451 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a80043c0 con 0x7fd3b4100900 2024-06-06T08:01:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3a67fc640 1 -- 172.21.15.87:0/3800460451 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8004a00 con 0x7fd3b4100900 2024-06-06T08:01:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.246+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a4ce0 con 0x7fd3b4100900 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.248+0000 7fd3a67fc640 1 -- 172.21.15.87:0/3800460451 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd3a80079b0 con 0x7fd3b4100900 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 Processor -- start 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 -- start start 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3a67fc640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace1085e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ace0fdf4f0 con 0x55ace1086350 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ace0f7d870 con 0x55ace107b170 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3be16e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ace0fdb4b0 con 0x55ace107ba90 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b911a640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46788/0 (socket says 172.21.15.87:46788) 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 -- 172.21.15.87:0/1504317645 learned_addr learned my addr 172.21.15.87:0/1504317645 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b8919640 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace1085e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3a67fc640 1 -- 172.21.15.87:0/3800460451 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fd3a80cf4c0 con 0x7fd3b4100900 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 -- 172.21.15.87:0/1504317645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 msgr2=0x55ace1085e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace1085e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 -- 172.21.15.87:0/1504317645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 msgr2=0x55ace107b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 -- 172.21.15.87:0/1504317645 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace0f999b0 con 0x55ace1086350 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b911a640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 0x7fd3a00a5260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80cc990 tx=0x7fd3a806d670 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.249+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac004af0 tx=0x7fd3ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ed5299cfe20e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd39f7fe640 1 -- 172.21.15.87:0/1504317645 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac044070 con 0x55ace1086350 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd39f7fe640 1 -- 172.21.15.87:0/1504317645 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac036040 con 0x55ace1086350 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd39f7fe640 1 -- 172.21.15.87:0/1504317645 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac004410 con 0x55ace1086350 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 -- 172.21.15.87:0/1504317645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 msgr2=0x55ace1088740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac004af0 tx=0x7fd3ac004a40 comp rx=0 tx=0).stop 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 -- 172.21.15.87:0/1504317645 shutdown_connections 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1088740 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace1085e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1504317645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 -- 172.21.15.87:0/1504317645 >> 172.21.15.87:0/1504317645 conn(0x55ace1077680 msgr2=0x55ace1077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 -- 172.21.15.87:0/1504317645 shutdown_connections 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.250+0000 7fd3be16e740 1 -- 172.21.15.87:0/1504317645 wait complete. 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 -- start start 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 Processor -- start 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107b170 0x55ace1083290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 0x55ace10837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ace1088f20 con 0x55ace1086350 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ace0f68ae0 con 0x55ace107ba90 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ace10890a0 con 0x55ace107b170 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3be16e740 10 client.0 _ll_register_callbacks cb 0x55ace0fa19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46804/0 (socket says 172.21.15.87:46804) 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3b991b640 1 -- 172.21.15.87:0/2792301451 learned_addr learned my addr 172.21.15.87:0/2792301451 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3b8919640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 0x55ace10837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.251+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107b170 0x55ace1083290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 -- 172.21.15.87:0/2792301451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107b170 msgr2=0x55ace1083290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107b170 0x55ace1083290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 -- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 msgr2=0x55ace10837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 0x55ace10837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 -- 172.21.15.87:0/2792301451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace0cf3210 con 0x55ace1086350 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b8919640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 0x55ace10837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac02ae00 tx=0x7fd3ac02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59b4ac376286e7c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac044040 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fd3ac003ba0 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ac036040 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac060050 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3be16e740 10 client.17472 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ace0dbf620 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac036040 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.252+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fd3ac030070 con 0x55ace1086350 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 10 client.17472 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 20 client.17472 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1096320 0x55ace1098710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 4 client.17472 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ace0ff2f10 con 0x55ace1096320 2024-06-06T08:01:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3b8919640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1096320 0x55ace1098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd39cff9640 10 client.17472 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.253+0000 7fd3b8919640 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1096320 0x55ace1098710 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e73bec1083e219 server_cookie=4ccfd6718ed166b2 in_seq=0 out_seq=0 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd39cff9640 1 -- 172.21.15.87:0/2792301451 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ace0ff2f10 con 0x55ace1096320 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd39cff9640 10 client.17472 handle_command_reply: tid=0 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 client.17472 shutdown 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 msgr2=0x55ace1083d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac02ae00 tx=0x7fd3ac02a8e0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 shutdown_connections 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1096320 0x55ace1098710 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace1083d10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd39cff9640 0 client.17472 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107ba90 0x55ace10837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2792301451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107b170 0x55ace1083290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 >> 172.21.15.87:0/2792301451 conn(0x55ace1077680 msgr2=0x55ace107d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 shutdown_connections 2024-06-06T08:01:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 1 -- 172.21.15.87:0/2792301451 wait complete. 2024-06-06T08:01:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.254+0000 7fd3be16e740 20 client.17472 trim_cache size 0 max 16384 2024-06-06T08:01:31.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:30 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:01:31.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:30 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:01:31.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:30 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:01:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 Processor -- start 2024-06-06T08:01:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 -- start start 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace109dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41b1750 con 0x55ace1086350 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41af020 con 0x55ace107b170 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.269+0000 7fd3be16e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41ace70 con 0x55ace107ba90 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46816/0 (socket says 172.21.15.87:46816) 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 -- 172.21.15.87:0/2318751995 learned_addr learned my addr 172.21.15.87:0/2318751995 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b8919640 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace109dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 -- 172.21.15.87:0/2318751995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 msgr2=0x55ace109dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace109dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 -- 172.21.15.87:0/2318751995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 msgr2=0x55ace107db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 -- 172.21.15.87:0/2318751995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace1088fc0 con 0x55ace1086350 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd3b991b640 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009bd0 tx=0x7fd3ac004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aedb0996a13f625 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd39effd640 1 -- 172.21.15.87:0/2318751995 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac044040 con 0x55ace1086350 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd39effd640 1 -- 172.21.15.87:0/2318751995 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac036070 con 0x55ace1086350 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.270+0000 7fd39effd640 1 -- 172.21.15.87:0/2318751995 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac031430 con 0x55ace1086350 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- 172.21.15.87:0/2318751995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 msgr2=0x55ace109e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009bd0 tx=0x7fd3ac004aa0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- 172.21.15.87:0/2318751995 shutdown_connections 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace1086350 0x55ace109e1f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace107ba90 0x55ace109dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- 172.21.15.87:0/2318751995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace107db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- 172.21.15.87:0/2318751995 >> 172.21.15.87:0/2318751995 conn(0x55ace1077680 msgr2=0x55ace10767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- 172.21.15.87:0/2318751995 shutdown_connections 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- 172.21.15.87:0/2318751995 wait complete. 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 -- start start 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 Processor -- start 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace108e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.271+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace1086350 0x55ace10802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41ad4e0 con 0x55ace107ba90 2024-06-06T08:01:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41ae950 con 0x55ace107b170 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41adbb0 con 0x55ace1086350 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 10 client.0 _ll_register_callbacks cb 0x7fd3b00092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46818/0 (socket says 172.21.15.87:46818) 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 -- 172.21.15.87:0/1634536637 learned_addr learned my addr 172.21.15.87:0/1634536637 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace1086350 0x55ace10802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace108e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 -- 172.21.15.87:0/1634536637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace1086350 msgr2=0x55ace10802b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace1086350 0x55ace10802b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 -- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 msgr2=0x55ace108e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace108e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 -- 172.21.15.87:0/1634536637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace10807f0 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3b8919640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005fd0 tx=0x7fd3b0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605bd308d7a42d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001b6d0 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fd3b001b870 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ac036070 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd3b002c050 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 10 client.17478 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ace1100a40 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.272+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac036070 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fd3b000f040 con 0x55ace107ba90 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 10 client.17478 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 20 client.17478 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1101240 0x55ace1103660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 4 client.17478 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ace0fecc20 con 0x55ace1101240 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1101240 0x55ace1103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd39cff9640 10 client.17478 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.273+0000 7fd3b991b640 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1101240 0x55ace1103660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8c09fb2798b257 server_cookie=d6dcfb95b0a1e83f in_seq=0 out_seq=0 2024-06-06T08:01:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd39cff9640 1 -- 172.21.15.87:0/1634536637 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ace0fecc20 con 0x55ace1101240 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd39cff9640 10 client.17478 handle_command_reply: tid=0 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 client.17478 shutdown 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 msgr2=0x55ace107fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005fd0 tx=0x7fd3b0012820 comp rx=0 tx=0).stop 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 shutdown_connections 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ace1101240 0x55ace1103660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ace1086350 0x55ace10802b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ace107ba90 0x55ace107fd70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd39cff9640 0 client.17478 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 --2- 172.21.15.87:0/1634536637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ace107b170 0x55ace108e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 >> 172.21.15.87:0/1634536637 conn(0x55ace1077680 msgr2=0x55ace1083250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 shutdown_connections 2024-06-06T08:01:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 1 -- 172.21.15.87:0/1634536637 wait complete. 2024-06-06T08:01:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.302+0000 7fd3be16e740 20 client.17478 trim_cache size 0 max 16384 2024-06-06T08:01:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:01:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:01:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 msgr2=0x7fd3a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 0x7fd3a00a5260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80cc990 tx=0x7fd3a806d670 comp rx=0 tx=0).stop 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 msgr2=0x7fd3b4198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c890 tx=0x7fd3a800cd50 comp rx=0 tx=0).stop 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 shutdown_connections 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3b41068c0 0x7fd3b419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3b41012c0 0x7fd3b41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 --2- 172.21.15.87:0/3800460451 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3b4100900 0x7fd3b4198170 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 >> 172.21.15.87:0/3800460451 conn(0x7fd3b4077ab0 msgr2=0x7fd3b406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 shutdown_connections 2024-06-06T08:01:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:31.303+0000 7fd3bb37e640 1 -- 172.21.15.87:0/3800460451 wait complete. 2024-06-06T08:01:31.310 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:30 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:01:31.310 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:30 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:01:31.310 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:30 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:01:31.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.23 MB/sec execute 575 sec latency 8962.581 ms 2024-06-06T08:01:32.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.22 MB/sec execute 576 sec latency 9962.699 ms 2024-06-06T08:01:32.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:32 smithi195 ceph-mon[37040]: pgmap v427: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 688 KiB/s wr, 71 op/s 2024-06-06T08:01:32.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:32 smithi181 ceph-mon[36790]: pgmap v427: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 688 KiB/s wr, 71 op/s 2024-06-06T08:01:32.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:32 smithi087 ceph-mon[32276]: pgmap v427: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 688 KiB/s wr, 71 op/s 2024-06-06T08:01:33.695 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.21 MB/sec execute 577 sec latency 10962.786 ms 2024-06-06T08:01:34.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.20 MB/sec execute 578 sec latency 11962.893 ms 2024-06-06T08:01:34.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:34 smithi195 ceph-mon[37040]: pgmap v428: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 726 KiB/s wr, 90 op/s 2024-06-06T08:01:34.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:34 smithi181 ceph-mon[36790]: pgmap v428: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 726 KiB/s wr, 90 op/s 2024-06-06T08:01:34.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:34 smithi087 ceph-mon[32276]: pgmap v428: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 726 KiB/s wr, 90 op/s 2024-06-06T08:01:35.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.19 MB/sec execute 579 sec latency 12963.015 ms 2024-06-06T08:01:36.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.18 MB/sec execute 580 sec latency 13963.078 ms 2024-06-06T08:01:36.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:36 smithi195 ceph-mon[37040]: pgmap v429: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 324 KiB/s wr, 32 op/s 2024-06-06T08:01:36.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:36 smithi181 ceph-mon[36790]: pgmap v429: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 324 KiB/s wr, 32 op/s 2024-06-06T08:01:36.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:36 smithi087 ceph-mon[32276]: pgmap v429: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 324 KiB/s wr, 32 op/s 2024-06-06T08:01:37.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.17 MB/sec execute 581 sec latency 14963.176 ms 2024-06-06T08:01:38.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.16 MB/sec execute 582 sec latency 15963.245 ms 2024-06-06T08:01:38.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:38 smithi195 ceph-mon[37040]: pgmap v430: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 324 KiB/s wr, 33 op/s 2024-06-06T08:01:38.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:38 smithi181 ceph-mon[36790]: pgmap v430: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 324 KiB/s wr, 33 op/s 2024-06-06T08:01:38.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:38 smithi087 ceph-mon[32276]: pgmap v430: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 324 KiB/s wr, 33 op/s 2024-06-06T08:01:39.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:01:39.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.15 MB/sec execute 583 sec latency 16963.309 ms 2024-06-06T08:01:39.789 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:01:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:40 smithi087 ceph-mon[32276]: pgmap v431: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 250 KiB/s wr, 27 op/s 2024-06-06T08:01:40.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.14 MB/sec execute 584 sec latency 17963.343 ms 2024-06-06T08:01:40.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:40 smithi195 ceph-mon[37040]: pgmap v431: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 250 KiB/s wr, 27 op/s 2024-06-06T08:01:40.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:40 smithi181 ceph-mon[36790]: pgmap v431: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 250 KiB/s wr, 27 op/s 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 -- 172.21.15.87:0/1505079359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 msgr2=0x7f619810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 --2- 172.21.15.87:0/1505079359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f619810f3e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f619000b0d0 tx=0x7f619000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 -- 172.21.15.87:0/1505079359 shutdown_connections 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 --2- 172.21.15.87:0/1505079359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f619810f3e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 --2- 172.21.15.87:0/1505079359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198102770 0x7f6198102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 --2- 172.21.15.87:0/1505079359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198103120 0x7f6198107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 -- 172.21.15.87:0/1505079359 >> 172.21.15.87:0/1505079359 conn(0x7f6198079600 msgr2=0x7f6198079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 -- 172.21.15.87:0/1505079359 shutdown_connections 2024-06-06T08:01:41.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.559+0000 7f619e270700 1 -- 172.21.15.87:0/1505079359 wait complete. 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 Processor -- start 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 -- start start 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198102770 0x7f619819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 0x7f619819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:41.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6198112870 con 0x7f6198107860 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61981126f0 con 0x7f6198102770 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619e270700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61981129f0 con 0x7f6198103120 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54604/0 (socket says 172.21.15.87:54604) 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 -- 172.21.15.87:0/3644391051 learned_addr learned my addr 172.21.15.87:0/3644391051 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:41.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f6197fff700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198102770 0x7f619819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f61977fe700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 0x7f619819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 -- 172.21.15.87:0/3644391051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 msgr2=0x7f619819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 0x7f619819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 -- 172.21.15.87:0/3644391051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198102770 msgr2=0x7f619819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198102770 0x7f619819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f619c80d700 1 -- 172.21.15.87:0/3644391051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6198199230 con 0x7f6198107860 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.560+0000 7f61977fe700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 0x7f619819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f619c80d700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6190006070 tx=0x7f6190004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190021070 con 0x7f6198107860 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6190007df0 con 0x7f6198107860 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61981994c0 con 0x7f6198107860 2024-06-06T08:01:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190005470 con 0x7f6198107860 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.561+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61981a57e0 con 0x7f6198107860 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.562+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6198069180 con 0x7f6198107860 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.564+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6190024070 con 0x7f6198107860 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.564+0000 7f61957fa700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 0x7f61800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.564+0000 7f6197fff700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 0x7f61800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:41.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.564+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f61900d0b10 con 0x7f6198107860 2024-06-06T08:01:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.565+0000 7f6197fff700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 0x7f61800a5250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.568+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f619009d3f0 con 0x7f6198107860 2024-06-06T08:01:41.698 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.14 MB/sec execute 585 sec latency 18963.409 ms 2024-06-06T08:01:41.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.808+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f619802dca0 con 0x7f6198107860 2024-06-06T08:01:41.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.809+0000 7f61957fa700 1 -- 172.21.15.87:0/3644391051 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11878 (secure 0 0 0) 0x7f619009ccf0 con 0x7f6198107860 2024-06-06T08:01:41.809 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:01:41.809 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:01:18.835086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26623":{"gid":26623,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1019415876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1019415876},{"type":"v1","addr":"172.21.15.195:6837","nonce":1019415876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 msgr2=0x7f61800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 0x7f61800a5250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188008040 comp rx=0 tx=0).stop 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 msgr2=0x7f6198198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6190006070 tx=0x7f6190004ef0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 shutdown_connections 2024-06-06T08:01:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6198102770 0x7f619819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f61800a2d90 0x7f61800a5250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6198107860 0x7f6198198ad0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 --2- 172.21.15.87:0/3644391051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6198103120 0x7f619819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 >> 172.21.15.87:0/3644391051 conn(0x7f6198079600 msgr2=0x7f6198100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 shutdown_connections 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:41.811+0000 7f619e270700 1 -- 172.21.15.87:0/3644391051 wait complete. 2024-06-06T08:01:41.812 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 45 2024-06-06T08:01:42.497 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id da05dcbd --release --await' 2024-06-06T08:01:42.498 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id da05dcbd --release --await 2024-06-06T08:01:42.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.13 MB/sec execute 586 sec latency 19963.436 ms 2024-06-06T08:01:42.735 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:01:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:42 smithi195 ceph-mon[37040]: pgmap v432: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-06T08:01:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:42 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3644391051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:42.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:42 smithi181 ceph-mon[36790]: pgmap v432: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-06T08:01:42.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:42 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3644391051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:42.855 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:42 smithi087 ceph-mon[32276]: pgmap v432: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-06T08:01:42.855 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:42 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3644391051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:01:43.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678450 5.12 MB/sec execute 587 sec latency 20963.461 ms 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 -- 172.21.15.87:0/3549901465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 msgr2=0x7f4460106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 --2- 172.21.15.87:0/3549901465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f4460106c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4454009900 tx=0x7f445402a410 comp rx=0 tx=0).stop 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 -- 172.21.15.87:0/3549901465 shutdown_connections 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 --2- 172.21.15.87:0/3549901465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 0x7f446010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 --2- 172.21.15.87:0/3549901465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f4460100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 --2- 172.21.15.87:0/3549901465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f4460106c70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 -- 172.21.15.87:0/3549901465 >> 172.21.15.87:0/3549901465 conn(0x7f4460077a20 msgr2=0x7f4460077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 -- 172.21.15.87:0/3549901465 shutdown_connections 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.399+0000 7f4466a95640 1 -- 172.21.15.87:0/3549901465 wait complete. 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 Processor -- start 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 -- start start 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 0x7f44601986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f446019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4460110cf0 con 0x7f44601008d0 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4460110b70 con 0x7f4460106890 2024-06-06T08:01:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4466a95640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4460110e70 con 0x7f4460101290 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54618/0 (socket says 172.21.15.87:54618) 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 -- 172.21.15.87:0/711513796 learned_addr learned my addr 172.21.15.87:0/711513796 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f445ffff640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 0x7f44601986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4465032640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f446019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 -- 172.21.15.87:0/711513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 msgr2=0x7f44601986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 0x7f44601986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 -- 172.21.15.87:0/711513796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 msgr2=0x7f446019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f446019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.400+0000 7f4464831640 1 -- 172.21.15.87:0/711513796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446019d190 con 0x7f44601008d0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f4464831640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f445402aa00 tx=0x7f4454003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f445dffb640 1 -- 172.21.15.87:0/711513796 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445403f070 con 0x7f44601008d0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f445dffb640 1 -- 172.21.15.87:0/711513796 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4454031c30 con 0x7f44601008d0 2024-06-06T08:01:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f445dffb640 1 -- 172.21.15.87:0/711513796 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454035030 con 0x7f44601008d0 2024-06-06T08:01:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446019d420 con 0x7f44601008d0 2024-06-06T08:01:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.401+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44601a4d10 con 0x7f44601008d0 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.403+0000 7f445dffb640 1 -- 172.21.15.87:0/711513796 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4454042070 con 0x7f44601008d0 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f445dffb640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 0x7f44500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f445dffb640 1 -- 172.21.15.87:0/711513796 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f44540efdd0 con 0x7f44601008d0 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f445ffff640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 0x7f44500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f445ffff640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 0x7f44500a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f446006a0a0 tx=0x7f444c005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f446979b740 1 Processor -- start 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.404+0000 7f446979b740 1 -- start start 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0aa90 0x558376e14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558376d6e4f0 con 0x558376e15350 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558376d0c870 con 0x558376e0aa90 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f446979b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558376d6a4b0 con 0x558376e0a170 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54624/0 (socket says 172.21.15.87:54624) 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 -- 172.21.15.87:0/4153076433 learned_addr learned my addr 172.21.15.87:0/4153076433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4464831640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 -- 172.21.15.87:0/4153076433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 msgr2=0x558376e0a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f445ffff640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0aa90 0x558376e14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e0a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 -- 172.21.15.87:0/4153076433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0aa90 msgr2=0x558376e14e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0aa90 0x558376e14e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 -- 172.21.15.87:0/4153076433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558376d289b0 con 0x558376e15350 2024-06-06T08:01:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4464831640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f4465032640 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4458009f50 tx=0x7f44580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c232e7202f117a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.405+0000 7f444affd640 1 -- 172.21.15.87:0/4153076433 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004d20 con 0x558376e15350 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f444affd640 1 -- 172.21.15.87:0/4153076433 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4458004050 con 0x558376e15350 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f444affd640 1 -- 172.21.15.87:0/4153076433 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458005220 con 0x558376e15350 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 -- 172.21.15.87:0/4153076433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 msgr2=0x558376e17740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4458009f50 tx=0x7f44580118d0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 -- 172.21.15.87:0/4153076433 shutdown_connections 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e17740 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0aa90 0x558376e14e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 --2- 172.21.15.87:0/4153076433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 -- 172.21.15.87:0/4153076433 >> 172.21.15.87:0/4153076433 conn(0x558376e06680 msgr2=0x558376e06a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 -- 172.21.15.87:0/4153076433 shutdown_connections 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.406+0000 7f446979b740 1 -- 172.21.15.87:0/4153076433 wait complete. 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 -- start start 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 Processor -- start 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558376e17f80 con 0x558376e0aa90 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558376cf7ae0 con 0x558376e15350 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558376e18100 con 0x558376e0a170 2024-06-06T08:01:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f446979b740 10 client.0 _ll_register_callbacks cb 0x558376d309b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f445ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f4464831640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f445ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54628/0 (socket says 172.21.15.87:54628) 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f4465032640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.407+0000 7f445ffff640 1 -- 172.21.15.87:0/456816481 learned_addr learned my addr 172.21.15.87:0/456816481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 -- 172.21.15.87:0/456816481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 msgr2=0x558376e122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e122a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 -- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 msgr2=0x558376e12d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e12d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 -- 172.21.15.87:0/456816481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558376a82210 con 0x558376e0aa90 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f445ffff640 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f444c005bc0 tx=0x7f444c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e96f8e31901c3e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f443ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c01d250 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f444c01c400 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4458004050 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f444c01c600 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f446979b740 10 client.17496 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558376b4e620 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.408+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4458004050 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f444c024070 con 0x558376e0aa90 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 10 client.17496 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 20 client.17496 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e298a0 0x558376e2bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 4 client.17496 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558376d81f10 con 0x558376e298a0 2024-06-06T08:01:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f4464831640 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e298a0 0x558376e2bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f443ffff640 10 client.17496 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.409+0000 7f4464831640 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e298a0 0x558376e2bd10 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f0af0229244e44 server_cookie=1b376c02ba4b1f70 in_seq=0 out_seq=0 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f443ffff640 1 -- 172.21.15.87:0/456816481 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558376d81f10 con 0x558376e298a0 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f443ffff640 10 client.17496 handle_command_reply: tid=0 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 client.17496 shutdown 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 msgr2=0x558376e127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f444c005bc0 tx=0x7f444c004730 comp rx=0 tx=0).stop 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 shutdown_connections 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e298a0 0x558376e2bd10 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e12d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e127e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f443ffff640 0 client.17496 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 --2- 172.21.15.87:0/456816481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.410+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 >> 172.21.15.87:0/456816481 conn(0x558376e06680 msgr2=0x558376e0c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.411+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 shutdown_connections 2024-06-06T08:01:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.411+0000 7f446979b740 1 -- 172.21.15.87:0/456816481 wait complete. 2024-06-06T08:01:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.411+0000 7f446979b740 20 client.17496 trim_cache size 0 max 16384 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 Processor -- start 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 -- start start 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e10b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44601af570 con 0x558376e0aa90 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44601ace40 con 0x558376e15350 2024-06-06T08:01:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f446979b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44601aac90 con 0x558376e0a170 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54636/0 (socket says 172.21.15.87:54636) 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 -- 172.21.15.87:0/3814734864 learned_addr learned my addr 172.21.15.87:0/3814734864 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f4464831640 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e10b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 -- 172.21.15.87:0/3814734864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 msgr2=0x558376e10b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e10b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 -- 172.21.15.87:0/3814734864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 msgr2=0x558376e31620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.426+0000 7f445ffff640 1 -- 172.21.15.87:0/3814734864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558376e17a80 con 0x558376e0aa90 2024-06-06T08:01:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f445ffff640 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f444c026d80 tx=0x7f444c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c2155f5b0ac68c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f444a7fc640 1 -- 172.21.15.87:0/3814734864 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c01c400 con 0x558376e0aa90 2024-06-06T08:01:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f444a7fc640 1 -- 172.21.15.87:0/3814734864 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f444c039070 con 0x558376e0aa90 2024-06-06T08:01:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f444a7fc640 1 -- 172.21.15.87:0/3814734864 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c025ba0 con 0x558376e0aa90 2024-06-06T08:01:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f446979b740 1 -- 172.21.15.87:0/3814734864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 msgr2=0x558376e110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.427+0000 7f446979b740 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f444c026d80 tx=0x7f444c005c70 comp rx=0 tx=0).stop 2024-06-06T08:01:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- 172.21.15.87:0/3814734864 shutdown_connections 2024-06-06T08:01:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e15350 0x558376e31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e0aa90 0x558376e110b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- 172.21.15.87:0/3814734864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0a170 0x558376e10b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- 172.21.15.87:0/3814734864 >> 172.21.15.87:0/3814734864 conn(0x558376e06680 msgr2=0x558376e06a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- 172.21.15.87:0/3814734864 shutdown_connections 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- 172.21.15.87:0/3814734864 wait complete. 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- start start 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 Processor -- start 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0a170 0x558376e21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0aa90 0x558376e21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44601ab300 con 0x558376e15350 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44601ac770 con 0x558376e0a170 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44601ab9d0 con 0x558376e0aa90 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f446979b740 10 client.0 _ll_register_callbacks cb 0x7f444c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.428+0000 7f4465032640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54642/0 (socket says 172.21.15.87:54642) 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 -- 172.21.15.87:0/1432943746 learned_addr learned my addr 172.21.15.87:0/1432943746 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4464831640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0a170 0x558376e21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f445ffff640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0aa90 0x558376e21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 -- 172.21.15.87:0/1432943746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0aa90 msgr2=0x558376e21ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0aa90 0x558376e21ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 -- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0a170 msgr2=0x558376e21ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0a170 0x558376e21ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 -- 172.21.15.87:0/1432943746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558376e1d6b0 con 0x558376e15350 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f4465032640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4458006010 tx=0x7f4458004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2690be78374f218b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f443ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004970 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f444c039070 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f4458004b10 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4458034050 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f446979b740 10 client.17502 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558376e94180 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.429+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f444c039070 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f445800f480 con 0x558376e15350 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 10 client.17502 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 20 client.17502 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e94700 0x558376e96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 4 client.17502 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558376d7bc20 con 0x558376e94700 2024-06-06T08:01:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f4464831640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e94700 0x558376e96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f443ffff640 10 client.17502 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.430+0000 7f4464831640 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e94700 0x558376e96af0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fa72e9b9366098 server_cookie=1986dde464886d63 in_seq=0 out_seq=0 2024-06-06T08:01:44.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1621071516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 msgr2=0x7fa4ec101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1621071516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec101800 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009990 tx=0x7fa4dc009ca0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1621071516 shutdown_connections 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1621071516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec1073f0 0x7fa4ec1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1621071516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec101d40 0x7fa4ec10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1621071516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec101800 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1621071516 >> 172.21.15.87:0/1621071516 conn(0x7fa4ec078300 msgr2=0x7fa4ec078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1621071516 shutdown_connections 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.435+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1621071516 wait complete. 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 Processor -- start 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 -- start start 2024-06-06T08:01:44.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec101d40 0x7fa4ec19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec1073f0 0x7fa4ec19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111690 con 0x7fa4ec101380 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111510 con 0x7fa4ec101d40 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f39fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111810 con 0x7fa4ec1073f0 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54662/0 (socket says 172.21.15.87:54662) 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.436+0000 7fa4f1797700 1 -- 172.21.15.87:0/1271992661 learned_addr learned my addr 172.21.15.87:0/1271992661 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec101d40 0x7fa4ec19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1f98700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec1073f0 0x7fa4ec19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 -- 172.21.15.87:0/1271992661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec1073f0 msgr2=0x7fa4ec19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec1073f0 0x7fa4ec19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 -- 172.21.15.87:0/1271992661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec101d40 msgr2=0x7fa4ec19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec101d40 0x7fa4ec19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 -- 172.21.15.87:0/1271992661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec19feb0 con 0x7fa4ec101380 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f1797700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800e9a0 tx=0x7fa4e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4e27fc700 1 -- 172.21.15.87:0/1271992661 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e800ca10 con 0x7fa4ec101380 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec1a01a0 con 0x7fa4ec101380 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4e27fc700 1 -- 172.21.15.87:0/1271992661 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e800cbb0 con 0x7fa4ec101380 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4e27fc700 1 -- 172.21.15.87:0/1271992661 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8018680 con 0x7fa4ec101380 2024-06-06T08:01:44.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.437+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a7a20 con 0x7fa4ec101380 2024-06-06T08:01:44.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.440+0000 7fa4e27fc700 1 -- 172.21.15.87:0/1271992661 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa4e8010430 con 0x7fa4ec101380 2024-06-06T08:01:44.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.440+0000 7fa4e27fc700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 0x7fa4d80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.441+0000 7fa4e27fc700 1 -- 172.21.15.87:0/1271992661 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fa4e80cf9d0 con 0x7fa4ec101380 2024-06-06T08:01:44.441 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:44 smithi087 ceph-mon[32276]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 350 KiB/s wr, 46 op/s 2024-06-06T08:01:44.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.442+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 0x7fa4d80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.442+0000 7fa4fe27eb80 1 Processor -- start 2024-06-06T08:01:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 -- start start 2024-06-06T08:01:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 0x7fa4d80a5250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00b2e0 tx=0x7fa4dc005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54674/0 (socket says 172.21.15.87:54674) 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4f1797700 1 -- 172.21.15.87:0/399319987 learned_addr learned my addr 172.21.15.87:0/399319987 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564881405910 con 0x5648814c2450 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648813ad820 con 0x5648814c2d70 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.443+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5648814018d0 con 0x5648814bcbe0 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 -- 172.21.15.87:0/399319987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 msgr2=0x5648814bd020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814bd020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 -- 172.21.15.87:0/399319987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 msgr2=0x5648814bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 -- 172.21.15.87:0/399319987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648813a4050 con 0x5648814c2450 2024-06-06T08:01:44.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.444+0000 7fa4f1797700 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80984f0 tx=0x7fa4e809c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8c498fb635c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.445+0000 7fa4d77fe700 1 -- 172.21.15.87:0/399319987 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80a4790 con 0x5648814c2450 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.445+0000 7fa4d77fe700 1 -- 172.21.15.87:0/399319987 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e809ce50 con 0x5648814c2450 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.445+0000 7fa4d77fe700 1 -- 172.21.15.87:0/399319987 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8010720 con 0x5648814c2450 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 msgr2=0x5648814c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80984f0 tx=0x7fa4e809c340 comp rx=0 tx=0).stop 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 shutdown_connections 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814c2830 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/399319987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 >> 172.21.15.87:0/399319987 conn(0x5648813dedc0 msgr2=0x5648814af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 shutdown_connections 2024-06-06T08:01:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.446+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/399319987 wait complete. 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.447+0000 7fa4fe27eb80 1 -- start start 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.447+0000 7fa4fe27eb80 1 Processor -- start 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2450 0x5648814c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54690/0 (socket says 172.21.15.87:54690) 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 -- 172.21.15.87:0/3989116167 learned_addr learned my addr 172.21.15.87:0/3989116167 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5648814c97e0 con 0x5648814bcbe0 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648813a5cf0 con 0x5648814c2450 2024-06-06T08:01:44.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5648814c9960 con 0x5648814c2d70 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4fe27eb80 10 client.0 _ll_register_callbacks cb 0x56488139fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1f98700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 -- 172.21.15.87:0/3989116167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 msgr2=0x5648814c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 -- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2450 msgr2=0x5648814c5f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2450 0x5648814c5f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.448+0000 7fa4f1797700 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648813549d0 con 0x5648814bcbe0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80a4260 tx=0x7fa4e809c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9fe6b36d485484 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8097dc0 con 0x5648814bcbe0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fa4e8010930 con 0x5648814bcbe0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e809ce50 con 0x5648814bcbe0 2024-06-06T08:01:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa4e80a4790 con 0x5648814bcbe0 2024-06-06T08:01:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4fe27eb80 10 client.17514 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5648814d4040 con 0x5648814bcbe0 2024-06-06T08:01:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.449+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e809af70 con 0x5648814bcbe0 2024-06-06T08:01:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fa4e80a1060 con 0x5648814bcbe0 2024-06-06T08:01:44.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 10 client.17514 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 10 client.17514 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 20 client.17514 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5648814d6510 0x5648814d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 4 client.17514 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4e809ce50 con 0x5648814d6510 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5648814d6510 0x5648814d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4d4ff9700 10 client.17514 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.450+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5648814d6510 0x5648814d8900 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fc5f5fb5c51cb6 server_cookie=b25795685ec86dd5 in_seq=0 out_seq=0 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3989116167 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa4e809ce50 con 0x5648814d6510 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4d4ff9700 10 client.17514 handle_command_reply: tid=0 2024-06-06T08:01:44.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 client.17514 shutdown 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 msgr2=0x5648814c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80a4260 tx=0x7fa4e809c040 comp rx=0 tx=0).stop 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 shutdown_connections 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5648814d6510 0x5648814d8900 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814bcbe0 0x5648814c8bc0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3989116167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2450 0x5648814c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 >> 172.21.15.87:0/3989116167 conn(0x5648813dedc0 msgr2=0x5648814aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4d4ff9700 0 client.17514 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 shutdown_connections 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3989116167 wait complete. 2024-06-06T08:01:44.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.452+0000 7fa4fe27eb80 20 client.17514 trim_cache size 0 max 16384 2024-06-06T08:01:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f443ffff640 1 -- 172.21.15.87:0/1432943746 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558376d7bc20 con 0x558376e94700 2024-06-06T08:01:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f443ffff640 10 client.17502 handle_command_reply: tid=0 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 client.17502 shutdown 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 msgr2=0x558376e1ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4458006010 tx=0x7f4458004280 comp rx=0 tx=0).stop 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 shutdown_connections 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558376e94700 0x558376e96af0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558376e15350 0x558376e1ce80 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558376e0aa90 0x558376e21ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f443ffff640 0 client.17502 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 --2- 172.21.15.87:0/1432943746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558376e0a170 0x558376e21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 >> 172.21.15.87:0/1432943746 conn(0x558376e06680 msgr2=0x558376e10ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 shutdown_connections 2024-06-06T08:01:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 1 -- 172.21.15.87:0/1432943746 wait complete. 2024-06-06T08:01:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.456+0000 7f446979b740 20 client.17502 trim_cache size 0 max 16384 2024-06-06T08:01:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:01:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:01:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 msgr2=0x7f44500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 0x7f44500a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f446006a0a0 tx=0x7f444c005f90 comp rx=0 tx=0).stop 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 msgr2=0x7f44601981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f445402aa00 tx=0x7f4454003c10 comp rx=0 tx=0).stop 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 shutdown_connections 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f44500a2da0 0x7f44500a5260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4460106890 0x7f446019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4460101290 0x7f44601986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 --2- 172.21.15.87:0/711513796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44601008d0 0x7f44601981a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 >> 172.21.15.87:0/711513796 conn(0x7f4460077a20 msgr2=0x7f44600ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 shutdown_connections 2024-06-06T08:01:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:44.457+0000 7f4466a95640 1 -- 172.21.15.87:0/711513796 wait complete. 2024-06-06T08:01:44.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 Processor -- start 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 -- start start 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814bcbe0 0x5648814bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1aed90 con 0x5648814c2450 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ac620 con 0x5648814bcbe0 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1aa490 con 0x5648814c2d70 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4f0f96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4f0f96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54706/0 (socket says 172.21.15.87:54706) 2024-06-06T08:01:44.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4f0f96700 1 -- 172.21.15.87:0/2947711238 learned_addr learned my addr 172.21.15.87:0/2947711238 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4f1f98700 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.473+0000 7fa4f1797700 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814bcbe0 0x5648814bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 -- 172.21.15.87:0/2947711238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 msgr2=0x5648814c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 -- 172.21.15.87:0/2947711238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814bcbe0 msgr2=0x5648814bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814bcbe0 0x5648814bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 -- 172.21.15.87:0/2947711238 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648814a6dc0 con 0x5648814c2450 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009990 tx=0x7fa4dc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456725a3581497e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4d6ffd700 1 -- 172.21.15.87:0/2947711238 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc036070 con 0x5648814c2450 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4d6ffd700 1 -- 172.21.15.87:0/2947711238 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc003d60 con 0x5648814c2450 2024-06-06T08:01:44.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.474+0000 7fa4d6ffd700 1 -- 172.21.15.87:0/2947711238 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00fd90 con 0x5648814c2450 2024-06-06T08:01:44.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/2947711238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 msgr2=0x5648814a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009990 tx=0x7fa4dc021000 comp rx=0 tx=0).stop 2024-06-06T08:01:44.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/2947711238 shutdown_connections 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814bcbe0 0x5648814bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814c2d70 0x5648814c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/2947711238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814a89c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/2947711238 >> 172.21.15.87:0/2947711238 conn(0x5648813dedc0 msgr2=0x5648814c3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/2947711238 shutdown_connections 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/2947711238 wait complete. 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 -- start start 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.475+0000 7fa4fe27eb80 1 Processor -- start 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1aaae0 con 0x5648814c2450 2024-06-06T08:01:44.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1abf50 con 0x5648814c2d70 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ab1b0 con 0x5648814bcbe0 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 10 client.0 _ll_register_callbacks cb 0x5648814a9980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54722/0 (socket says 172.21.15.87:54722) 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 -- 172.21.15.87:0/3087379303 learned_addr learned my addr 172.21.15.87:0/3087379303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f1f98700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 -- 172.21.15.87:0/3087379303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 msgr2=0x5648814db5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814db5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 -- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 msgr2=0x5648814d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:44.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 -- 172.21.15.87:0/3087379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648814da2b0 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4f0f96700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb4329397fb7a93 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc03e070 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4dc003d60 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.476+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fa4dc00fbf0 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc047050 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 10 client.17520 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564881541960 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc003d60 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fa4dc041020 con 0x5648814c2450 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 10 client.17520 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:44.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 10 client.17520 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 20 client.17520 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564881541f50 0x564881544340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 4 client.17520 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "da05dcbd", "release": true, "await": true}] 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "da05dcbd", "release": true, "await": true}) -- 0x564881412e10 con 0x564881541f50 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.477+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564881541f50 0x564881544340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.478+0000 7fa4d4ff9700 10 client.17520 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.478+0000 7fa4f1797700 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564881541f50 0x564881544340 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8751897dd853c1 server_cookie=d264bacbdcb7cb45 in_seq=0 out_seq=0 2024-06-06T08:01:44.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.486+0000 7fa4d4ff9700 1 -- 172.21.15.87:0/3087379303 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa4e80b3260 con 0x564881541f50 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.486+0000 7fa4d4ff9700 10 client.17520 handle_command_reply: tid=0 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.486+0000 7fa4fe27eb80 1 client.17520 shutdown 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.486+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 msgr2=0x5648814dbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.486+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc0057f0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 shutdown_connections 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564881541f50 0x564881544340 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648814c2d70 0x5648814d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648814bcbe0 0x5648814db5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4d4ff9700 0 client.17520 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 --2- 172.21.15.87:0/3087379303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648814c2450 0x5648814dbae0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 >> 172.21.15.87:0/3087379303 conn(0x5648813dedc0 msgr2=0x5648814c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 shutdown_connections 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 1 -- 172.21.15.87:0/3087379303 wait complete. 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4fe27eb80 20 client.17520 trim_cache size 0 max 16384 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:01:44.488 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 45, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 40, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "da05dcbd": { 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 40, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 142.3, 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:01:44.489 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:44.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 msgr2=0x7fa4d80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 0x7fa4d80a5250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00b2e0 tx=0x7fa4dc005960 comp rx=0 tx=0).stop 2024-06-06T08:01:44.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 msgr2=0x7fa4ec19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800e9a0 tx=0x7fa4e800ecb0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 shutdown_connections 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa4d80a2d90 0x7fa4d80a5250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4ec1073f0 0x7fa4ec19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4ec101380 0x7fa4ec19aea0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 --2- 172.21.15.87:0/1271992661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ec101d40 0x7fa4ec19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.487+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 >> 172.21.15.87:0/1271992661 conn(0x7fa4ec078300 msgr2=0x7fa4ec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.488+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 shutdown_connections 2024-06-06T08:01:44.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:01:44.488+0000 7fa4f39fb700 1 -- 172.21.15.87:0/1271992661 wait complete. 2024-06-06T08:01:44.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 678671 5.11 MB/sec execute 588 sec latency 21756.189 ms 2024-06-06T08:01:44.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:44 smithi195 ceph-mon[37040]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 350 KiB/s wr, 46 op/s 2024-06-06T08:01:44.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:44 smithi181 ceph-mon[36790]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 350 KiB/s wr, 46 op/s 2024-06-06T08:01:45.382 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'da05dcbd', total seconds elapsed: 28.2 2024-06-06T08:01:45.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 680412 5.11 MB/sec execute 589 sec latency 17.424 ms 2024-06-06T08:01:46.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681862 5.12 MB/sec execute 590 sec latency 432.188 ms 2024-06-06T08:01:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:46 smithi195 ceph-mon[37040]: pgmap v434: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 300 KiB/s wr, 26 op/s 2024-06-06T08:01:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:46 smithi181 ceph-mon[36790]: pgmap v434: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 300 KiB/s wr, 26 op/s 2024-06-06T08:01:46.861 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:46 smithi087 ceph-mon[32276]: pgmap v434: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 300 KiB/s wr, 26 op/s 2024-06-06T08:01:47.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681862 5.12 MB/sec execute 591 sec latency 1432.287 ms 2024-06-06T08:01:48.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681926 5.11 MB/sec execute 592 sec latency 1587.802 ms 2024-06-06T08:01:48.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:48 smithi195 ceph-mon[37040]: pgmap v435: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-06-06T08:01:48.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:48 smithi181 ceph-mon[36790]: pgmap v435: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-06-06T08:01:48.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:48 smithi087 ceph-mon[32276]: pgmap v435: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-06-06T08:01:49.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681926 5.10 MB/sec execute 593 sec latency 1825.141 ms 2024-06-06T08:01:50.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:50 smithi087 ceph-mon[32276]: pgmap v436: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 611 KiB/s wr, 61 op/s 2024-06-06T08:01:50.696 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681926 5.09 MB/sec execute 594 sec latency 2825.188 ms 2024-06-06T08:01:50.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:50 smithi195 ceph-mon[37040]: pgmap v436: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 611 KiB/s wr, 61 op/s 2024-06-06T08:01:50.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:50 smithi181 ceph-mon[36790]: pgmap v436: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 611 KiB/s wr, 61 op/s 2024-06-06T08:01:51.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681926 5.08 MB/sec execute 595 sec latency 3825.242 ms 2024-06-06T08:01:52.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681926 5.07 MB/sec execute 596 sec latency 4825.580 ms 2024-06-06T08:01:52.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:52 smithi195 ceph-mon[37040]: pgmap v437: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 607 KiB/s wr, 60 op/s 2024-06-06T08:01:52.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:52 smithi181 ceph-mon[36790]: pgmap v437: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 607 KiB/s wr, 60 op/s 2024-06-06T08:01:52.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:52 smithi087 ceph-mon[32276]: pgmap v437: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 607 KiB/s wr, 60 op/s 2024-06-06T08:01:53.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681993 5.06 MB/sec execute 597 sec latency 4975.678 ms 2024-06-06T08:01:54.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681993 5.06 MB/sec execute 598 sec latency 1755.686 ms 2024-06-06T08:01:54.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:54 smithi195 ceph-mon[37040]: pgmap v438: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 629 KiB/s wr, 66 op/s 2024-06-06T08:01:54.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:54 smithi181 ceph-mon[36790]: pgmap v438: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 629 KiB/s wr, 66 op/s 2024-06-06T08:01:54.860 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:54 smithi087 ceph-mon[32276]: pgmap v438: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 629 KiB/s wr, 66 op/s 2024-06-06T08:01:55.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 681993 5.05 MB/sec execute 599 sec latency 2755.790 ms 2024-06-06T08:01:56.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 600 sec 2024-06-06T08:01:56.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:56 smithi195 ceph-mon[37040]: pgmap v439: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 333 KiB/s wr, 43 op/s 2024-06-06T08:01:56.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:56 smithi181 ceph-mon[36790]: pgmap v439: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 333 KiB/s wr, 43 op/s 2024-06-06T08:01:56.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:56 smithi087 ceph-mon[32276]: pgmap v439: 129 pgs: 129 active+clean; 82 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 333 KiB/s wr, 43 op/s 2024-06-06T08:01:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 -- 172.21.15.87:0/3871289654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 msgr2=0x7f9070102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 --2- 172.21.15.87:0/3871289654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070102fb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402a480 comp rx=0 tx=0).stop 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 -- 172.21.15.87:0/3871289654 shutdown_connections 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 --2- 172.21.15.87:0/3871289654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 0x7f907010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 --2- 172.21.15.87:0/3871289654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f907010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.555+0000 7f9078510640 1 --2- 172.21.15.87:0/3871289654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070102fb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- 172.21.15.87:0/3871289654 >> 172.21.15.87:0/3871289654 conn(0x7f9070077a20 msgr2=0x7f9070077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- 172.21.15.87:0/3871289654 shutdown_connections 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- 172.21.15.87:0/3871289654 wait complete. 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 Processor -- start 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- start start 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f9070198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 0x7f907019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111d80 con 0x7f9070102bd0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111c00 con 0x7f90701034f0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.556+0000 7f9078510640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111f00 con 0x7f907010bfa0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51658/0 (socket says 172.21.15.87:51658) 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 -- 172.21.15.87:0/2788609342 learned_addr learned my addr 172.21.15.87:0/2788609342 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f9075aab640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f9070198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 -- 172.21.15.87:0/2788609342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 msgr2=0x7f907019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f9076aad640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 0x7f907019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 0x7f907019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 -- 172.21.15.87:0/2788609342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 msgr2=0x7f9070198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f9070198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 -- 172.21.15.87:0/2788609342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019d1c0 con 0x7f9070102bd0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f9075aab640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f9070198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90762ac640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9064006010 tx=0x7f90640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90637fe640 1 -- 172.21.15.87:0/2788609342 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906403f030 con 0x7f9070102bd0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90637fe640 1 -- 172.21.15.87:0/2788609342 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064003ce0 con 0x7f9070102bd0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019d450 con 0x7f9070102bd0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f90637fe640 1 -- 172.21.15.87:0/2788609342 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064035070 con 0x7f9070102bd0 2024-06-06T08:01:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.557+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a4d40 con 0x7f9070102bd0 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.559+0000 7f90637fe640 1 -- 172.21.15.87:0/2788609342 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9064042030 con 0x7f9070102bd0 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 Processor -- start 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 -- start start 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x556334427550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55633438b4f0 con 0x556334427a90 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556334329870 con 0x556334432350 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.560+0000 7f907b21d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5563343874b0 con 0x556334427170 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90637fe640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 0x7f905c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90762ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90762ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51666/0 (socket says 172.21.15.87:51666) 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56976/0 (socket says 172.21.15.87:56976) 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90762ac640 1 -- 172.21.15.87:0/2273199732 learned_addr learned my addr 172.21.15.87:0/2273199732 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9076aad640 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x556334427550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9076aad640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 0x7f905c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90637fe640 1 -- 172.21.15.87:0/2788609342 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f90640efe10 con 0x7f9070102bd0 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 -- 172.21.15.87:0/2273199732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 msgr2=0x556334427550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x556334427550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 -- 172.21.15.87:0/2273199732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 msgr2=0x556334431e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 -- 172.21.15.87:0/2273199732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563343459b0 con 0x556334432350 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f90762ac640 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9076aad640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 0x7f905c0a5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9068009fd0 tx=0x7f9068009820 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.561+0000 7f9075aab640 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f906c0058c0 tx=0x7f906c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1793ff838f5688 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f9060bfe640 1 -- 172.21.15.87:0/2273199732 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c005220 con 0x556334432350 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f9060bfe640 1 -- 172.21.15.87:0/2273199732 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004c80 con 0x556334432350 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f9060bfe640 1 -- 172.21.15.87:0/2273199732 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c010830 con 0x556334432350 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 -- 172.21.15.87:0/2273199732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 msgr2=0x556334434740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f906c0058c0 tx=0x7f906c00e720 comp rx=0 tx=0).stop 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 -- 172.21.15.87:0/2273199732 shutdown_connections 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x556334434740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334431e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 --2- 172.21.15.87:0/2273199732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x556334427550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 -- 172.21.15.87:0/2273199732 >> 172.21.15.87:0/2273199732 conn(0x556334423680 msgr2=0x556334423a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.562+0000 7f907b21d740 1 -- 172.21.15.87:0/2273199732 shutdown_connections 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 -- 172.21.15.87:0/2273199732 wait complete. 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 -- start start 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 Processor -- start 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x55633442fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556334434f80 con 0x556334427a90 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556334314ae0 con 0x556334432350 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556334435100 con 0x556334427170 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f907b21d740 10 client.0 _ll_register_callbacks cb 0x55633434d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f9075aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f9075aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51680/0 (socket says 172.21.15.87:51680) 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f9075aab640 1 -- 172.21.15.87:0/1288199509 learned_addr learned my addr 172.21.15.87:0/1288199509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.563+0000 7f9076aad640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x55633442fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f90762ac640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 -- 172.21.15.87:0/1288199509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 msgr2=0x55633442f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 -- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 msgr2=0x55633442fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x55633442fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 -- 172.21.15.87:0/1288199509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633409f210 con 0x556334427a90 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f90762ac640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9075aab640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f906c010470 tx=0x7f906c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2af4208b578f63 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c02a060 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f906c005040 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c004c80 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f906c031050 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f907b21d740 10 client.17532 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55633416b620 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004c80 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.564+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f906c01a460 con 0x556334427a90 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 10 client.17532 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 20 client.17532 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556334442360 0x556334444750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 4 client.17532 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55633439ef10 con 0x556334442360 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f90762ac640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556334442360 0x556334444750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f9059ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.565+0000 7f90762ac640 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556334442360 0x556334444750 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c10bf31aee2cae server_cookie=4eca8090665472b0 in_seq=0 out_seq=0 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f9059ffb640 1 -- 172.21.15.87:0/1288199509 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55633439ef10 con 0x556334442360 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f9059ffb640 10 client.17532 handle_command_reply: tid=0 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 client.17532 shutdown 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 msgr2=0x55633442f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f906c010470 tx=0x7f906c00cea0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 shutdown_connections 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556334442360 0x556334444750 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334432350 0x55633442fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x55633442f780 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 --2- 172.21.15.87:0/1288199509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f9059ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 >> 172.21.15.87:0/1288199509 conn(0x556334423680 msgr2=0x556334429400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 shutdown_connections 2024-06-06T08:01:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 1 -- 172.21.15.87:0/1288199509 wait complete. 2024-06-06T08:01:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.567+0000 7f907b21d740 20 client.17532 trim_cache size 0 max 16384 2024-06-06T08:01:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.582+0000 7f907b21d740 1 Processor -- start 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.582+0000 7f907b21d740 1 -- start start 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427a90 0x556334449de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b1770 con 0x556334432350 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90701af040 con 0x556334427a90 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f907b21d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ace90 con 0x556334427170 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51690/0 (socket says 172.21.15.87:51690) 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 -- 172.21.15.87:0/582831352 learned_addr learned my addr 172.21.15.87:0/582831352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f90762ac640 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 -- 172.21.15.87:0/582831352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 msgr2=0x55633442f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 -- 172.21.15.87:0/582831352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427a90 msgr2=0x556334449de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427a90 0x556334449de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 -- 172.21.15.87:0/582831352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633441ef70 con 0x556334432350 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f90762ac640 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f9076aad640 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9068011ab0 tx=0x7f906801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dacdc4a68eb8e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f905bfff640 1 -- 172.21.15.87:0/582831352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004d20 con 0x556334432350 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f905bfff640 1 -- 172.21.15.87:0/582831352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9068004510 con 0x556334432350 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.583+0000 7f905bfff640 1 -- 172.21.15.87:0/582831352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068005770 con 0x556334432350 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- 172.21.15.87:0/582831352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 msgr2=0x55633444a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9068011ab0 tx=0x7f906801f000 comp rx=0 tx=0).stop 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- 172.21.15.87:0/582831352 shutdown_connections 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334432350 0x55633444a320 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427a90 0x556334449de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- 172.21.15.87:0/582831352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334427170 0x55633442f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- 172.21.15.87:0/582831352 >> 172.21.15.87:0/582831352 conn(0x556334423680 msgr2=0x55633442f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- 172.21.15.87:0/582831352 shutdown_connections 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- 172.21.15.87:0/582831352 wait complete. 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- start start 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 Processor -- start 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 0x556334435910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 0x5563344af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ad500 con 0x556334427a90 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ae970 con 0x556334427170 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.584+0000 7f907b21d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90701adbd0 con 0x556334432350 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f907b21d740 10 client.0 _ll_register_callbacks cb 0x7f906c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:01:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51700/0 (socket says 172.21.15.87:51700) 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 -- 172.21.15.87:0/2628712344 learned_addr learned my addr 172.21.15.87:0/2628712344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9076aad640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 0x5563344af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f90762ac640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 0x556334435910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 -- 172.21.15.87:0/2628712344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 msgr2=0x5563344af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 0x5563344af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 -- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 msgr2=0x556334435910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 0x556334435910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 -- 172.21.15.87:0/2628712344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563344af6f0 con 0x556334427a90 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f90762ac640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 0x556334435910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9076aad640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 0x5563344af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9075aab640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f906c00cb20 tx=0x7f906c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff39febe74260a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:01:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c01e020 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7f906c01aa50 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9068004510 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f906c03b050 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f907b21d740 10 client.17538 fetch_fsmap learned FSMap version 45 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.585+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5563344b09a0 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9068004510 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f906c004020 con 0x556334427a90 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 10 client.17538 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 20 client.17538 populate_metadata read hostname 'smithi087' 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5563344b1020 0x5563344b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:01:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 4 client.17538 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:01:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556334398c20 con 0x5563344b1020 2024-06-06T08:01:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f90762ac640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5563344b1020 0x5563344b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:01:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f9059ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.586+0000 7f90762ac640 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5563344b1020 0x5563344b3440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6810a2d49899fc server_cookie=f4c2b2a661b6a4d4 in_seq=0 out_seq=0 2024-06-06T08:01:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f9059ffb640 1 -- 172.21.15.87:0/2628712344 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556334398c20 con 0x5563344b1020 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f9059ffb640 10 client.17538 handle_command_reply: tid=0 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 client.17538 shutdown 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 msgr2=0x556334435e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f906c00cb20 tx=0x7f906c005a90 comp rx=0 tx=0).stop 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 shutdown_connections 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5563344b1020 0x5563344b3440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556334432350 0x5563344af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f9059ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556334427a90 0x556334435e50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 --2- 172.21.15.87:0/2628712344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556334427170 0x556334435910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 >> 172.21.15.87:0/2628712344 conn(0x556334423680 msgr2=0x55633442fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 shutdown_connections 2024-06-06T08:01:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 1 -- 172.21.15.87:0/2628712344 wait complete. 2024-06-06T08:01:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.658+0000 7f907b21d740 20 client.17538 trim_cache size 0 max 16384 2024-06-06T08:01:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:01:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:01:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 msgr2=0x7f905c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 0x7f905c0a5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9068009fd0 tx=0x7f9068009820 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 msgr2=0x7f9070198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9064006010 tx=0x7f90640042f0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 shutdown_connections 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f905c0a2da0 0x7f905c0a5260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f907010bfa0 0x7f907019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90701034f0 0x7f9070198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 --2- 172.21.15.87:0/2788609342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9070102bd0 0x7f9070198220 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 >> 172.21.15.87:0/2788609342 conn(0x7f9070077a20 msgr2=0x7f90700fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:01:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 shutdown_connections 2024-06-06T08:01:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:01:57.659+0000 7f9078510640 1 -- 172.21.15.87:0/2788609342 wait complete. 2024-06-06T08:01:57.697 INFO:tasks.workunit.client.0.smithi087.stdout: 1 cleanup 601 sec 2024-06-06T08:01:58.053 INFO:tasks.workunit.client.0.smithi087.stdout: 0 cleanup 601 sec 2024-06-06T08:01:58.053 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-06-06T08:01:58.053 INFO:tasks.workunit.client.0.smithi087.stdout: Operation Count AvgLat MaxLat 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: ---------------------------------------- 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: NTCreateX 96441 2.219 29194.390 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Close 70843 0.002 0.088 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Rename 4086 22.650 4975.669 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Unlink 19474 4.977 29985.429 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Deltree 2 78.603 98.448 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Mkdir 1 0.027 0.027 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Qpathinfo 87417 1.262 11903.452 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Qfileinfo 15304 0.001 0.030 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Qfsinfo 16031 0.175 42.956 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Sfileinfo 7854 3.259 25320.668 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: Find 33790 0.497 10260.767 2024-06-06T08:01:58.054 INFO:tasks.workunit.client.0.smithi087.stdout: WriteX 47994 0.180 672.420 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout: ReadX 151187 0.023 106.980 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout: LockX 314 0.780 4.076 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout: UnlockX 314 0.743 3.799 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout: Flush 6757 4.154 292.461 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stdout:Throughput 5.04782 MB/sec 1 clients 1 procs max_latency=29985.684 ms 2024-06-06T08:01:58.055 INFO:tasks.workunit.client.0.smithi087.stderr:+ dbench 10 2024-06-06T08:01:58.056 INFO:tasks.workunit.client.0.smithi087.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-06T08:01:58.056 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-06-06T08:01:58.056 INFO:tasks.workunit.client.0.smithi087.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-06T08:01:58.056 INFO:tasks.workunit.client.0.smithi087.stdout:8 of 10 processes prepared for launch 0 sec 2024-06-06T08:01:58.155 INFO:tasks.workunit.client.0.smithi087.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-06T08:01:58.155 INFO:tasks.workunit.client.0.smithi087.stdout:releasing clients 2024-06-06T08:01:58.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:01:58 smithi195 ceph-mon[37040]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 333 KiB/s wr, 45 op/s 2024-06-06T08:01:58.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:01:58 smithi181 ceph-mon[36790]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 333 KiB/s wr, 45 op/s 2024-06-06T08:01:58.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:01:58 smithi087 ceph-mon[32276]: pgmap v440: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 333 KiB/s wr, 45 op/s 2024-06-06T08:01:59.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 344 123.95 MB/sec warmup 1 sec latency 403.516 ms 2024-06-06T08:02:00.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 677 99.71 MB/sec warmup 2 sec latency 1403.555 ms 2024-06-06T08:02:00.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:00 smithi195 ceph-mon[37040]: pgmap v441: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 21 op/s 2024-06-06T08:02:00.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:00 smithi181 ceph-mon[36790]: pgmap v441: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 21 op/s 2024-06-06T08:02:00.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:00 smithi087 ceph-mon[32276]: pgmap v441: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 21 op/s 2024-06-06T08:02:01.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 800 67.65 MB/sec warmup 3 sec latency 2075.883 ms 2024-06-06T08:02:02.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 800 50.74 MB/sec warmup 4 sec latency 1213.883 ms 2024-06-06T08:02:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:02 smithi195 ceph-mon[37040]: pgmap v442: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-06-06T08:02:02.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:02 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:02:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:02 smithi181 ceph-mon[36790]: pgmap v442: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-06-06T08:02:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:02 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:02:02.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:02 smithi087 ceph-mon[32276]: pgmap v442: 129 pgs: 129 active+clean; 82 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-06-06T08:02:02.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:02 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:02:03.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 800 40.59 MB/sec warmup 5 sec latency 2213.980 ms 2024-06-06T08:02:04.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 800 33.82 MB/sec warmup 6 sec latency 3214.070 ms 2024-06-06T08:02:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:04 smithi195 ceph-mon[37040]: pgmap v443: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 16 MiB/s wr, 162 op/s 2024-06-06T08:02:04.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:04 smithi181 ceph-mon[36790]: pgmap v443: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 16 MiB/s wr, 162 op/s 2024-06-06T08:02:04.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:04 smithi087 ceph-mon[32276]: pgmap v443: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 16 MiB/s wr, 162 op/s 2024-06-06T08:02:05.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 800 28.99 MB/sec warmup 7 sec latency 4214.136 ms 2024-06-06T08:02:06.155 INFO:tasks.workunit.client.0.smithi087.stdout: 10 937 26.32 MB/sec warmup 8 sec latency 4802.226 ms 2024-06-06T08:02:06.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:06 smithi195 ceph-mon[37040]: pgmap v444: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 16 MiB/s wr, 155 op/s 2024-06-06T08:02:06.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:06 smithi181 ceph-mon[36790]: pgmap v444: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 16 MiB/s wr, 155 op/s 2024-06-06T08:02:06.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:06 smithi087 ceph-mon[32276]: pgmap v444: 129 pgs: 129 active+clean; 259 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 16 MiB/s wr, 155 op/s 2024-06-06T08:02:07.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1347 25.87 MB/sec warmup 9 sec latency 1446.643 ms 2024-06-06T08:02:08.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 1871 27.16 MB/sec warmup 10 sec latency 2446.677 ms 2024-06-06T08:02:08.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:08 smithi195 ceph-mon[37040]: pgmap v445: 129 pgs: 129 active+clean; 260 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 16 MiB/s wr, 166 op/s 2024-06-06T08:02:08.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:08 smithi181 ceph-mon[36790]: pgmap v445: 129 pgs: 129 active+clean; 260 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 16 MiB/s wr, 166 op/s 2024-06-06T08:02:08.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:08 smithi087 ceph-mon[32276]: pgmap v445: 129 pgs: 129 active+clean; 260 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 16 MiB/s wr, 166 op/s 2024-06-06T08:02:09.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2406 27.26 MB/sec warmup 11 sec latency 3446.711 ms 2024-06-06T08:02:10.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2751 27.39 MB/sec warmup 12 sec latency 4446.749 ms 2024-06-06T08:02:10.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:10 smithi195 ceph-mon[37040]: pgmap v446: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 -- 172.21.15.87:0/2727711353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 msgr2=0x7fbd2c069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 --2- 172.21.15.87:0/2727711353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c069050 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009ca0 tx=0x7fbd2402a8c0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 -- 172.21.15.87:0/2727711353 shutdown_connections 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 --2- 172.21.15.87:0/2727711353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c069050 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 --2- 172.21.15.87:0/2727711353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 --2- 172.21.15.87:0/2727711353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c106890 0x7fbd2c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 -- 172.21.15.87:0/2727711353 >> 172.21.15.87:0/2727711353 conn(0x7fbd2c077a20 msgr2=0x7fbd2c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 -- 172.21.15.87:0/2727711353 shutdown_connections 2024-06-06T08:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 -- 172.21.15.87:0/2727711353 wait complete. 2024-06-06T08:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.768+0000 7fbd329c8640 1 Processor -- start 2024-06-06T08:02:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 -- start start 2024-06-06T08:02:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c106890 0x7fbd2c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c111db0 con 0x7fbd2c101290 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c111c30 con 0x7fbd2c106890 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd329c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c111f30 con 0x7fbd2c1008d0 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45200/0 (socket says 172.21.15.87:45200) 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/3300422525 learned_addr learned my addr 172.21.15.87:0/3300422525 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.769+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52904/0 (socket says 172.21.15.87:52904) 2024-06-06T08:02:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/3300422525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 msgr2=0x7fbd2c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/3300422525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c106890 msgr2=0x7fbd2c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c106890 0x7fbd2c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/3300422525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c19d1a0 con 0x7fbd2c101290 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e9f0 tx=0x7fbd2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd297fa640 1 -- 172.21.15.87:0/3300422525 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000ccc0 con 0x7fbd2c101290 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd297fa640 1 -- 172.21.15.87:0/3300422525 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2000ce60 con 0x7fbd2c101290 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c19d490 con 0x7fbd2c101290 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd297fa640 1 -- 172.21.15.87:0/3300422525 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20010690 con 0x7fbd2c101290 2024-06-06T08:02:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.770+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a4d20 con 0x7fbd2c101290 2024-06-06T08:02:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 Processor -- start 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 -- start start 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a90f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 0x55fe7a91c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a8734f0 con 0x55fe7a90f170 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a811870 con 0x55fe7a90fa90 2024-06-06T08:02:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.774+0000 7fbd357b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a86f4b0 con 0x55fe7a91a350 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd297fa640 1 -- 172.21.15.87:0/3300422525 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbd200046c0 con 0x7fbd2c101290 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 0x55fe7a91c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:35494/0 (socket says 172.21.15.87:35494) 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2bfff640 1 -- 172.21.15.87:0/562465943 learned_addr learned my addr 172.21.15.87:0/562465943 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd30f65640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a90f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/562465943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 msgr2=0x55fe7a919e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/562465943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 msgr2=0x55fe7a90f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a90f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.775+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/562465943 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7a82d9b0 con 0x55fe7a91a350 2024-06-06T08:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.776+0000 7fbd297fa640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 0x7fbd140a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.776+0000 7fbd30f65640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a90f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.776+0000 7fbd297fa640 1 -- 172.21.15.87:0/3300422525 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbd200cfcb0 con 0x7fbd2c101290 2024-06-06T08:02:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.776+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 0x55fe7a91c740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd200ccfd0 tx=0x7fbd20098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a1f6a9b00a5c35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.776+0000 7fbd30f65640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 0x7fbd140a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.777+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.777+0000 7fbd127fc640 1 -- 172.21.15.87:0/562465943 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2009bb70 con 0x55fe7a91a350 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.777+0000 7fbd127fc640 1 -- 172.21.15.87:0/562465943 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2009c4a0 con 0x55fe7a91a350 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.777+0000 7fbd127fc640 1 -- 172.21.15.87:0/562465943 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2009b700 con 0x55fe7a91a350 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 -- 172.21.15.87:0/562465943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 msgr2=0x55fe7a91c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 0x55fe7a91c740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd200ccfd0 tx=0x7fbd20098040 comp rx=0 tx=0).stop 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd30f65640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 0x7fbd140a5360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009bf0 tx=0x7fbd24005d80 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 -- 172.21.15.87:0/562465943 shutdown_connections 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a91a350 0x55fe7a91c740 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd200ccfd0 tx=0x7fbd20098040 comp rx=0 tx=0).stop 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90fa90 0x55fe7a919e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 --2- 172.21.15.87:0/562465943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a90f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.778+0000 7fbd357b2740 1 -- 172.21.15.87:0/562465943 >> 172.21.15.87:0/562465943 conn(0x55fe7a90b680 msgr2=0x55fe7a90ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.779+0000 7fbd357b2740 1 -- 172.21.15.87:0/562465943 shutdown_connections 2024-06-06T08:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.779+0000 7fbd357b2740 1 -- 172.21.15.87:0/562465943 wait complete. 2024-06-06T08:02:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.779+0000 7fbd357b2740 1 -- start start 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.780+0000 7fbd357b2740 1 Processor -- start 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.780+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a917430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.780+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90fa90 0x55fe7a917970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.780+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd357b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a91cec0 con 0x55fe7a90fa90 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd357b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a7fcae0 con 0x55fe7a90f170 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd357b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe7a91d040 con 0x55fe7a917eb0 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:52928/0 (socket says 172.21.15.87:52928) 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 -- 172.21.15.87:0/3233624371 learned_addr learned my addr 172.21.15.87:0/3233624371 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd357b2740 10 client.0 _ll_register_callbacks cb 0x55fe7a8359b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 -- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 msgr2=0x55fe7a917430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a917430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 -- 172.21.15.87:0/3233624371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90fa90 msgr2=0x55fe7a917970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90fa90 0x55fe7a917970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.781+0000 7fbd30f65640 1 -- 172.21.15.87:0/3233624371 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7a587210 con 0x55fe7a917eb0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.782+0000 7fbd30f65640 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd24003c50 tx=0x7fbd240095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e831e778a7faea5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.782+0000 7fbd0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.783+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240047c0 con 0x55fe7a917eb0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.783+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbd2402f610 con 0x55fe7a917eb0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.783+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd2009c4a0 con 0x55fe7a917eb0 2024-06-06T08:02:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.784+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbd240319a0 con 0x55fe7a917eb0 2024-06-06T08:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.784+0000 7fbd357b2740 10 client.26707 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.784+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fe7a653620 con 0x55fe7a917eb0 2024-06-06T08:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.784+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2009c4a0 con 0x55fe7a917eb0 2024-06-06T08:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fbd24030990 con 0x55fe7a917eb0 2024-06-06T08:02:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 10 client.26707 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 10 client.26707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 20 client.26707 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a9335c0 0x55fe7a935a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 4 client.26707 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:02:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.785+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe7a886f10 con 0x55fe7a9335c0 2024-06-06T08:02:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.789+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a9335c0 0x55fe7a935a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.790+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a9335c0 0x55fe7a935a80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bc10cf70fa7428 server_cookie=3afa42ac9c768cce in_seq=0 out_seq=0 2024-06-06T08:02:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.790+0000 7fbd0b7fe640 10 client.26707 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.799+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/3233624371 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe7a886f10 con 0x55fe7a9335c0 2024-06-06T08:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.799+0000 7fbd0b7fe640 10 client.26707 handle_command_reply: tid=0 2024-06-06T08:02:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.799+0000 7fbd357b2740 1 client.26707 shutdown 2024-06-06T08:02:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 msgr2=0x55fe7a92eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd24003c50 tx=0x7fbd240095c0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 shutdown_connections 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a9335c0 0x55fe7a935a80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a917eb0 0x55fe7a92eb40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90fa90 0x55fe7a917970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 --2- 172.21.15.87:0/3233624371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a917430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 >> 172.21.15.87:0/3233624371 conn(0x55fe7a90b680 msgr2=0x55fe7a911480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd0b7fe640 0 client.26707 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 shutdown_connections 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 1 -- 172.21.15.87:0/3233624371 wait complete. 2024-06-06T08:02:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.802+0000 7fbd357b2740 20 client.26707 trim_cache size 0 max 16384 2024-06-06T08:02:10.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:10 smithi181 ceph-mon[36790]: pgmap v446: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-06-06T08:02:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 Processor -- start 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 -- start start 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a92e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a911c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1af600 con 0x55fe7a90b680 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ace70 con 0x55fe7a90f170 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd357b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1aacc0 con 0x55fe7a90fa90 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45224/0 (socket says 172.21.15.87:45224) 2024-06-06T08:02:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2bfff640 1 -- 172.21.15.87:0/2471997288 learned_addr learned my addr 172.21.15.87:0/2471997288 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd30f65640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a911c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a92e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2bfff640 1 -- 172.21.15.87:0/2471997288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 msgr2=0x55fe7a911c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.822+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a911c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd2bfff640 1 -- 172.21.15.87:0/2471997288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 msgr2=0x55fe7a92e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a92e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd2bfff640 1 -- 172.21.15.87:0/2471997288 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7a92eef0 con 0x55fe7a90b680 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a92e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009860 tx=0x7fbd1c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6299956797dad3cd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd11ffb640 1 -- 172.21.15.87:0/2471997288 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00f040 con 0x55fe7a90b680 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd11ffb640 1 -- 172.21.15.87:0/2471997288 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c023070 con 0x55fe7a90b680 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.823+0000 7fbd11ffb640 1 -- 172.21.15.87:0/2471997288 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c01e670 con 0x55fe7a90b680 2024-06-06T08:02:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- 172.21.15.87:0/2471997288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 msgr2=0x55fe7a9355d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009860 tx=0x7fbd1c009110 comp rx=0 tx=0).stop 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- 172.21.15.87:0/2471997288 shutdown_connections 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a911c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90f170 0x55fe7a92e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2471997288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90b680 0x55fe7a9355d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- 172.21.15.87:0/2471997288 >> 172.21.15.87:0/2471997288 conn(0x55fe7a917ad0 msgr2=0x55fe7a915d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- 172.21.15.87:0/2471997288 shutdown_connections 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- 172.21.15.87:0/2471997288 wait complete. 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- start start 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 Processor -- start 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90b680 0x55fe7a925ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a926560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ab330 con 0x55fe7a90f170 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ac7a0 con 0x55fe7a90b680 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1aba00 con 0x55fe7a90fa90 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd357b2740 10 client.0 _ll_register_callbacks cb 0x7fbd20005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.824+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45240/0 (socket says 172.21.15.87:45240) 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/2094335772 learned_addr learned my addr 172.21.15.87:0/2094335772 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90b680 0x55fe7a925ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd30f65640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a926560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/2094335772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 msgr2=0x55fe7a926560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a926560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90b680 msgr2=0x55fe7a925ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90b680 0x55fe7a925ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 -- 172.21.15.87:0/2094335772 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7a99c350 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd2b7fe640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbd2009baa0 tx=0x7fbd200a0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e35cac622524ca6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2009f040 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9495+0+0 (secure 0 0 0) 0x7fbd200d3cd0 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd1c023070 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbd2009b8e0 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd357b2740 10 client.17550 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fe7a99d640 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c023070 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.825+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fbd200cd250 con 0x55fe7a90f170 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 10 client.17550 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 20 client.17550 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a917eb0 0x55fe7a99f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 4 client.17550 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe7a880c20 con 0x55fe7a917eb0 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a917eb0 0x55fe7a99f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd0b7fe640 10 client.17550 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:10.826+0000 7fbd2bfff640 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a917eb0 0x55fe7a99f940 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4bdf361f293413 server_cookie=b9e1c0eac0fcd0ca in_seq=0 out_seq=0 2024-06-06T08:02:10.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:10 smithi087 ceph-mon[32276]: pgmap v446: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-06-06T08:02:11.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 2965 26.51 MB/sec warmup 13 sec latency 5446.783 ms 2024-06-06T08:02:12.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 3086 24.99 MB/sec warmup 14 sec latency 6446.826 ms 2024-06-06T08:02:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:12 smithi195 ceph-mon[37040]: pgmap v447: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 16 MiB/s wr, 204 op/s 2024-06-06T08:02:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:12 smithi195 ceph-mon[37040]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T08:02:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:12 smithi181 ceph-mon[36790]: pgmap v447: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 16 MiB/s wr, 204 op/s 2024-06-06T08:02:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:12 smithi181 ceph-mon[36790]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T08:02:12.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:12 smithi087 ceph-mon[32276]: pgmap v447: 129 pgs: 129 active+clean; 264 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 16 MiB/s wr, 204 op/s 2024-06-06T08:02:12.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:12 smithi087 ceph-mon[32276]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T08:02:13.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 3262 23.69 MB/sec warmup 15 sec latency 7446.877 ms 2024-06-06T08:02:14.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 3509 22.91 MB/sec warmup 16 sec latency 8446.911 ms 2024-06-06T08:02:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:14 smithi195 ceph-mon[37040]: pgmap v449: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:14 smithi181 ceph-mon[36790]: pgmap v449: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:14.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:14 smithi087 ceph-mon[32276]: pgmap v449: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:15.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 3769 22.25 MB/sec warmup 17 sec latency 9446.943 ms 2024-06-06T08:02:16.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 4178 21.86 MB/sec warmup 18 sec latency 10010.409 ms 2024-06-06T08:02:16.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:16 smithi195 ceph-mon[37040]: pgmap v450: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:16.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:16 smithi181 ceph-mon[36790]: pgmap v450: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:16.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:16 smithi087 ceph-mon[32276]: pgmap v450: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 274 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 491 op/s 2024-06-06T08:02:17.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 4758 22.59 MB/sec warmup 19 sec latency 5038.525 ms 2024-06-06T08:02:17.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:17 smithi195 ceph-mon[37040]: pgmap v451: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 276 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 517 op/s 2024-06-06T08:02:17.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:17 smithi181 ceph-mon[36790]: pgmap v451: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 276 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 517 op/s 2024-06-06T08:02:17.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:17 smithi087 ceph-mon[32276]: pgmap v451: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 276 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 517 op/s 2024-06-06T08:02:18.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 5471 23.02 MB/sec warmup 20 sec latency 6038.573 ms 2024-06-06T08:02:19.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 5943 23.85 MB/sec warmup 21 sec latency 7038.606 ms 2024-06-06T08:02:20.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:19 smithi195 ceph-mon[37040]: pgmap v452: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-06-06T08:02:20.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:19 smithi181 ceph-mon[36790]: pgmap v452: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-06-06T08:02:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:19 smithi087 ceph-mon[32276]: pgmap v452: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-06-06T08:02:20.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 6306 23.95 MB/sec warmup 22 sec latency 8038.632 ms 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.747+0000 7fbd0b7fe640 1 -- 172.21.15.87:0/2094335772 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fe7a880c20 con 0x55fe7a917eb0 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.747+0000 7fbd0b7fe640 10 client.17550 handle_command_reply: tid=0 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.747+0000 7fbd357b2740 1 client.17550 shutdown 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.747+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 msgr2=0x55fe7a926020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.747+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbd2009baa0 tx=0x7fbd200a0240 comp rx=0 tx=0).stop 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 shutdown_connections 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fe7a917eb0 0x55fe7a99f940 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe7a90fa90 0x55fe7a926560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fe7a90f170 0x55fe7a926020 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 --2- 172.21.15.87:0/2094335772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe7a90b680 0x55fe7a925ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 >> 172.21.15.87:0/2094335772 conn(0x55fe7a917ad0 msgr2=0x55fe7a916160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd0b7fe640 0 client.17550 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 shutdown_connections 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 1 -- 172.21.15.87:0/2094335772 wait complete. 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd357b2740 20 client.17550 trim_cache size 0 max 16384 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:02:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:02:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 msgr2=0x7fbd140a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 0x7fbd140a5360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009bf0 tx=0x7fbd24005d80 comp rx=0 tx=0).stop 2024-06-06T08:02:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 msgr2=0x7fbd2c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.748+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e9f0 tx=0x7fbd2000edf0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 shutdown_connections 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbd140a2ea0 0x7fbd140a5360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c106890 0x7fbd2c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd2c101290 0x7fbd2c1986f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 --2- 172.21.15.87:0/3300422525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd2c1008d0 0x7fbd2c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 >> 172.21.15.87:0/3300422525 conn(0x7fbd2c077a20 msgr2=0x7fbd2c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 shutdown_connections 2024-06-06T08:02:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:20.749+0000 7fbd329c8640 1 -- 172.21.15.87:0/3300422525 wait complete. 2024-06-06T08:02:21.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:20 smithi087 ceph-mon[32276]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T08:02:21.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 6623 23.48 MB/sec warmup 23 sec latency 8637.471 ms 2024-06-06T08:02:21.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:20 smithi195 ceph-mon[37040]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T08:02:21.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:20 smithi181 ceph-mon[36790]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T08:02:22.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:21 smithi087 ceph-mon[32276]: pgmap v454: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-06T08:02:22.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 6956 23.16 MB/sec warmup 24 sec latency 78.079 ms 2024-06-06T08:02:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:21 smithi195 ceph-mon[37040]: pgmap v454: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-06T08:02:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:21 smithi181 ceph-mon[36790]: pgmap v454: 129 pgs: 129 active+clean; 276 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-06-06T08:02:23.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 7265 22.84 MB/sec warmup 25 sec latency 146.739 ms 2024-06-06T08:02:24.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:23 smithi087 ceph-mon[32276]: pgmap v455: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:24.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 7671 22.45 MB/sec warmup 26 sec latency 253.903 ms 2024-06-06T08:02:24.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:23 smithi195 ceph-mon[37040]: pgmap v455: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:23 smithi181 ceph-mon[36790]: pgmap v455: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:25.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 8237 22.57 MB/sec warmup 27 sec latency 204.377 ms 2024-06-06T08:02:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:25 smithi087 ceph-mon[32276]: pgmap v456: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:26.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 8857 23.04 MB/sec warmup 28 sec latency 181.339 ms 2024-06-06T08:02:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:25 smithi195 ceph-mon[37040]: pgmap v456: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:26.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:25 smithi181 ceph-mon[36790]: pgmap v456: 129 pgs: 129 active+clean; 277 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-06-06T08:02:26.797 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T08:02:26.797 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T08:02:26.798 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:02:27.106 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:02:27.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 9409 23.24 MB/sec warmup 29 sec latency 204.898 ms 2024-06-06T08:02:27.931 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:27 smithi087 ceph-mon[32276]: pgmap v457: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-06-06T08:02:28.156 INFO:tasks.workunit.client.0.smithi087.stdout: 10 9991 23.80 MB/sec warmup 30 sec latency 186.834 ms 2024-06-06T08:02:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:27 smithi195 ceph-mon[37040]: pgmap v457: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-06-06T08:02:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:27 smithi181 ceph-mon[36790]: pgmap v457: 129 pgs: 129 active+clean; 278 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.5 MiB/s wr, 255 op/s 2024-06-06T08:02:29.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 10383 23.94 MB/sec warmup 31 sec latency 72.424 ms 2024-06-06T08:02:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/2450204584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc107450 msgr2=0x7fe5dc107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/2450204584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc107450 0x7fe5dc107830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098f0 tx=0x7fe5cc009c00 comp rx=0 tx=0).stop 2024-06-06T08:02:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/2450204584 shutdown_connections 2024-06-06T08:02:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/2450204584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/2450204584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc107450 0x7fe5dc107830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/2450204584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc102540 0x7fe5dc1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/2450204584 >> 172.21.15.87:0/2450204584 conn(0x7fe5dc0795f0 msgr2=0x7fe5dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:29.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/2450204584 shutdown_connections 2024-06-06T08:02:29.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.349+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/2450204584 wait complete. 2024-06-06T08:02:29.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 Processor -- start 2024-06-06T08:02:29.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 -- start start 2024-06-06T08:02:29.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc102540 0x7fe5dc10f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:29.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc107450 0x7fe5dc10c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc075f00 con 0x7fe5dc102ee0 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc075d80 con 0x7fe5dc107450 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5e1ce5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc076080 con 0x7fe5dc102540 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5db7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc102540 0x7fe5dc10f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5daffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35606/0 (socket says 172.21.15.87:35606) 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5daffd700 1 -- 172.21.15.87:0/1344017059 learned_addr learned my addr 172.21.15.87:0/1344017059 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.350+0000 7fe5dbfff700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc107450 0x7fe5dc10c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 -- 172.21.15.87:0/1344017059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc102540 msgr2=0x7fe5dc10f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc102540 0x7fe5dc10f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 -- 172.21.15.87:0/1344017059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc107450 msgr2=0x7fe5dc10c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:29.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc107450 0x7fe5dc10c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 -- 172.21.15.87:0/1344017059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc10cd30 con 0x7fe5dc102ee0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5daffd700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e9c0 tx=0x7fe5c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400cbd0 con 0x7fe5dc102ee0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1ab930 con 0x7fe5dc102ee0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c400ee60 con 0x7fe5dc102ee0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4005630 con 0x7fe5dc102ee0 2024-06-06T08:02:29.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.351+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1abd90 con 0x7fe5dc102ee0 2024-06-06T08:02:29.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.352+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc103c00 con 0x7fe5dc102ee0 2024-06-06T08:02:29.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.354+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe5c4004750 con 0x7fe5dc102ee0 2024-06-06T08:02:29.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.355+0000 7fe5d8ff9700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 0x7fe5c80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:29.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.355+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7fe5c40cfae0 con 0x7fe5dc102ee0 2024-06-06T08:02:29.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.355+0000 7fe5db7fe700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 0x7fe5c80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:29.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.356+0000 7fe5db7fe700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 0x7fe5c80a5250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005e90 tx=0x7fe5cc019040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:29.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.360+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5c409c2e0 con 0x7fe5dc102ee0 2024-06-06T08:02:29.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.641+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5dc1abff0 con 0x7fe5dc102ee0 2024-06-06T08:02:29.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.642+0000 7fe5d8ff9700 1 -- 172.21.15.87:0/1344017059 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11881 (secure 0 0 0) 0x7fe5c409bbe0 con 0x7fe5dc102ee0 2024-06-06T08:02:29.642 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:02:29.642 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:02:00.625810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26623":{"gid":26623,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1019415876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1019415876},{"type":"v1","addr":"172.21.15.195:6837","nonce":1019415876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:02:29.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 msgr2=0x7fe5c80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:29.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 0x7fe5c80a5250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005e90 tx=0x7fe5cc019040 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 msgr2=0x7fe5dc10c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e9c0 tx=0x7fe5c400ecd0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 shutdown_connections 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5dc107450 0x7fe5dc10c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe5c80a2d90 0x7fe5c80a5250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5dc102540 0x7fe5dc10f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 --2- 172.21.15.87:0/1344017059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc102ee0 0x7fe5dc10c0f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 >> 172.21.15.87:0/1344017059 conn(0x7fe5dc0795f0 msgr2=0x7fe5dc100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 shutdown_connections 2024-06-06T08:02:29.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:29.644+0000 7fe5e1ce5700 1 -- 172.21.15.87:0/1344017059 wait complete. 2024-06-06T08:02:29.645 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 46 2024-06-06T08:02:29.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:29 smithi087 ceph-mon[32276]: pgmap v458: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-06-06T08:02:29.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:29 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1344017059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:30.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 10670 23.69 MB/sec warmup 32 sec latency 120.266 ms 2024-06-06T08:02:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:29 smithi195 ceph-mon[37040]: pgmap v458: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-06-06T08:02:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:29 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1344017059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:29 smithi181 ceph-mon[36790]: pgmap v458: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-06-06T08:02:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:29 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1344017059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:30.360 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T08:02:30.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T08:02:30.659 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:02:31.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:30 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:02:31.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11040 23.44 MB/sec warmup 33 sec latency 124.176 ms 2024-06-06T08:02:31.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:30 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:02:31.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:30 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:02:32.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:31 smithi087 ceph-mon[32276]: pgmap v459: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-06-06T08:02:32.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11367 23.32 MB/sec warmup 34 sec latency 411.440 ms 2024-06-06T08:02:32.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:31 smithi195 ceph-mon[37040]: pgmap v459: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-06-06T08:02:32.207 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:31 smithi181 ceph-mon[36790]: pgmap v459: 129 pgs: 129 active+clean; 291 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 -- 172.21.15.87:0/3121851722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06b420 msgr2=0x7fe2ec10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/3121851722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06b420 0x7fe2ec10f340 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b0a0 tx=0x7fe2e400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 -- 172.21.15.87:0/3121851722 shutdown_connections 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/3121851722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 0x7fe2ec108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/3121851722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06b420 0x7fe2ec10f340 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/3121851722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 -- 172.21.15.87:0/3121851722 >> 172.21.15.87:0/3121851722 conn(0x7fe2ec0783d0 msgr2=0x7fe2ec0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 -- 172.21.15.87:0/3121851722 shutdown_connections 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.581+0000 7fe2f41c8700 1 -- 172.21.15.87:0/3121851722 wait complete. 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 Processor -- start 2024-06-06T08:02:32.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 -- start start 2024-06-06T08:02:32.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 0x7fe2ec198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec112780 con 0x7fe2ec06aa60 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec112600 con 0x7fe2ec108520 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f41c8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec112900 con 0x7fe2ec06b420 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f2765700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 0x7fe2ec198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:53390/0 (socket says 172.21.15.87:53390) 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 -- 172.21.15.87:0/2533312239 learned_addr learned my addr 172.21.15.87:0/2533312239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.582+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 -- 172.21.15.87:0/2533312239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 msgr2=0x7fe2ec198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 0x7fe2ec198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 -- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 msgr2=0x7fe2ec19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 -- 172.21.15.87:0/2533312239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec199080 con 0x7fe2ec06b420 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e9a0 tx=0x7fe2dc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.583+0000 7fe2e2ffd700 1 -- 172.21.15.87:0/2533312239 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00cbd0 con 0x7fe2ec06b420 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.584+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec199370 con 0x7fe2ec06b420 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.584+0000 7fe2e2ffd700 1 -- 172.21.15.87:0/2533312239 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc00cd70 con 0x7fe2ec06b420 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.584+0000 7fe2e2ffd700 1 -- 172.21.15.87:0/2533312239 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc018750 con 0x7fe2ec06b420 2024-06-06T08:02:32.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.584+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec1a55e0 con 0x7fe2ec06b420 2024-06-06T08:02:32.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.586+0000 7fe2e2ffd700 1 -- 172.21.15.87:0/2533312239 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe2dc0188f0 con 0x7fe2ec06b420 2024-06-06T08:02:32.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.587+0000 7fe2e2ffd700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 0x7fe2d80a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.587+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 0x7fe2d80a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.587+0000 7fe2e2ffd700 1 -- 172.21.15.87:0/2533312239 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7fe2dc0cfb80 con 0x7fe2ec06b420 2024-06-06T08:02:32.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.587+0000 7fe2fea4bb80 1 Processor -- start 2024-06-06T08:02:32.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 -- start start 2024-06-06T08:02:32.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 0x7fe2d80a52f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec078960 tx=0x7fe2e8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e2d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 0x5595d7e28110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5595d7d1a100 con 0x5595d7e27cf0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595d7d789f0 con 0x5595d7e2d5a0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5595d7ce0600 con 0x5595d7e2dec0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e2d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f1763700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e2d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54994/0 (socket says 172.21.15.87:54994) 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f2765700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 0x5595d7e28110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.588+0000 7fe2f1f64700 1 -- 172.21.15.87:0/4137225548 learned_addr learned my addr 172.21.15.87:0/4137225548 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 -- 172.21.15.87:0/4137225548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 msgr2=0x5595d7e277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 -- 172.21.15.87:0/4137225548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 msgr2=0x5595d7e2d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e2d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 -- 172.21.15.87:0/4137225548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d7cc7a70 con 0x5595d7e27cf0 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2f2765700 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 0x5595d7e28110 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b070 tx=0x7fe2e40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c370dfd5412e0b88 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2d7fff700 1 -- 172.21.15.87:0/4137225548 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4020070 con 0x5595d7e27cf0 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2d7fff700 1 -- 172.21.15.87:0/4137225548 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4006020 con 0x5595d7e27cf0 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.589+0000 7fe2d7fff700 1 -- 172.21.15.87:0/4137225548 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4007be0 con 0x5595d7e27cf0 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/4137225548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 msgr2=0x5595d7e28110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 0x5595d7e28110 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b070 tx=0x7fe2e40092e0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/4137225548 shutdown_connections 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e2d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e27cf0 0x5595d7e28110 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/4137225548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/4137225548 >> 172.21.15.87:0/4137225548 conn(0x5595d7d54220 msgr2=0x5595d7e22aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/4137225548 shutdown_connections 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.590+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/4137225548 wait complete. 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 -- start start 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 Processor -- start 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 0x5595d7e30f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e31440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5595d7d19160 con 0x5595d7e2d5a0 2024-06-06T08:02:32.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595d7d749b0 con 0x5595d7e27cf0 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5595d7e34840 con 0x5595d7e2dec0 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.591+0000 7fe2fea4bb80 10 client.0 _ll_register_callbacks cb 0x5595d7d0e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 0x5595d7e30f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54998/0 (socket says 172.21.15.87:54998) 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1f64700 1 -- 172.21.15.87:0/3698117430 learned_addr learned my addr 172.21.15.87:0/3698117430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:32.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f2765700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e31440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 -- 172.21.15.87:0/3698117430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 msgr2=0x5595d7e31440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e31440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 -- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 msgr2=0x5595d7e33c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 -- 172.21.15.87:0/3698117430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d7d3e8f0 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2f1763700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 0x5595d7e30f00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0cceb0 tx=0x7fe2dc09b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853112869cac828b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc09c0e0 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7fe2dc0a4460 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.592+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e4006020 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe2dc09c380 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 10 client.17565 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5595d7e3f000 con 0x5595d7e2d5a0 2024-06-06T08:02:32.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4006020 con 0x5595d7e2d5a0 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fe2dc0b4030 con 0x5595d7e2d5a0 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 10 client.17565 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 10 client.17565 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 20 client.17565 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e41590 0x5595d7e43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 4 client.17565 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d7d85ef0 con 0x5595d7e41590 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.593+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e41590 0x5595d7e43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.594+0000 7fe2d57fa700 10 client.17565 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:32.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.594+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e41590 0x5595d7e43980 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95033e9c3bfc9322 server_cookie=36e902f39bde3082 in_seq=0 out_seq=0 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3698117430 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595d7d85ef0 con 0x5595d7e41590 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2d57fa700 10 client.17565 handle_command_reply: tid=0 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 client.17565 shutdown 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 msgr2=0x5595d7e30f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 0x5595d7e30f00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0cceb0 tx=0x7fe2dc09b840 comp rx=0 tx=0).stop 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 shutdown_connections 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e41590 0x5595d7e43980 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e27cf0 0x5595d7e33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e2dec0 0x5595d7e31440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2d57fa700 0 client.17565 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:32.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3698117430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7e2d5a0 0x5595d7e30f00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 >> 172.21.15.87:0/3698117430 conn(0x5595d7d54220 msgr2=0x5595d7e21c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:32.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 shutdown_connections 2024-06-06T08:02:32.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3698117430 wait complete. 2024-06-06T08:02:32.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.599+0000 7fe2fea4bb80 20 client.17565 trim_cache size 0 max 16384 2024-06-06T08:02:32.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 Processor -- start 2024-06-06T08:02:32.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 -- start start 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e2e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b0fd0 con 0x5595d7d54220 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1ae800 con 0x5595d7e2d5a0 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.622+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1ac670 con 0x5595d7e27cf0 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35648/0 (socket says 172.21.15.87:35648) 2024-06-06T08:02:32.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1f64700 1 -- 172.21.15.87:0/2454475836 learned_addr learned my addr 172.21.15.87:0/2454475836 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e2e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f2765700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 -- 172.21.15.87:0/2454475836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 msgr2=0x5595d7e223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 -- 172.21.15.87:0/2454475836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 msgr2=0x5595d7e277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 -- 172.21.15.87:0/2454475836 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d7e30cd0 con 0x5595d7e27cf0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2f1763700 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e2e9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e970 tx=0x7fe2dc010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc85ddcb7f35147c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2d77fe700 1 -- 172.21.15.87:0/2454475836 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0a4660 con 0x5595d7e27cf0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2d77fe700 1 -- 172.21.15.87:0/2454475836 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc062030 con 0x5595d7e27cf0 2024-06-06T08:02:32.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.623+0000 7fe2d77fe700 1 -- 172.21.15.87:0/2454475836 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc09bee0 con 0x5595d7e27cf0 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/2454475836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 msgr2=0x5595d7e2e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e2e9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e970 tx=0x7fe2dc010bb0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/2454475836 shutdown_connections 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/2454475836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e2e9d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/2454475836 >> 172.21.15.87:0/2454475836 conn(0x5595d7e315e0 msgr2=0x5595d7e32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/2454475836 shutdown_connections 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/2454475836 wait complete. 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 -- start start 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.624+0000 7fe2fea4bb80 1 Processor -- start 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e36ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1accc0 con 0x5595d7d54220 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1ae130 con 0x5595d7e2d5a0 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1ad390 con 0x5595d7e27cf0 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2fea4bb80 10 client.0 _ll_register_callbacks cb 0x5595d7d0e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1763700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f2765700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e36ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35662/0 (socket says 172.21.15.87:35662) 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 -- 172.21.15.87:0/3846182697 learned_addr learned my addr 172.21.15.87:0/3846182697 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 -- 172.21.15.87:0/3846182697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 msgr2=0x5595d7e3a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e3a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 -- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 msgr2=0x5595d7e36ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e36ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.625+0000 7fe2f1f64700 1 -- 172.21.15.87:0/3846182697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d7e372e0 con 0x5595d7d54220 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2f1f64700 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800f930 tx=0x7fe2e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af720d1d13b06289 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8038060 con 0x5595d7d54220 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7fe2e8033680 con 0x5595d7d54220 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc062030 con 0x5595d7d54220 2024-06-06T08:02:32.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe2e803f050 con 0x5595d7d54220 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2fea4bb80 10 client.17571 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5595d7eaccc0 con 0x5595d7d54220 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.626+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e8033d60 con 0x5595d7d54220 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fe2e80119a0 con 0x5595d7d54220 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 10 client.17571 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 10 client.17571 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 20 client.17571 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e2dec0 0x5595d7eaf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 4 client.17571 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x7fe2dc062030 con 0x5595d7e2dec0 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2f1763700 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e2dec0 0x5595d7eaf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2d57fa700 10 client.17571 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:32.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:32.627+0000 7fe2f1763700 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e2dec0 0x5595d7eaf480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdaa13ac7b3e58d0 server_cookie=fc4d4bfc7c41154 in_seq=0 out_seq=0 2024-06-06T08:02:33.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:32 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:02:33.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:32 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:02:33.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:32 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:02:33.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11526 22.74 MB/sec warmup 35 sec latency 1001.321 ms 2024-06-06T08:02:33.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:32 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:02:33.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:32 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:02:33.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:32 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:02:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:32 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:02:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:32 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:02:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:32 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:02:33.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2d57fa700 1 -- 172.21.15.87:0/3846182697 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe2dc062030 con 0x5595d7e2dec0 2024-06-06T08:02:33.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2d57fa700 10 client.17571 handle_command_reply: tid=0 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 client.17571 shutdown 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 msgr2=0x5595d7e39ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800f930 tx=0x7fe2e8004750 comp rx=0 tx=0).stop 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 shutdown_connections 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5595d7e2dec0 0x5595d7eaf480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595d7e2d5a0 0x5595d7e36ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5595d7d54220 0x5595d7e39ce0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 --2- 172.21.15.87:0/3846182697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595d7e27cf0 0x5595d7e3a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2d57fa700 0 client.17571 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 >> 172.21.15.87:0/3846182697 conn(0x5595d7e315e0 msgr2=0x5595d7e32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 shutdown_connections 2024-06-06T08:02:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.650+0000 7fe2fea4bb80 1 -- 172.21.15.87:0/3846182697 wait complete. 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2fea4bb80 20 client.17571 trim_cache size 0 max 16384 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 46, 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 42, 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "ce4f972e": { 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 42, 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:02:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 msgr2=0x7fe2d80a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 0x7fe2d80a52f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec078960 tx=0x7fe2e8009260 comp rx=0 tx=0).stop 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 msgr2=0x7fe2ec19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e9a0 tx=0x7fe2dc00ecb0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 shutdown_connections 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec108520 0x7fe2ec198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2d80a2e30 0x7fe2d80a52f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec06aa60 0x7fe2ec19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 --2- 172.21.15.87:0/2533312239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2ec06b420 0x7fe2ec19ec50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 >> 172.21.15.87:0/2533312239 conn(0x7fe2ec0783d0 msgr2=0x7fe2ec106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 shutdown_connections 2024-06-06T08:02:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:33.651+0000 7fe2f41c8700 1 -- 172.21.15.87:0/2533312239 wait complete. 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 -- 172.21.15.87:0/1937901834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 msgr2=0x7f760810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 --2- 172.21.15.87:0/1937901834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760810e400 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7600009ca0 tx=0x7f760002a950 comp rx=0 tx=0).stop 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 -- 172.21.15.87:0/1937901834 shutdown_connections 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 --2- 172.21.15.87:0/1937901834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760810e400 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 --2- 172.21.15.87:0/1937901834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 0x7f760810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 --2- 172.21.15.87:0/1937901834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 -- 172.21.15.87:0/1937901834 >> 172.21.15.87:0/1937901834 conn(0x7f7608077a20 msgr2=0x7f7608077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 -- 172.21.15.87:0/1937901834 shutdown_connections 2024-06-06T08:02:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 -- 172.21.15.87:0/1937901834 wait complete. 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.837+0000 7f760eae1640 1 Processor -- start 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 -- start start 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 0x7f7608198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7608111d60 con 0x7f760810bf90 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7608111be0 con 0x7f7608103570 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760eae1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7608111ee0 con 0x7f7608102c50 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760c87d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760c87d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:53452/0 (socket says 172.21.15.87:53452) 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35678/0 (socket says 172.21.15.87:35678) 2024-06-06T08:02:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760c87d640 1 -- 172.21.15.87:0/2326672454 learned_addr learned my addr 172.21.15.87:0/2326672454 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f7607fff640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 0x7f7608198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 -- 172.21.15.87:0/2326672454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 msgr2=0x7f7608198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 -- 172.21.15.87:0/2326672454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 msgr2=0x7f7608198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 0x7f7608198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 -- 172.21.15.87:0/2326672454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760819d130 con 0x7f760810bf90 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f760d07e640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7600005cc0 tx=0x7f760002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.838+0000 7f7605ffb640 1 -- 172.21.15.87:0/2326672454 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600034040 con 0x7f760810bf90 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.839+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760819d3c0 con 0x7f760810bf90 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.839+0000 7f7605ffb640 1 -- 172.21.15.87:0/2326672454 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7600030920 con 0x7f760810bf90 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.839+0000 7f7605ffb640 1 -- 172.21.15.87:0/2326672454 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600004750 con 0x7f760810bf90 2024-06-06T08:02:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.839+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a4cb0 con 0x7f760810bf90 2024-06-06T08:02:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7605ffb640 1 -- 172.21.15.87:0/2326672454 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7600037020 con 0x7f760810bf90 2024-06-06T08:02:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 Processor -- start 2024-06-06T08:02:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 -- start start 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f4c350 0x565429f4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x565429ea54f0 con 0x565429f41170 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565429e43870 con 0x565429f4c350 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f76117f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565429ea14b0 con 0x565429f41a90 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35686/0 (socket says 172.21.15.87:35686) 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 -- 172.21.15.87:0/4187536406 learned_addr learned my addr 172.21.15.87:0/4187536406 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f760c87d640 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7605ffb640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 0x7f75f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 -- 172.21.15.87:0/4187536406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 msgr2=0x565429f4be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 -- 172.21.15.87:0/4187536406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f4c350 msgr2=0x565429f4e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f4c350 0x565429f4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f7607fff640 1 -- 172.21.15.87:0/4187536406 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565429e5f9b0 con 0x565429f41170 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.842+0000 7f760d07e640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 0x7f75f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f7605ffb640 1 -- 172.21.15.87:0/2326672454 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f76000f0510 con 0x7f760810bf90 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f7607fff640 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0058a0 tx=0x7f75fc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6518675bb5f362e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f75eeffd640 1 -- 172.21.15.87:0/4187536406 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0120d0 con 0x565429f41170 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f75eeffd640 1 -- 172.21.15.87:0/4187536406 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75fc016070 con 0x565429f41170 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f75eeffd640 1 -- 172.21.15.87:0/4187536406 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0107b0 con 0x565429f41170 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 -- 172.21.15.87:0/4187536406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 msgr2=0x565429f41550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f760d07e640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 0x7f75f00a5260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76000edaa0 tx=0x7f76000a72f0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0058a0 tx=0x7f75fc0127c0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 -- 172.21.15.87:0/4187536406 shutdown_connections 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f4c350 0x565429f4e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 --2- 172.21.15.87:0/4187536406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41170 0x565429f41550 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 -- 172.21.15.87:0/4187536406 >> 172.21.15.87:0/4187536406 conn(0x565429f3d680 msgr2=0x565429f3da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.843+0000 7f76117f3740 1 -- 172.21.15.87:0/4187536406 shutdown_connections 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 -- 172.21.15.87:0/4187536406 wait complete. 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 -- start start 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 Processor -- start 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41170 0x565429f492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x565429f4ef80 con 0x565429f4c350 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565429e2eae0 con 0x565429f41a90 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565429f4f100 con 0x565429f41170 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f76117f3740 10 client.0 _ll_register_callbacks cb 0x565429e679b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f7607fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f760c87d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f7607fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41170 0x565429f492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f760c87d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:55062/0 (socket says 172.21.15.87:55062) 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.844+0000 7f760c87d640 1 -- 172.21.15.87:0/1986382948 learned_addr learned my addr 172.21.15.87:0/1986382948 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 -- 172.21.15.87:0/1986382948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41170 msgr2=0x565429f492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41170 0x565429f492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 -- 172.21.15.87:0/1986382948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 msgr2=0x565429f49d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f49d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 -- 172.21.15.87:0/1986382948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565429bb9210 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f760c87d640 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75f800bb30 tx=0x7f75f80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe5a7469d01228 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f75e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8005020 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75fc016070 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f75f80051c0 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f75f8024050 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f76117f3740 10 client.26627 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565429c85620 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.845+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75fc016070 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f75f8011560 con 0x565429f41a90 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 10 client.26627 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 20 client.26627 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429f5c5a0 0x565429f5e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 4 client.26627 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565429eb8f10 con 0x565429f5c5a0 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f7607fff640 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429f5c5a0 0x565429f5e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f7607fff640 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429f5c5a0 0x565429f5e990 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab660166be49d3d server_cookie=6e4f6d6d07f621e7 in_seq=0 out_seq=0 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.846+0000 7f75e7fff640 10 client.26627 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f75e7fff640 1 -- 172.21.15.87:0/1986382948 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565429eb8f10 con 0x565429f5c5a0 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f75e7fff640 10 client.26627 handle_command_reply: tid=0 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 client.26627 shutdown 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 msgr2=0x565429f49800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75f800bb30 tx=0x7f75f80035f0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 shutdown_connections 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429f5c5a0 0x565429f5e990 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f49d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41a90 0x565429f49800 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f75e7fff640 0 client.26627 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 --2- 172.21.15.87:0/1986382948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41170 0x565429f492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 >> 172.21.15.87:0/1986382948 conn(0x565429f3d680 msgr2=0x565429f43400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 shutdown_connections 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 1 -- 172.21.15.87:0/1986382948 wait complete. 2024-06-06T08:02:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.847+0000 7f76117f3740 20 client.26627 trim_cache size 0 max 16384 2024-06-06T08:02:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.862+0000 7f76117f3740 1 Processor -- start 2024-06-06T08:02:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.862+0000 7f76117f3740 1 -- start start 2024-06-06T08:02:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 0x565429f642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b18a0 con 0x565429f41a90 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76081af110 con 0x565429f41170 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f76117f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f76081acf80 con 0x565429f4c350 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35708/0 (socket says 172.21.15.87:35708) 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 -- 172.21.15.87:0/4064387686 learned_addr learned my addr 172.21.15.87:0/4064387686 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f7607fff640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f49b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f7607fff640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 0x565429f642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 -- 172.21.15.87:0/4064387686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 msgr2=0x565429f642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 0x565429f642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 -- 172.21.15.87:0/4064387686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 msgr2=0x565429f49b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f49b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 -- 172.21.15.87:0/4064387686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565429f5f010 con 0x565429f41a90 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f7607fff640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 0x565429f642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f760c87d640 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000c00 tx=0x7f75f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d54a34fea0add84 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f75ee7fc640 1 -- 172.21.15.87:0/4064387686 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f802d030 con 0x565429f41a90 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.863+0000 7f75ee7fc640 1 -- 172.21.15.87:0/4064387686 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f800b5d0 con 0x565429f41a90 2024-06-06T08:02:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f75ee7fc640 1 -- 172.21.15.87:0/4064387686 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f80047b0 con 0x565429f41a90 2024-06-06T08:02:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 -- 172.21.15.87:0/4064387686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 msgr2=0x565429f3c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000c00 tx=0x7f75f8013000 comp rx=0 tx=0).stop 2024-06-06T08:02:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 -- 172.21.15.87:0/4064387686 shutdown_connections 2024-06-06T08:02:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f4c350 0x565429f642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f41a90 0x565429f3c660 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 --2- 172.21.15.87:0/4064387686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f49b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 -- 172.21.15.87:0/4064387686 >> 172.21.15.87:0/4064387686 conn(0x565429f3d680 msgr2=0x565429f3bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 -- 172.21.15.87:0/4064387686 shutdown_connections 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.864+0000 7f76117f3740 1 -- 172.21.15.87:0/4064387686 wait complete. 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 -- start start 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 Processor -- start 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76081ad5d0 con 0x565429f4c350 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76081aea40 con 0x565429f41170 2024-06-06T08:02:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f76081adca0 con 0x565429f41a90 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f76117f3740 10 client.0 _ll_register_callbacks cb 0x7f75f80041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f7607fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f7607fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35722/0 (socket says 172.21.15.87:35722) 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f7607fff640 1 -- 172.21.15.87:0/1538281815 learned_addr learned my addr 172.21.15.87:0/1538281815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.865+0000 7f760c87d640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f54a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 -- 172.21.15.87:0/1538281815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 msgr2=0x565429f4fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 -- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 msgr2=0x565429f54a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f54a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 -- 172.21.15.87:0/1538281815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565429f50a90 con 0x565429f4c350 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f760c87d640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:02:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f7607fff640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0058a0 tx=0x7f75fc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee61a0435d3ea4ab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc030040 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f75fc0050d0 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f800b5d0 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f75fc0052d0 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 10 client.17589 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565429fc7170 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f800b5d0 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f75fc036030 con 0x565429f4c350 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 10 client.17589 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 20 client.17589 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429fc7790 0x565429fc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 4 client.17589 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:02:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.866+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565429eb2c20 con 0x565429fc7790 2024-06-06T08:02:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.867+0000 7f760c87d640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429fc7790 0x565429fc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.867+0000 7f75e7fff640 10 client.17589 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:33.867+0000 7f760c87d640 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429fc7790 0x565429fc9b80 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0fbc0ed3c95efb server_cookie=7ce90a185b893cf4 in_seq=0 out_seq=0 2024-06-06T08:02:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:33 smithi087 ceph-mon[32276]: pgmap v460: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:02:34.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 22.11 MB/sec warmup 36 sec latency 1309.192 ms 2024-06-06T08:02:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:33 smithi195 ceph-mon[37040]: pgmap v460: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:02:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:33 smithi181 ceph-mon[36790]: pgmap v460: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:02:34.616 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ce4f972e', quiesce took 7.8 seconds. Will release after: 17.45 2024-06-06T08:02:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f75e7fff640 1 -- 172.21.15.87:0/1538281815 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565429eb2c20 con 0x565429fc7790 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f75e7fff640 10 client.17589 handle_command_reply: tid=0 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 client.17589 shutdown 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 msgr2=0x565429f502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0058a0 tx=0x7f75fc0127c0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 shutdown_connections 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x565429fc7790 0x565429fc9b80 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x565429f4c350 0x565429f502e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f75e7fff640 0 client.17589 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565429f41a90 0x565429f4fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 --2- 172.21.15.87:0/1538281815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565429f41170 0x565429f54a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.696+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 >> 172.21.15.87:0/1538281815 conn(0x565429f3d680 msgr2=0x565429f3c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.697+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 shutdown_connections 2024-06-06T08:02:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.697+0000 7f76117f3740 1 -- 172.21.15.87:0/1538281815 wait complete. 2024-06-06T08:02:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.697+0000 7f76117f3740 20 client.17589 trim_cache size 0 max 16384 2024-06-06T08:02:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:02:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:02:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:02:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 msgr2=0x7f75f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 0x7f75f00a5260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76000edaa0 tx=0x7f76000a72f0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 msgr2=0x7f760819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7600005cc0 tx=0x7f760002f630 comp rx=0 tx=0).stop 2024-06-06T08:02:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 shutdown_connections 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f75f00a2da0 0x7f75f00a5260 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f760810bf90 0x7f760819ca00 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608103570 0x7f7608198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 --2- 172.21.15.87:0/2326672454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7608102c50 0x7f7608198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 >> 172.21.15.87:0/2326672454 conn(0x7f7608077a20 msgr2=0x7f76080fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 shutdown_connections 2024-06-06T08:02:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:34.698+0000 7f760eae1640 1 -- 172.21.15.87:0/2326672454 wait complete. 2024-06-06T08:02:35.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 21.51 MB/sec warmup 37 sec latency 2309.300 ms 2024-06-06T08:02:36.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 20.94 MB/sec warmup 38 sec latency 3309.344 ms 2024-06-06T08:02:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:35 smithi195 ceph-mon[37040]: pgmap v461: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-06-06T08:02:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:35 smithi181 ceph-mon[36790]: pgmap v461: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-06-06T08:02:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:35 smithi087 ceph-mon[32276]: pgmap v461: 129 pgs: 129 active+clean; 303 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-06-06T08:02:37.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 20.41 MB/sec warmup 39 sec latency 4309.443 ms 2024-06-06T08:02:38.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 19.90 MB/sec warmup 40 sec latency 5309.535 ms 2024-06-06T08:02:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:37 smithi195 ceph-mon[37040]: pgmap v462: 129 pgs: 129 active+clean; 296 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-06T08:02:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:37 smithi181 ceph-mon[36790]: pgmap v462: 129 pgs: 129 active+clean; 296 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-06T08:02:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:37 smithi087 ceph-mon[32276]: pgmap v462: 129 pgs: 129 active+clean; 296 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-06T08:02:39.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 19.41 MB/sec warmup 41 sec latency 6309.627 ms 2024-06-06T08:02:40.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 18.95 MB/sec warmup 42 sec latency 7309.727 ms 2024-06-06T08:02:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:39 smithi195 ceph-mon[37040]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.9 MiB/s wr, 599 op/s 2024-06-06T08:02:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:39 smithi181 ceph-mon[36790]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.9 MiB/s wr, 599 op/s 2024-06-06T08:02:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:39 smithi087 ceph-mon[32276]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.9 MiB/s wr, 599 op/s 2024-06-06T08:02:41.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 18.51 MB/sec warmup 43 sec latency 8309.785 ms 2024-06-06T08:02:42.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 18.09 MB/sec warmup 44 sec latency 9309.913 ms 2024-06-06T08:02:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:41 smithi195 ceph-mon[37040]: pgmap v464: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-06-06T08:02:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:41 smithi181 ceph-mon[36790]: pgmap v464: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-06-06T08:02:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:41 smithi087 ceph-mon[32276]: pgmap v464: 129 pgs: 129 active+clean; 286 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 488 op/s 2024-06-06T08:02:43.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 17.69 MB/sec warmup 45 sec latency 10309.956 ms 2024-06-06T08:02:44.157 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 17.30 MB/sec warmup 46 sec latency 11310.060 ms 2024-06-06T08:02:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:43 smithi195 ceph-mon[37040]: pgmap v465: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-06-06T08:02:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:43 smithi181 ceph-mon[36790]: pgmap v465: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-06-06T08:02:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:43 smithi087 ceph-mon[32276]: pgmap v465: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-06-06T08:02:45.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 16.93 MB/sec warmup 47 sec latency 12310.144 ms 2024-06-06T08:02:46.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 16.58 MB/sec warmup 48 sec latency 13310.195 ms 2024-06-06T08:02:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:45 smithi195 ceph-mon[37040]: pgmap v466: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 5.8 MiB/s wr, 455 op/s 2024-06-06T08:02:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:45 smithi181 ceph-mon[36790]: pgmap v466: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 5.8 MiB/s wr, 455 op/s 2024-06-06T08:02:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:45 smithi087 ceph-mon[32276]: pgmap v466: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 5.8 MiB/s wr, 455 op/s 2024-06-06T08:02:47.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 16.24 MB/sec warmup 49 sec latency 14310.312 ms 2024-06-06T08:02:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/2392359886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f081079c0 msgr2=0x7f6f08107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/2392359886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f081079c0 0x7f6f08107da0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009930 tx=0x7f6ef802a420 comp rx=0 tx=0).stop 2024-06-06T08:02:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/2392359886 shutdown_connections 2024-06-06T08:02:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/2392359886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f08102370 0x7f6f0810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/2392359886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f08101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/2392359886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f081079c0 0x7f6f08107da0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/2392359886 >> 172.21.15.87:0/2392359886 conn(0x7f6f08078e30 msgr2=0x7f6f08079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/2392359886 shutdown_connections 2024-06-06T08:02:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.795+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/2392359886 wait complete. 2024-06-06T08:02:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 Processor -- start 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 -- start start 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f081981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f081079c0 0x7f6f0819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08111dd0 con 0x7f6f08102370 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08111c50 con 0x7f6f081019b0 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f0d4f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08111f50 con 0x7f6f081079c0 2024-06-06T08:02:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f067fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f067fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46792/0 (socket says 172.21.15.87:46792) 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f067fc640 1 -- 172.21.15.87:0/4056657714 learned_addr learned my addr 172.21.15.87:0/4056657714 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f077fe640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f081079c0 0x7f6f0819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.796+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f081981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 -- 172.21.15.87:0/4056657714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f081079c0 msgr2=0x7f6f0819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f081079c0 0x7f6f0819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 -- 172.21.15.87:0/4056657714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 msgr2=0x7f6f081981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f081981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 -- 172.21.15.87:0/4056657714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0819d1a0 con 0x7f6f08102370 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f081981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c890 tx=0x7f6efc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6eeffff640 1 -- 172.21.15.87:0/4056657714 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc004220 con 0x7f6f08102370 2024-06-06T08:02:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0819d490 con 0x7f6f08102370 2024-06-06T08:02:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6eeffff640 1 -- 172.21.15.87:0/4056657714 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc0043c0 con 0x7f6f08102370 2024-06-06T08:02:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6eeffff640 1 -- 172.21.15.87:0/4056657714 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc004a00 con 0x7f6f08102370 2024-06-06T08:02:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.797+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a4d20 con 0x7f6f08102370 2024-06-06T08:02:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.799+0000 7f6f101ff740 1 Processor -- start 2024-06-06T08:02:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 -- start start 2024-06-06T08:02:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6eeffff640 1 -- 172.21.15.87:0/4056657714 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6efc0079b0 con 0x7f6f08102370 2024-06-06T08:02:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a040550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a04d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561d29fa44f0 con 0x561d2a040a90 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d29f42870 con 0x561d2a04b350 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f101ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d29fa04b0 con 0x561d2a040170 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46794/0 (socket says 172.21.15.87:46794) 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 -- 172.21.15.87:0/1592818106 learned_addr learned my addr 172.21.15.87:0/1592818106 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f077fe640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a040550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f067fc640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a04d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 -- 172.21.15.87:0/1592818106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 msgr2=0x561d2a040550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a040550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 -- 172.21.15.87:0/1592818106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 msgr2=0x561d2a04d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a04d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 -- 172.21.15.87:0/1592818106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d29f5e9b0 con 0x561d2a040a90 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f067fc640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a04d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.800+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009930 tx=0x7f6ef8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3202b1c4d5bfe88b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6eeffff640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 0x7f6ef00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6f077fe640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 0x7f6ef00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6eed7fa640 1 -- 172.21.15.87:0/1592818106 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef803f070 con 0x561d2a040a90 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6eed7fa640 1 -- 172.21.15.87:0/1592818106 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef80317b0 con 0x561d2a040a90 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6eed7fa640 1 -- 172.21.15.87:0/1592818106 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8035070 con 0x561d2a040a90 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6eeffff640 1 -- 172.21.15.87:0/4056657714 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f6efc0cf570 con 0x7f6f08102370 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6f101ff740 1 -- 172.21.15.87:0/1592818106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 msgr2=0x561d2a04ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6f077fe640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 0x7f6ef00a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005fd0 tx=0x7f6f00009550 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.801+0000 7f6f101ff740 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009930 tx=0x7f6ef8032d60 comp rx=0 tx=0).stop 2024-06-06T08:02:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- 172.21.15.87:0/1592818106 shutdown_connections 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a04d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a04ae10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- 172.21.15.87:0/1592818106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a040550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- 172.21.15.87:0/1592818106 >> 172.21.15.87:0/1592818106 conn(0x561d2a03c680 msgr2=0x561d2a03ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- 172.21.15.87:0/1592818106 shutdown_connections 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- 172.21.15.87:0/1592818106 wait complete. 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- start start 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 Processor -- start 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040a90 0x561d2a048820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a048d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561d2a04df20 con 0x561d2a040170 2024-06-06T08:02:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d29f2dae0 con 0x561d2a04b350 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d2a04e0a0 con 0x561d2a040a90 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f101ff740 10 client.0 _ll_register_callbacks cb 0x561d29f669b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f06ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46796/0 (socket says 172.21.15.87:46796) 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f06ffd640 1 -- 172.21.15.87:0/4016984227 learned_addr learned my addr 172.21.15.87:0/4016984227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.802+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040a90 0x561d2a048820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f077fe640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a048d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 -- 172.21.15.87:0/4016984227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040a90 msgr2=0x561d2a048820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040a90 0x561d2a048820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 -- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 msgr2=0x561d2a048d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a048d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 -- 172.21.15.87:0/4016984227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d29cb8210 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009900 tx=0x7f6ef8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b14c32cb3e131e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6eeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef804a020 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f6ef80035f0 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef80317b0 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6ef8055050 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f101ff740 10 client.17601 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561d29d84620 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.803+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef80317b0 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f6ef8004b50 con 0x561d2a040170 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 10 client.17601 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 20 client.17601 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a05b270 0x561d2a05d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 4 client.17601 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d29fb7f10 con 0x561d2a05b270 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a05b270 0x561d2a05d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6eeaff5640 10 client.17601 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.804+0000 7f6f067fc640 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a05b270 0x561d2a05d660 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fac918d1664159 server_cookie=6c02ba4e1e217b95 in_seq=0 out_seq=0 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6eeaff5640 1 -- 172.21.15.87:0/4016984227 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561d29fb7f10 con 0x561d2a05b270 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6eeaff5640 10 client.17601 handle_command_reply: tid=0 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 client.17601 shutdown 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 msgr2=0x561d2a0482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009900 tx=0x7f6ef8009250 comp rx=0 tx=0).stop 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 shutdown_connections 2024-06-06T08:02:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a05b270 0x561d2a05d660 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a048d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6eeaff5640 0 client.17601 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040a90 0x561d2a048820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 --2- 172.21.15.87:0/4016984227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040170 0x561d2a0482e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 >> 172.21.15.87:0/4016984227 conn(0x561d2a03c680 msgr2=0x561d2a0423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 shutdown_connections 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.805+0000 7f6f101ff740 1 -- 172.21.15.87:0/4016984227 wait complete. 2024-06-06T08:02:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.806+0000 7f6f101ff740 20 client.17601 trim_cache size 0 max 16384 2024-06-06T08:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 Processor -- start 2024-06-06T08:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 -- start start 2024-06-06T08:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a062cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081b1850 con 0x561d2a040a90 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081af0c0 con 0x561d2a04b350 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f101ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081acf30 con 0x561d2a040170 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46808/0 (socket says 172.21.15.87:46808) 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 -- 172.21.15.87:0/2254554864 learned_addr learned my addr 172.21.15.87:0/2254554864 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 -- 172.21.15.87:0/2254554864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 msgr2=0x561d2a0413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 -- 172.21.15.87:0/2254554864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 msgr2=0x561d2a062cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a062cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 -- 172.21.15.87:0/2254554864 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2a03cfa0 con 0x561d2a040a90 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.821+0000 7f6f067fc640 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6efc09b600 tx=0x7f6efc0054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7976209e8f13495 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6eecff9640 1 -- 172.21.15.87:0/2254554864 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc01d040 con 0x561d2a040a90 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6eecff9640 1 -- 172.21.15.87:0/2254554864 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc09bd70 con 0x561d2a040a90 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6eecff9640 1 -- 172.21.15.87:0/2254554864 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc099a40 con 0x561d2a040a90 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- 172.21.15.87:0/2254554864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 msgr2=0x561d2a03ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6efc09b600 tx=0x7f6efc0054a0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- 172.21.15.87:0/2254554864 shutdown_connections 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a04b350 0x561d2a062cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a040a90 0x561d2a03ca60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2254554864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- 172.21.15.87:0/2254554864 >> 172.21.15.87:0/2254554864 conn(0x561d2a03c680 msgr2=0x561d2a047b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- 172.21.15.87:0/2254554864 shutdown_connections 2024-06-06T08:02:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- 172.21.15.87:0/2254554864 wait complete. 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.822+0000 7f6f101ff740 1 -- start start 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 Processor -- start 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 0x561d2a0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081ad580 con 0x561d2a04b350 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081ae9f0 con 0x561d2a040a90 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081adc50 con 0x561d2a040170 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f101ff740 10 client.0 _ll_register_callbacks cb 0x7f6f000099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46818/0 (socket says 172.21.15.87:46818) 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 -- 172.21.15.87:0/2330061617 learned_addr learned my addr 172.21.15.87:0/2330061617 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f067fc640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 0x561d2a0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 -- 172.21.15.87:0/2330061617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 msgr2=0x561d2a0c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 -- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 msgr2=0x561d2a0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 0x561d2a0c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 -- 172.21.15.87:0/2330061617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2a062e50 con 0x561d2a04b350 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f067fc640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 0x561d2a0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6f077fe640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6f000042b0 tx=0x7f6f00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175506d6c11458bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.823+0000 7f6eeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00011c00 con 0x561d2a04b350 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f6f00011da0 con 0x561d2a04b350 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc09bd70 con 0x561d2a04b350 2024-06-06T08:02:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6f00033050 con 0x561d2a04b350 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 10 client.17607 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561d2a0c9ca0 con 0x561d2a04b350 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc09bd70 con 0x561d2a04b350 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f6f0000f040 con 0x561d2a04b350 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 10 client.17607 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 20 client.17607 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a0ca4b0 0x561d2a0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 4 client.17607 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d29fb1c20 con 0x561d2a0ca4b0 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.824+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a0ca4b0 0x561d2a0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.825+0000 7f6eeaff5640 10 client.17607 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.825+0000 7f6f06ffd640 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a0ca4b0 0x561d2a0cc8d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba42b3d2794f7e98 server_cookie=94a68db5b2ab7ca2 in_seq=0 out_seq=0 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6eeaff5640 1 -- 172.21.15.87:0/2330061617 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561d29fb1c20 con 0x561d2a0ca4b0 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6eeaff5640 10 client.17607 handle_command_reply: tid=0 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 client.17607 shutdown 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 msgr2=0x561d2a062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6f000042b0 tx=0x7f6f00004ab0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 shutdown_connections 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561d2a0ca4b0 0x561d2a0cc8d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561d2a04b350 0x561d2a062710 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6eeaff5640 0 client.17607 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d2a040a90 0x561d2a0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 --2- 172.21.15.87:0/2330061617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2a040170 0x561d2a0c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 >> 172.21.15.87:0/2330061617 conn(0x561d2a03c680 msgr2=0x561d2a03b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 shutdown_connections 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.930+0000 7f6f101ff740 1 -- 172.21.15.87:0/2330061617 wait complete. 2024-06-06T08:02:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f101ff740 20 client.17607 trim_cache size 0 max 16384 2024-06-06T08:02:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:02:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:02:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 msgr2=0x7f6ef00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 0x7f6ef00a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005fd0 tx=0x7f6f00009550 comp rx=0 tx=0).stop 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 msgr2=0x7f6f081986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c890 tx=0x7f6efc00cd50 comp rx=0 tx=0).stop 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 shutdown_connections 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ef00a2da0 0x7f6ef00a5260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f081079c0 0x7f6f0819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f08102370 0x7f6f081986f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 --2- 172.21.15.87:0/4056657714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f081019b0 0x7f6f081981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 >> 172.21.15.87:0/4056657714 conn(0x7f6f08078e30 msgr2=0x7f6f080ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 shutdown_connections 2024-06-06T08:02:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:02:47.931+0000 7f6f0d4f5640 1 -- 172.21.15.87:0/4056657714 wait complete. 2024-06-06T08:02:48.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 15.92 MB/sec warmup 50 sec latency 15310.438 ms 2024-06-06T08:02:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:47 smithi195 ceph-mon[37040]: pgmap v467: 129 pgs: 129 active+clean; 337 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 7.0 MiB/s wr, 523 op/s 2024-06-06T08:02:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:47 smithi181 ceph-mon[36790]: pgmap v467: 129 pgs: 129 active+clean; 337 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 7.0 MiB/s wr, 523 op/s 2024-06-06T08:02:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:47 smithi087 ceph-mon[32276]: pgmap v467: 129 pgs: 129 active+clean; 337 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 7.0 MiB/s wr, 523 op/s 2024-06-06T08:02:49.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 15.61 MB/sec warmup 51 sec latency 16310.501 ms 2024-06-06T08:02:50.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 15.31 MB/sec warmup 52 sec latency 17310.636 ms 2024-06-06T08:02:50.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:49 smithi087 ceph-mon[32276]: pgmap v468: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-06T08:02:50.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:49 smithi195 ceph-mon[37040]: pgmap v468: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-06T08:02:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:49 smithi181 ceph-mon[36790]: pgmap v468: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 8.7 MiB/s wr, 579 op/s 2024-06-06T08:02:51.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 15.02 MB/sec warmup 53 sec latency 18310.685 ms 2024-06-06T08:02:52.067 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:02:52.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 14.74 MB/sec warmup 54 sec latency 19310.751 ms 2024-06-06T08:02:52.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:51 smithi195 ceph-mon[37040]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 8.0 MiB/s wr, 340 op/s 2024-06-06T08:02:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:51 smithi181 ceph-mon[36790]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 8.0 MiB/s wr, 340 op/s 2024-06-06T08:02:52.318 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:02:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:51 smithi087 ceph-mon[32276]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 8.0 MiB/s wr, 340 op/s 2024-06-06T08:02:53.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 14.47 MB/sec warmup 55 sec latency 20310.860 ms 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 -- 172.21.15.87:0/404077481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 msgr2=0x7fc79c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/404077481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c06b0e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc7900099c0 tx=0x7fc790009cd0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 -- 172.21.15.87:0/404077481 shutdown_connections 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/404077481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/404077481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 0x7fc79c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/404077481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c06b0e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 -- 172.21.15.87:0/404077481 >> 172.21.15.87:0/404077481 conn(0x7fc79c078430 msgr2=0x7fc79c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 -- 172.21.15.87:0/404077481 shutdown_connections 2024-06-06T08:02:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.965+0000 7fc7a0c33700 1 -- 172.21.15.87:0/404077481 wait complete. 2024-06-06T08:02:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 Processor -- start 2024-06-06T08:02:53.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 -- start start 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 0x7fc79c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c112880 con 0x7fc79c06ac60 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c112700 con 0x7fc79c108580 2024-06-06T08:02:53.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc7a0c33700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c112a00 con 0x7fc79c06b620 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46836/0 (socket says 172.21.15.87:46836) 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 -- 172.21.15.87:0/4225209932 learned_addr learned my addr 172.21.15.87:0/4225209932 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc799d9b700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 0x7fc79c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79ad9d700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:53.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 -- 172.21.15.87:0/4225209932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 msgr2=0x7fc79c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 0x7fc79c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 -- 172.21.15.87:0/4225209932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 msgr2=0x7fc79c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79a59c700 1 -- 172.21.15.87:0/4225209932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c1991c0 con 0x7fc79c06ac60 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.966+0000 7fc79ad9d700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:53.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc79a59c700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0060b0 tx=0x7fc78c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c01a070 con 0x7fc79c06ac60 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78c00ddd0 con 0x7fc79c06ac60 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c1994b0 con 0x7fc79c06ac60 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0147a0 con 0x7fc79c06ac60 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.967+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a57f0 con 0x7fc79c06ac60 2024-06-06T08:02:53.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.968+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c069180 con 0x7fc79c06ac60 2024-06-06T08:02:53.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.969+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc78c01d070 con 0x7fc79c06ac60 2024-06-06T08:02:53.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.969+0000 7fc78b7fe700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 0x7fc7840a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:53.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.970+0000 7fc799d9b700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 0x7fc7840a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.970+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7fc78c0cf100 con 0x7fc79c06ac60 2024-06-06T08:02:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.970+0000 7fc799d9b700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 0x7fc7840a52d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc79c078950 tx=0x7fc790005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:53.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:53.974+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc78c010070 con 0x7fc79c06ac60 2024-06-06T08:02:53.977 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:53 smithi087 ceph-mon[32276]: pgmap v470: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 8.1 MiB/s wr, 384 op/s 2024-06-06T08:02:54.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 14.21 MB/sec warmup 56 sec latency 21310.913 ms 2024-06-06T08:02:54.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.215+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc79c02dca0 con 0x7fc79c06ac60 2024-06-06T08:02:54.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.216+0000 7fc78b7fe700 1 -- 172.21.15.87:0/4225209932 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11881 (secure 0 0 0) 0x7fc78c09b900 con 0x7fc79c06ac60 2024-06-06T08:02:54.215 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:02:54.216 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:02:00.625810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26623":{"gid":26623,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1019415876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1019415876},{"type":"v1","addr":"172.21.15.195:6837","nonce":1019415876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:02:54.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 msgr2=0x7fc7840a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:54.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 0x7fc7840a52d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc79c078950 tx=0x7fc790005fb0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 msgr2=0x7fc79c19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0060b0 tx=0x7fc78c00d3b0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 shutdown_connections 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c108580 0x7fc79c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc7840a2e10 0x7fc7840a52d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc79c06ac60 0x7fc79c19e8a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 --2- 172.21.15.87:0/4225209932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79c06b620 0x7fc79c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 >> 172.21.15.87:0/4225209932 conn(0x7fc79c078430 msgr2=0x7fc79c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 shutdown_connections 2024-06-06T08:02:54.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:54.218+0000 7fc7a0c33700 1 -- 172.21.15.87:0/4225209932 wait complete. 2024-06-06T08:02:54.219 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 46 2024-06-06T08:02:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:53 smithi195 ceph-mon[37040]: pgmap v470: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 8.1 MiB/s wr, 384 op/s 2024-06-06T08:02:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:53 smithi181 ceph-mon[36790]: pgmap v470: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 8.1 MiB/s wr, 384 op/s 2024-06-06T08:02:55.046 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id ce4f972e --release --await' 2024-06-06T08:02:55.046 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id ce4f972e --release --await 2024-06-06T08:02:55.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 13.96 MB/sec warmup 57 sec latency 22310.968 ms 2024-06-06T08:02:55.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:54 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4225209932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:55.283 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:02:55.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:54 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4225209932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:55.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:54 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4225209932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:02:56.092 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:55 smithi087 ceph-mon[32276]: pgmap v471: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-06-06T08:02:56.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11540 13.72 MB/sec warmup 58 sec latency 23311.005 ms 2024-06-06T08:02:56.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:55 smithi195 ceph-mon[37040]: pgmap v471: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-06-06T08:02:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:55 smithi181 ceph-mon[36790]: pgmap v471: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.971+0000 7f976e3a7700 1 -- 172.21.15.87:0/3119891574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101380 msgr2=0x7f9768101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.971+0000 7f976e3a7700 1 --2- 172.21.15.87:0/3119891574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101380 0x7f9768101800 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9750009990 tx=0x7f9750009ca0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 -- 172.21.15.87:0/3119891574 shutdown_connections 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 --2- 172.21.15.87:0/3119891574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101d40 0x7f976810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 --2- 172.21.15.87:0/3119891574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 0x7f97681077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 --2- 172.21.15.87:0/3119891574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101380 0x7f9768101800 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 -- 172.21.15.87:0/3119891574 >> 172.21.15.87:0/3119891574 conn(0x7f9768078300 msgr2=0x7f9768078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 -- 172.21.15.87:0/3119891574 shutdown_connections 2024-06-06T08:02:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 -- 172.21.15.87:0/3119891574 wait complete. 2024-06-06T08:02:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 Processor -- start 2024-06-06T08:02:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 -- start start 2024-06-06T08:02:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.972+0000 7f976e3a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101380 0x7f976819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 0x7f9768198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9768111700 con 0x7f9768101d40 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9768111580 con 0x7f9768101380 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9768111880 con 0x7f97681073f0 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f9767fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101380 0x7f976819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54974/0 (socket says 172.21.15.87:54974) 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 -- 172.21.15.87:0/4216408183 learned_addr learned my addr 172.21.15.87:0/4216408183 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976c944700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 0x7f9768198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 -- 172.21.15.87:0/4216408183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 msgr2=0x7f9768198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 0x7f9768198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 -- 172.21.15.87:0/4216408183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101380 msgr2=0x7f976819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101380 0x7f976819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 -- 172.21.15.87:0/4216408183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97681991e0 con 0x7f9768101d40 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f975ffff700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f9765ffb700 1 -- 172.21.15.87:0/4216408183 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750027070 con 0x7f9768101d40 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f9765ffb700 1 -- 172.21.15.87:0/4216408183 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750003e40 con 0x7f9768101d40 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9768199470 con 0x7f9768101d40 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f9765ffb700 1 -- 172.21.15.87:0/4216408183 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750022d10 con 0x7f9768101d40 2024-06-06T08:02:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.973+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a5790 con 0x7f9768101d40 2024-06-06T08:02:56.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.976+0000 7f9765ffb700 1 -- 172.21.15.87:0/4216408183 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f975002a070 con 0x7f9768101d40 2024-06-06T08:02:56.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.976+0000 7f9765ffb700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 0x7f97540a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.976+0000 7f9767fff700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 0x7f97540a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.976+0000 7f9765ffb700 1 -- 172.21.15.87:0/4216408183 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f97500d05b0 con 0x7f9768101d40 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.977+0000 7f9767fff700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 0x7f97540a52d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9768199bc0 tx=0x7f975800b3c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.977+0000 7f9778c2ab80 1 Processor -- start 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.977+0000 7f9778c2ab80 1 -- start start 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 0x55fa9b718710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b719070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b60b060 con 0x55fa9b71e500 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b669950 con 0x55fa9b718c50 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b5d1560 con 0x55fa9b71ee20 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54990/0 (socket says 172.21.15.87:54990) 2024-06-06T08:02:56.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 -- 172.21.15.87:0/953581494 learned_addr learned my addr 172.21.15.87:0/953581494 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f975ffff700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 0x55fa9b718710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f976c944700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b719070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 -- 172.21.15.87:0/953581494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 msgr2=0x55fa9b718710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 0x55fa9b718710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 -- 172.21.15.87:0/953581494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 msgr2=0x55fa9b719070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b719070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 -- 172.21.15.87:0/953581494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9b5b89d0 con 0x55fa9b71e500 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f975ffff700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 0x55fa9b718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f9767fff700 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9758005d70 tx=0x7f9758013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acd393a439bd78e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.978+0000 7f975e7fc700 1 -- 172.21.15.87:0/953581494 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758011ad0 con 0x55fa9b71e500 2024-06-06T08:02:56.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f975e7fc700 1 -- 172.21.15.87:0/953581494 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758003e90 con 0x55fa9b71e500 2024-06-06T08:02:56.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f975e7fc700 1 -- 172.21.15.87:0/953581494 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975801c610 con 0x55fa9b71e500 2024-06-06T08:02:56.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 -- 172.21.15.87:0/953581494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 msgr2=0x55fa9b71e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9758005d70 tx=0x7f9758013000 comp rx=0 tx=0).stop 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 -- 172.21.15.87:0/953581494 shutdown_connections 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b719070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71e500 0x55fa9b71e8e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/953581494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71ee20 0x55fa9b718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 -- 172.21.15.87:0/953581494 >> 172.21.15.87:0/953581494 conn(0x55fa9b645180 msgr2=0x55fa9b713a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.979+0000 7f9778c2ab80 1 -- 172.21.15.87:0/953581494 shutdown_connections 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 -- 172.21.15.87:0/953581494 wait complete. 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 -- start start 2024-06-06T08:02:56.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 Processor -- start 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b724ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b721dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b60a0c0 con 0x55fa9b71ee20 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b665910 con 0x55fa9b718c50 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa9b725800 con 0x55fa9b71e500 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f9778c2ab80 10 client.0 _ll_register_callbacks cb 0x55fa9b5ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.980+0000 7f976c944700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54992/0 (socket says 172.21.15.87:54992) 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 -- 172.21.15.87:0/2652280892 learned_addr learned my addr 172.21.15.87:0/2652280892 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:56.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f9767fff700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b724ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f975ffff700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b721dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 -- 172.21.15.87:0/2652280892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 msgr2=0x55fa9b721dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b721dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 -- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 msgr2=0x55fa9b724ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b724ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 -- 172.21.15.87:0/2652280892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9b62f850 con 0x55fa9b71ee20 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f9767fff700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b724ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f976c944700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f976000aaf0 tx=0x7f976000ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e625749bb1b5cd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f97477fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760007860 con 0x55fa9b71ee20 2024-06-06T08:02:56.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f9760007a00 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9758003e90 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9760007c40 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f9778c2ab80 10 client.17625 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.981+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fa9b72ff60 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758003e90 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f976001f420 con 0x55fa9b71ee20 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 10 client.17625 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 10 client.17625 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 20 client.17625 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b7325f0 0x55fa9b734a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:56.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 4 client.17625 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9b676e50 con 0x55fa9b7325f0 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.982+0000 7f9767fff700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b7325f0 0x55fa9b734a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.983+0000 7f9767fff700 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b7325f0 0x55fa9b734a20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47845c4367b8534 server_cookie=78537d60814c6310 in_seq=0 out_seq=0 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.983+0000 7f97477fe700 10 client.17625 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.983+0000 7f97477fe700 1 -- 172.21.15.87:0/2652280892 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa9b676e50 con 0x55fa9b7325f0 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.983+0000 7f97477fe700 10 client.17625 handle_command_reply: tid=0 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.983+0000 7f9778c2ab80 1 client.17625 shutdown 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 msgr2=0x55fa9b722310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f976000aaf0 tx=0x7f976000ab20 comp rx=0 tx=0).stop 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 shutdown_connections 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b7325f0 0x55fa9b734a20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b718c50 0x55fa9b724ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b71ee20 0x55fa9b722310 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/2652280892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b721dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 >> 172.21.15.87:0/2652280892 conn(0x55fa9b645180 msgr2=0x55fa9b712320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f97477fe700 0 client.17625 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 shutdown_connections 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 1 -- 172.21.15.87:0/2652280892 wait complete. 2024-06-06T08:02:56.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:56.984+0000 7f9778c2ab80 20 client.17625 trim_cache size 0 max 16384 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 Processor -- start 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 -- start start 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 0x55fa9b718710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b71f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b1020 con 0x55fa9b718c50 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ae850 con 0x55fa9b645180 2024-06-06T08:02:57.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ac6c0 con 0x55fa9b71e500 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55004/0 (socket says 172.21.15.87:55004) 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.004+0000 7f975ffff700 1 -- 172.21.15.87:0/4142429048 learned_addr learned my addr 172.21.15.87:0/4142429048 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 -- 172.21.15.87:0/4142429048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 msgr2=0x55fa9b71f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b71f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f9767fff700 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 0x55fa9b718710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 -- 172.21.15.87:0/4142429048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 msgr2=0x55fa9b718710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 0x55fa9b718710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 -- 172.21.15.87:0/4142429048 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9b734d20 con 0x55fa9b718c50 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f9767fff700 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 0x55fa9b718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975ffff700 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9750099110 tx=0x7f97500a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a785e4f2f0df4045 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:57.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975dffb700 1 -- 172.21.15.87:0/4142429048 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97500b5070 con 0x55fa9b718c50 2024-06-06T08:02:57.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975dffb700 1 -- 172.21.15.87:0/4142429048 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975009c740 con 0x55fa9b718c50 2024-06-06T08:02:57.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.005+0000 7f975dffb700 1 -- 172.21.15.87:0/4142429048 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975009c9a0 con 0x55fa9b718c50 2024-06-06T08:02:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- 172.21.15.87:0/4142429048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 msgr2=0x55fa9b720d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9750099110 tx=0x7f97500a3040 comp rx=0 tx=0).stop 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- 172.21.15.87:0/4142429048 shutdown_connections 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b645180 0x55fa9b718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b71e500 0x55fa9b71f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/4142429048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b720d90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- 172.21.15.87:0/4142429048 >> 172.21.15.87:0/4142429048 conn(0x55fa9b71c330 msgr2=0x55fa9b71f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- 172.21.15.87:0/4142429048 shutdown_connections 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- 172.21.15.87:0/4142429048 wait complete. 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 -- start start 2024-06-06T08:02:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 Processor -- start 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.006+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b645180 0x55fa9b739500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b71e500 0x55fa9b73a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97681acd10 con 0x55fa9b718c50 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ae180 con 0x55fa9b71e500 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ad3e0 con 0x55fa9b645180 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 10 client.0 _ll_register_callbacks cb 0x55fa9b5ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55006/0 (socket says 172.21.15.87:55006) 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 -- 172.21.15.87:0/322176091 learned_addr learned my addr 172.21.15.87:0/322176091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:02:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9767fff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b645180 0x55fa9b739500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f976c944700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b71e500 0x55fa9b73a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 -- 172.21.15.87:0/322176091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b645180 msgr2=0x55fa9b739500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b645180 0x55fa9b739500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 -- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b71e500 msgr2=0x55fa9b73a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b71e500 0x55fa9b73a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 -- 172.21.15.87:0/322176091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9b73a9c0 con 0x55fa9b718c50 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f975ffff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9750022750 tx=0x7f9750009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89294813c60b0fc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f97477fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97500b5040 con 0x55fa9b718c50 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f975009cbb0 con 0x55fa9b718c50 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975009c740 con 0x55fa9b718c50 2024-06-06T08:02:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9750084050 con 0x55fa9b718c50 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 10 client.17631 fetch_fsmap learned FSMap version 46 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.007+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fa9b7a1d90 con 0x55fa9b718c50 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975009c740 con 0x55fa9b718c50 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f97500bc040 con 0x55fa9b718c50 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 10 client.17631 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 10 client.17631 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 20 client.17631 populate_metadata read hostname 'smithi087' 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b71ee20 0x55fa9b7a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 4 client.17631 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ce4f972e", "release": true, "await": true}] 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ce4f972e", "release": true, "await": true}) -- 0x55fa9b6713d0 con 0x55fa9b71ee20 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9767fff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b71ee20 0x55fa9b7a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:02:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f97477fe700 10 client.17631 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:57.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.008+0000 7f9767fff700 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b71ee20 0x55fa9b7a4400 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b971998dbc6cd875 server_cookie=33b414ce18b425e1 in_seq=0 out_seq=0 2024-06-06T08:02:57.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f97477fe700 1 -- 172.21.15.87:0/322176091 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fa9b6713d0 con 0x55fa9b71ee20 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f97477fe700 10 client.17631 handle_command_reply: tid=0 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 client.17631 shutdown 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 msgr2=0x55fa9b739a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9750022750 tx=0x7f9750009310 comp rx=0 tx=0).stop 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 shutdown_connections 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fa9b71ee20 0x55fa9b7a4400 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fa9b71e500 0x55fa9b73a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9b645180 0x55fa9b739500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f97477fe700 0 client.17631 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 --2- 172.21.15.87:0/322176091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fa9b718c50 0x55fa9b739a40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.079+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 >> 172.21.15.87:0/322176091 conn(0x55fa9b71c330 msgr2=0x55fa9b712320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:57.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 shutdown_connections 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f9778c2ab80 1 -- 172.21.15.87:0/322176091 wait complete. 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f9778c2ab80 20 client.17631 trim_cache size 0 max 16384 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 46, 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:02:57.081 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 44, 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "ce4f972e": { 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 44, 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:02:57.082 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:02:57.083 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 msgr2=0x7f97540a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 0x7f97540a52d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9768199bc0 tx=0x7f975800b3c0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 msgr2=0x7f976819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500047b0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 shutdown_connections 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9768101380 0x7f976819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f97540a2e10 0x7f97540a52d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97681073f0 0x7f9768198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 --2- 172.21.15.87:0/4216408183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9768101d40 0x7f976819ee00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:02:57.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 >> 172.21.15.87:0/4216408183 conn(0x7f9768078300 msgr2=0x7f97681005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:02:57.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 shutdown_connections 2024-06-06T08:02:57.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:02:57.080+0000 7f976e3a7700 1 -- 172.21.15.87:0/4216408183 wait complete. 2024-06-06T08:02:57.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11561 13.50 MB/sec warmup 59 sec latency 24231.338 ms 2024-06-06T08:02:57.811 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ce4f972e', total seconds elapsed: 31.0 2024-06-06T08:02:57.811 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T08:02:57.812 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:02:58.075 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:02:58.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 11993 13.54 MB/sec warmup 60 sec latency 82.184 ms 2024-06-06T08:02:58.199 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:57 smithi087 ceph-mon[32276]: pgmap v472: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.5 MiB/s wr, 300 op/s 2024-06-06T08:02:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:57 smithi195 ceph-mon[37040]: pgmap v472: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.5 MiB/s wr, 300 op/s 2024-06-06T08:02:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:57 smithi181 ceph-mon[36790]: pgmap v472: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.5 MiB/s wr, 300 op/s 2024-06-06T08:02:59.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 12624 13.95 MB/sec warmup 61 sec latency 51.554 ms 2024-06-06T08:03:00.104 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:02:59 smithi087 ceph-mon[32276]: pgmap v473: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-06-06T08:03:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.111+0000 7f3f88c33700 1 -- 172.21.15.87:0/1080165374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 msgr2=0x7f3f8410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.111+0000 7f3f88c33700 1 --2- 172.21.15.87:0/1080165374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8410e240 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b0a0 tx=0x7f3f7c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 -- 172.21.15.87:0/1080165374 shutdown_connections 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 --2- 172.21.15.87:0/1080165374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f84107440 0x7f3f84107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 --2- 172.21.15.87:0/1080165374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8410e240 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 --2- 172.21.15.87:0/1080165374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f841013c0 0x7f3f84101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 -- 172.21.15.87:0/1080165374 >> 172.21.15.87:0/1080165374 conn(0x7f3f84077400 msgr2=0x7f3f84077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 -- 172.21.15.87:0/1080165374 shutdown_connections 2024-06-06T08:03:00.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 -- 172.21.15.87:0/1080165374 wait complete. 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 Processor -- start 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.112+0000 7f3f88c33700 1 -- start start 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f841013c0 0x7f3f8419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f84107440 0x7f3f8419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84111660 con 0x7f3f84101d80 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841114e0 con 0x7f3f841013c0 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841117e0 con 0x7f3f84107440 2024-06-06T08:03:00.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55026/0 (socket says 172.21.15.87:55026) 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 -- 172.21.15.87:0/4153185973 learned_addr learned my addr 172.21.15.87:0/4153185973 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f82d9d700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f84107440 0x7f3f8419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f8259c700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f841013c0 0x7f3f8419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 -- 172.21.15.87:0/4153185973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f84107440 msgr2=0x7f3f8419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f84107440 0x7f3f8419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 -- 172.21.15.87:0/4153185973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f841013c0 msgr2=0x7f3f8419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f841013c0 0x7f3f8419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 -- 172.21.15.87:0/4153185973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8419fd50 con 0x7f3f84101d80 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f81d9b700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c920 tx=0x7f3f7800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78004a30 con 0x7f3f84101d80 2024-06-06T08:03:00.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78005500 con 0x7f3f84101d80 2024-06-06T08:03:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f841a0040 con 0x7f3f84101d80 2024-06-06T08:03:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78007d50 con 0x7f3f84101d80 2024-06-06T08:03:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.113+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a78c0 con 0x7f3f84101d80 2024-06-06T08:03:00.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.114+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f84102ab0 con 0x7f3f84101d80 2024-06-06T08:03:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.116+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3f78004d60 con 0x7f3f84101d80 2024-06-06T08:03:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.117+0000 7f3f737fe700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 0x7f3f6c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.117+0000 7f3f8259c700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 0x7f3f6c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.117+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f3f780cf750 con 0x7f3f84101d80 2024-06-06T08:03:00.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.118+0000 7f3f8259c700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 0x7f3f6c0a5250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009be0 tx=0x7f3f74009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:00.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.122+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f78014030 con 0x7f3f84101d80 2024-06-06T08:03:00.158 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13141 14.22 MB/sec warmup 62 sec latency 220.909 ms 2024-06-06T08:03:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:02:59 smithi195 ceph-mon[37040]: pgmap v473: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-06-06T08:03:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:02:59 smithi181 ceph-mon[36790]: pgmap v473: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 2.3 MiB/s wr, 233 op/s 2024-06-06T08:03:00.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.446+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8402dca0 con 0x7f3f84101d80 2024-06-06T08:03:00.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.447+0000 7f3f737fe700 1 -- 172.21.15.87:0/4153185973 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11881 (secure 0 0 0) 0x7f3f7809bf50 con 0x7f3f84101d80 2024-06-06T08:03:00.446 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:03:00.447 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:02:00.625810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26623":{"gid":26623,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/1019415876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1019415876},{"type":"v1","addr":"172.21.15.195:6837","nonce":1019415876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 msgr2=0x7f3f6c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 0x7f3f6c0a5250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009be0 tx=0x7f3f74009260 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 msgr2=0x7f3f8419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c920 tx=0x7f3f7800cc30 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 shutdown_connections 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f841013c0 0x7f3f8419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3f6c0a2d90 0x7f3f6c0a5250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f84107440 0x7f3f8419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 --2- 172.21.15.87:0/4153185973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f84101d80 0x7f3f8419b280 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 >> 172.21.15.87:0/4153185973 conn(0x7f3f84077400 msgr2=0x7f3f841005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 shutdown_connections 2024-06-06T08:03:00.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:00.449+0000 7f3f88c33700 1 -- 172.21.15.87:0/4153185973 wait complete. 2024-06-06T08:03:00.450 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 46 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- 172.21.15.87:0/961752970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb810bf70 msgr2=0x7f7eb810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 --2- 172.21.15.87:0/961752970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb810bf70 0x7f7eb810e410 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009ca0 tx=0x7f7eb002a9e0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- 172.21.15.87:0/961752970 shutdown_connections 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 --2- 172.21.15.87:0/961752970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb810bf70 0x7f7eb810e410 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 --2- 172.21.15.87:0/961752970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb8103550 0x7f7eb810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 --2- 172.21.15.87:0/961752970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 0x7f7eb8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- 172.21.15.87:0/961752970 >> 172.21.15.87:0/961752970 conn(0x7f7eb8077a20 msgr2=0x7f7eb8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- 172.21.15.87:0/961752970 shutdown_connections 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- 172.21.15.87:0/961752970 wait complete. 2024-06-06T08:03:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 Processor -- start 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.020+0000 7f7ebf212640 1 -- start start 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 0x7f7eb8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb810bf70 0x7f7eb819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8111d70 con 0x7f7eb8103550 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8111bf0 con 0x7f7eb810bf70 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebf212640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8111ef0 con 0x7f7eb8102c30 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55032/0 (socket says 172.21.15.87:55032) 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 -- 172.21.15.87:0/2604525945 learned_addr learned my addr 172.21.15.87:0/2604525945 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 0x7f7eb8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb810bf70 0x7f7eb819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 -- 172.21.15.87:0/2604525945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 msgr2=0x7f7eb8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 0x7f7eb8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 -- 172.21.15.87:0/2604525945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb810bf70 msgr2=0x7f7eb819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb810bf70 0x7f7eb819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 -- 172.21.15.87:0/2604525945 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb819d1d0 con 0x7f7eb8103550 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c970 tx=0x7f7ea400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.021+0000 7f7eb5ffb640 1 -- 172.21.15.87:0/2604525945 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4007bb0 con 0x7f7eb8103550 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.022+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb819d4c0 con 0x7f7eb8103550 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.022+0000 7f7eb5ffb640 1 -- 172.21.15.87:0/2604525945 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea4007d50 con 0x7f7eb8103550 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.022+0000 7f7eb5ffb640 1 -- 172.21.15.87:0/2604525945 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400f450 con 0x7f7eb8103550 2024-06-06T08:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.022+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81a4d50 con 0x7f7eb8103550 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7eb5ffb640 1 -- 172.21.15.87:0/2604525945 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7ea4005020 con 0x7f7eb8103550 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 Processor -- start 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 -- start start 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x560598449e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5605983a34f0 con 0x56059843f170 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560598341870 con 0x56059843fa90 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.024+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56059839f4b0 con 0x56059844a350 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7eb5ffb640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 0x7f7ea80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55048/0 (socket says 172.21.15.87:55048) 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 -- 172.21.15.87:0/2990314339 learned_addr learned my addr 172.21.15.87:0/2990314339 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x560598449e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7eb5ffb640 1 -- 172.21.15.87:0/2604525945 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea40cf600 con 0x7f7eb8103550 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 0x7f7ea80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 -- 172.21.15.87:0/2990314339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 msgr2=0x56059844c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 -- 172.21.15.87:0/2990314339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 msgr2=0x560598449e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x560598449e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 -- 172.21.15.87:0/2990314339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059835d9b0 con 0x56059843f170 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002ef30 tx=0x7f7eb002ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6277d7f1ca5afde server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.025+0000 7f7ea2ffd640 1 -- 172.21.15.87:0/2990314339 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb004b070 con 0x56059843f170 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ea2ffd640 1 -- 172.21.15.87:0/2990314339 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0003cf0 con 0x56059843f170 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ea2ffd640 1 -- 172.21.15.87:0/2990314339 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0033050 con 0x56059843f170 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/2990314339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 msgr2=0x56059843f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002ef30 tx=0x7f7eb002ef60 comp rx=0 tx=0).stop 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 0x7f7ea80a5260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7eb004d000 tx=0x7f7eb004b000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/2990314339 shutdown_connections 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x560598449e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/2990314339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x56059843f550 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/2990314339 >> 172.21.15.87:0/2990314339 conn(0x56059843b680 msgr2=0x56059843ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/2990314339 shutdown_connections 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.026+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/2990314339 wait complete. 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 -- start start 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 Processor -- start 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843f170 0x560598447250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x560598447cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56059844cf80 con 0x56059843fa90 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56059832cae0 con 0x56059843f170 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56059844d100 con 0x56059844a350 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ec1ff9740 10 client.0 _ll_register_callbacks cb 0x5605983659b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55050/0 (socket says 172.21.15.87:55050) 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 -- 172.21.15.87:0/1158954488 learned_addr learned my addr 172.21.15.87:0/1158954488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843f170 0x560598447250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x560598447cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 -- 172.21.15.87:0/1158954488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 msgr2=0x560598447cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x560598447cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 -- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843f170 msgr2=0x560598447250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.027+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843f170 0x560598447250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7eb7fff640 1 -- 172.21.15.87:0/1158954488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605980b7210 con 0x56059843fa90 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c940 tx=0x7f7ea409b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a9a71bd3917038 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401d040 con 0x56059843fa90 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea4097d70 con 0x56059843fa90 2024-06-06T08:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb0003cf0 con 0x56059843fa90 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7ea40860c0 con 0x56059843fa90 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 10 client.17649 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560598183620 con 0x56059843fa90 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0003cf0 con 0x56059843fa90 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f7ea4005310 con 0x56059843fa90 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 10 client.17649 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 20 client.17649 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56059845a3e0 0x56059845c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 4 client.17649 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.028+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605983b6f10 con 0x56059845a3e0 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.029+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56059845a3e0 0x56059845c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.029+0000 7f7e97fff640 10 client.17649 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.029+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56059845a3e0 0x56059845c7d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e354ad680b3c624 server_cookie=78aaf8565e81d086 in_seq=0 out_seq=0 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7e97fff640 1 -- 172.21.15.87:0/1158954488 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5605983b6f10 con 0x56059845a3e0 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7e97fff640 10 client.17649 handle_command_reply: tid=0 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 client.17649 shutdown 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 msgr2=0x560598447790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c940 tx=0x7f7ea409b5a0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 shutdown_connections 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56059845a3e0 0x56059845c7d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x560598447cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598447790 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7e97fff640 0 client.17649 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/1158954488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843f170 0x560598447250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 >> 172.21.15.87:0/1158954488 conn(0x56059843b680 msgr2=0x560598441400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 shutdown_connections 2024-06-06T08:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/1158954488 wait complete. 2024-06-06T08:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.030+0000 7f7ec1ff9740 20 client.17649 trim_cache size 0 max 16384 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 Processor -- start 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 -- start start 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059843f170 0x56059843a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 0x5605984625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b1900 con 0x56059843fa90 2024-06-06T08:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81af170 con 0x56059844a350 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81acfe0 con 0x56059843f170 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55058/0 (socket says 172.21.15.87:55058) 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7eb7fff640 1 -- 172.21.15.87:0/3824642258 learned_addr learned my addr 172.21.15.87:0/3824642258 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.045+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059843f170 0x56059843a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 0x5605984625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 -- 172.21.15.87:0/3824642258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059843f170 msgr2=0x56059843a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059843f170 0x56059843a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 -- 172.21.15.87:0/3824642258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 msgr2=0x5605984625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 0x5605984625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 -- 172.21.15.87:0/3824642258 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059843b4b0 con 0x56059843fa90 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 0x5605984625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4097b60 tx=0x7f7ea4086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef705d8d365df97f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7ea27fc640 1 -- 172.21.15.87:0/3824642258 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401d040 con 0x56059843fa90 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7ea27fc640 1 -- 172.21.15.87:0/3824642258 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea409b700 con 0x56059843fa90 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.046+0000 7f7ea27fc640 1 -- 172.21.15.87:0/3824642258 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4005310 con 0x56059843fa90 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/3824642258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 msgr2=0x560598462070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4097b60 tx=0x7f7ea4086050 comp rx=0 tx=0).stop 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/3824642258 shutdown_connections 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059844a350 0x5605984625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843fa90 0x560598462070 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/3824642258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059843f170 0x56059843a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/3824642258 >> 172.21.15.87:0/3824642258 conn(0x56059843b680 msgr2=0x560598439f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/3824642258 shutdown_connections 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/3824642258 wait complete. 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 -- start start 2024-06-06T08:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 Processor -- start 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.047+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x56059844dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81ad630 con 0x56059843f170 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81aeaa0 con 0x56059843fa90 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81add00 con 0x56059844a350 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 10 client.0 _ll_register_callbacks cb 0x7f7ea4004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55066/0 (socket says 172.21.15.87:55066) 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 -- 172.21.15.87:0/794597797 learned_addr learned my addr 172.21.15.87:0/794597797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebd7af640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x56059844dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 -- 172.21.15.87:0/794597797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 msgr2=0x56059844e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 -- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 msgr2=0x56059844dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x56059844dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 -- 172.21.15.87:0/794597797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605984561e0 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ebcfae640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c704d472001f1b96 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac004220 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f7eac0043c0 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea409b700 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.048+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7eac01e790 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 10 client.17655 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5605984c4ef0 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea409b700 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f7eac003cd0 con 0x56059843f170 2024-06-06T08:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 10 client.17655 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 20 client.17655 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5605984c56f0 0x5605984c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 4 client.17655 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605983b0c20 con 0x5605984c56f0 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5605984c56f0 0x5605984c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7e97fff640 10 client.17655 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.049+0000 7f7eb7fff640 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5605984c56f0 0x5605984c7b10 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25f6e6ca51ad924 server_cookie=6a94b11265995c6b in_seq=0 out_seq=0 2024-06-06T08:03:01.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4153185973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:01.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13183 14.01 MB/sec warmup 63 sec latency 834.085 ms 2024-06-06T08:03:01.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:00 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4153185973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:01.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:00 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4153185973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7e97fff640 1 -- 172.21.15.87:0/794597797 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5605983b0c20 con 0x5605984c56f0 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7e97fff640 10 client.17655 handle_command_reply: tid=0 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 client.17655 shutdown 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 msgr2=0x560598452b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac0118d0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 shutdown_connections 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5605984c56f0 0x5605984c7b10 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7e97fff640 0 client.17655 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56059844a350 0x56059844e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56059843fa90 0x56059844dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 --2- 172.21.15.87:0/794597797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56059843f170 0x560598452b50 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 >> 172.21.15.87:0/794597797 conn(0x56059843b680 msgr2=0x5605984413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 shutdown_connections 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 1 -- 172.21.15.87:0/794597797 wait complete. 2024-06-06T08:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.355+0000 7f7ec1ff9740 20 client.17655 trim_cache size 0 max 16384 2024-06-06T08:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 msgr2=0x7f7ea80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 0x7f7ea80a5260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7eb004d000 tx=0x7f7eb004b000 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 msgr2=0x7f7eb8198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c970 tx=0x7f7ea400ce30 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 shutdown_connections 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea80a2da0 0x7f7ea80a5260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb810bf70 0x7f7eb819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8103550 0x7f7eb8198770 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 --2- 172.21.15.87:0/2604525945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb8102c30 0x7f7eb8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 >> 172.21.15.87:0/2604525945 conn(0x7f7eb8077a20 msgr2=0x7f7eb80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 shutdown_connections 2024-06-06T08:03:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:01.356+0000 7f7ebf212640 1 -- 172.21.15.87:0/2604525945 wait complete. 2024-06-06T08:03:01.387 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T08:03:01.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T08:03:01.662 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:03:02.064 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:01 smithi087 ceph-mon[32276]: pgmap v474: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-06-06T08:03:02.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13576 14.07 MB/sec warmup 64 sec latency 445.292 ms 2024-06-06T08:03:02.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:01 smithi195 ceph-mon[37040]: pgmap v474: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-06-06T08:03:02.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:01 smithi181 ceph-mon[36790]: pgmap v474: 129 pgs: 129 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-06-06T08:03:03.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13710 13.97 MB/sec warmup 65 sec latency 312.402 ms 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.592+0000 7f9b4ce6c700 1 -- 172.21.15.87:0/2596896359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44003d10 con 0x7f9b48113340 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 -- 172.21.15.87:0/2596896359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b48113340 msgr2=0x7f9b48115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 --2- 172.21.15.87:0/2596896359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b48113340 0x7f9b48115730 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9b44009990 tx=0x7f9b44009ca0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 -- 172.21.15.87:0/2596896359 shutdown_connections 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 --2- 172.21.15.87:0/2596896359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 0x7f9b48076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 --2- 172.21.15.87:0/2596896359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b48113340 0x7f9b48115730 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 --2- 172.21.15.87:0/2596896359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b480772e0 0x7f9b48112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.593+0000 7f9b500d2700 1 -- 172.21.15.87:0/2596896359 >> 172.21.15.87:0/2596896359 conn(0x7f9b48100170 msgr2=0x7f9b48102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:03.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.597+0000 7f9b500d2700 1 -- 172.21.15.87:0/2596896359 shutdown_connections 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.597+0000 7f9b500d2700 1 -- 172.21.15.87:0/2596896359 wait complete. 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 Processor -- start 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 -- start start 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 0x7f9b48112900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b48113340 0x7f9b4810cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48104fb0 con 0x7f9b480772e0 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48104e30 con 0x7f9b480769c0 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.598+0000 7f9b500d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48105130 con 0x7f9b48113340 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.599+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.599+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55090/0 (socket says 172.21.15.87:55090) 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.599+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3683415156 learned_addr learned my addr 172.21.15.87:0/3683415156 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.599+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b48113340 0x7f9b4810cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.599+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 0x7f9b48112900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3683415156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b48113340 msgr2=0x7f9b4810cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b48113340 0x7f9b4810cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3683415156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 msgr2=0x7f9b48112900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 0x7f9b48112900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3683415156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4810d500 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.600+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c920 tx=0x7f9b3800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.601+0000 7f9b3effd700 1 -- 172.21.15.87:0/3683415156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38007aa0 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.601+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4810d7f0 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.601+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4811e8b0 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.601+0000 7f9b3effd700 1 -- 172.21.15.87:0/3683415156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b38004d10 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.601+0000 7f9b3effd700 1 -- 172.21.15.87:0/3683415156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38007aa0 con 0x7f9b480772e0 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.608+0000 7f9b5a955b80 1 Processor -- start 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.608+0000 7f9b5a955b80 1 -- start start 2024-06-06T08:03:03.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.608+0000 7f9b3effd700 1 -- 172.21.15.87:0/3683415156 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9b38005430 con 0x7f9b480772e0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b3effd700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 0x7f9b340a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b3effd700 1 -- 172.21.15.87:0/3683415156 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f9b380078a0 con 0x7f9b480772e0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348d5480 0x5625348d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5625348159b0 con 0x5625348db610 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625347bd8c0 con 0x5625348d5480 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.611+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562534811970 con 0x5625348dacf0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.612+0000 7f9b4e66f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.612+0000 7f9b4e66f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:59122/0 (socket says 172.21.15.87:59122) 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.612+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1408910196 learned_addr learned my addr 172.21.15.87:0/1408910196 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.612+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348d5480 0x5625348d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.612+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 0x7f9b340a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1408910196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348d5480 msgr2=0x5625348d58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348d5480 0x5625348d58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1408910196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 msgr2=0x5625348d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1408910196 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625347b40f0 con 0x5625348dacf0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b440038d0 tx=0x7f9b44009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c8fead2872912 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.613+0000 7f9b33fff700 1 -- 172.21.15.87:0/1408910196 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44021040 con 0x5625348dacf0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.614+0000 7f9b33fff700 1 -- 172.21.15.87:0/1408910196 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4400bc10 con 0x5625348dacf0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.614+0000 7f9b33fff700 1 -- 172.21.15.87:0/1408910196 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44005680 con 0x5625348dacf0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.614+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 0x7f9b340a5750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009f40 tx=0x7f9b400094d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1408910196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 msgr2=0x5625348db0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b440038d0 tx=0x7f9b44009ca0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1408910196 shutdown_connections 2024-06-06T08:03:03.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348d5480 0x5625348d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348db0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1408910196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1408910196 >> 172.21.15.87:0/1408910196 conn(0x5625347eee60 msgr2=0x5625348bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.615+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1408910196 shutdown_connections 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.616+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1408910196 wait complete. 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.616+0000 7f9b5a955b80 1 -- start start 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 Processor -- start 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 0x5625348e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5625348e2020 con 0x5625348db610 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625347b5d90 con 0x5625348dacf0 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5625348e21a0 con 0x5625348d5480 2024-06-06T08:03:03.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b4e66f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b4e66f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348debc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55100/0 (socket says 172.21.15.87:55100) 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b4e66f700 1 -- 172.21.15.87:0/2321938293 learned_addr learned my addr 172.21.15.87:0/2321938293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 0x5625348e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.617+0000 7f9b5a955b80 10 client.0 _ll_register_callbacks cb 0x5625347afac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4de6e700 1 -- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 msgr2=0x5625348de680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348de680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4de6e700 1 -- 172.21.15.87:0/2321938293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 msgr2=0x5625348debc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348debc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.618+0000 7f9b4de6e700 1 -- 172.21.15.87:0/2321938293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562534764a70 con 0x5625348d5480 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.623+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 0x5625348e1310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b40017bd0 tx=0x7f9b40004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1491fedaa7d8cc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.623+0000 7f9b317fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:03:03.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.624+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40027060 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.624+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f9b40012030 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.624+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4400bc10 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.624+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9b4000ce00 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.625+0000 7f9b5a955b80 10 client.26809 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.625+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5625348ec7a0 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b40016c10 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f9b40022c50 con 0x5625348d5480 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 10 client.26809 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 20 client.26809 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5625348eed30 0x5625348f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 4 client.26809 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.626+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9b4400bc10 con 0x5625348eed30 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.629+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5625348eed30 0x5625348f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.629+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5625348eed30 0x5625348f1120 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284c46e4ae54182f server_cookie=c58fac18a09ba3b8 in_seq=0 out_seq=0 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.629+0000 7f9b317fa700 10 client.26809 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.630+0000 7f9b317fa700 1 -- 172.21.15.87:0/2321938293 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f9b4400bc10 con 0x5625348eed30 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.630+0000 7f9b317fa700 10 client.26809 handle_command_reply: tid=0 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.631+0000 7f9b5a955b80 1 client.26809 shutdown 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 msgr2=0x5625348e1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 0x5625348e1310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b40017bd0 tx=0x7f9b40004b80 comp rx=0 tx=0).stop 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 shutdown_connections 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5625348eed30 0x5625348f1120 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348db610 0x5625348debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348d5480 0x5625348e1310 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b40017bd0 tx=0x7f9b40004b80 comp rx=0 tx=0).stop 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/2321938293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348de680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.635+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 >> 172.21.15.87:0/2321938293 conn(0x5625347eee60 msgr2=0x5625348bedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.636+0000 7f9b317fa700 0 client.26809 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:03.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.638+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 shutdown_connections 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.639+0000 7f9b5a955b80 1 -- 172.21.15.87:0/2321938293 wait complete. 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.639+0000 7f9b5a955b80 20 client.26809 trim_cache size 0 max 16384 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 Processor -- start 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 -- start start 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.679+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c011940 con 0x5625348bf330 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c00f170 con 0x5625348db610 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c00cfe0 con 0x5625348dacf0 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4de6e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41526/0 (socket says 172.21.15.87:41526) 2024-06-06T08:03:03.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3624076054 learned_addr learned my addr 172.21.15.87:0/3624076054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3624076054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 msgr2=0x5625348dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3624076054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 msgr2=0x5625348db9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.680+0000 7f9b4d66d700 1 -- 172.21.15.87:0/3624076054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625348df590 con 0x5625348db610 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.681+0000 7f9b4d66d700 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b380051c0 tx=0x7f9b380bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2aaa3cb9753ad6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.683+0000 7f9b337fe700 1 -- 172.21.15.87:0/3624076054 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b4070 con 0x5625348db610 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.683+0000 7f9b337fe700 1 -- 172.21.15.87:0/3624076054 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3809a0c0 con 0x5625348db610 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 -- 172.21.15.87:0/3624076054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 msgr2=0x5625348dd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b380051c0 tx=0x7f9b380bc040 comp rx=0 tx=0).stop 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 -- 172.21.15.87:0/3624076054 shutdown_connections 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348db9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348dacf0 0x5625348dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/3624076054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348db610 0x5625348dd4b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.684+0000 7f9b5a955b80 1 -- 172.21.15.87:0/3624076054 >> 172.21.15.87:0/3624076054 conn(0x5625347eee60 msgr2=0x5625348d8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.686+0000 7f9b5a955b80 1 -- 172.21.15.87:0/3624076054 shutdown_connections 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.687+0000 7f9b5a955b80 1 -- 172.21.15.87:0/3624076054 wait complete. 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.687+0000 7f9b5a955b80 1 -- start start 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.687+0000 7f9b5a955b80 1 Processor -- start 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348e4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 0x5625348e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c00d630 con 0x5625348bf330 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c00eaa0 con 0x5625348dacf0 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c00dd00 con 0x5625348db610 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b5a955b80 10 client.0 _ll_register_callbacks cb 0x5625347afac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b4d66d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348e49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41536/0 (socket says 172.21.15.87:41536) 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.688+0000 7f9b4d66d700 1 -- 172.21.15.87:0/1998800458 learned_addr learned my addr 172.21.15.87:0/1998800458 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 0x5625348e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 msgr2=0x5625348e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1998800458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 msgr2=0x5625348e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 -- 172.21.15.87:0/1998800458 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562534959c00 con 0x5625348db610 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b4e66f700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 0x5625348e4f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5625348bb5c0 tx=0x7f9b44009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe578eba5361a97 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.689+0000 7f9b317fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.690+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44005bc0 con 0x5625348db610 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.690+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9583+0+0 (secure 0 0 0) 0x7f9b44003ba0 con 0x5625348db610 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.690+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625348b9140 con 0x5625348db610 2024-06-06T08:03:03.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.690+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9b4400be20 con 0x5625348db610 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.693+0000 7f9b5a955b80 10 client.26815 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.693+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56253495af10 con 0x5625348db610 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.694+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5625348b9140 con 0x5625348db610 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.694+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f9b440288f0 con 0x5625348db610 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 10 client.26815 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 20 client.26815 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56253495b190 0x56253495d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 4 client.26815 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.695+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x562534822eb0 con 0x56253495b190 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.696+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56253495b190 0x56253495d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.698+0000 7f9b4de6e700 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56253495b190 0x56253495d580 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d91f3b0dbe8d0a server_cookie=acca76438d0d7871 in_seq=0 out_seq=0 2024-06-06T08:03:03.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:03.698+0000 7f9b317fa700 10 client.26815 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:04.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:03 smithi087 ceph-mon[32276]: pgmap v475: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 264 op/s 2024-06-06T08:03:04.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13718 13.80 MB/sec warmup 66 sec latency 316.777 ms 2024-06-06T08:03:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:03 smithi195 ceph-mon[37040]: pgmap v475: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 264 op/s 2024-06-06T08:03:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:03 smithi181 ceph-mon[36790]: pgmap v475: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 264 op/s 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.641+0000 7f9b317fa700 1 -- 172.21.15.87:0/1998800458 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562534822eb0 con 0x56253495b190 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.641+0000 7f9b317fa700 10 client.26815 handle_command_reply: tid=0 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.641+0000 7f9b5a955b80 1 client.26815 shutdown 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 msgr2=0x5625348e4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 0x5625348e4f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5625348bb5c0 tx=0x7f9b44009710 comp rx=0 tx=0).stop 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 shutdown_connections 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56253495b190 0x56253495d580 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5625348db610 0x5625348e4f10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625348bf330 0x5625348e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 --2- 172.21.15.87:0/1998800458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625348dacf0 0x5625348e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 >> 172.21.15.87:0/1998800458 conn(0x5625347eee60 msgr2=0x5625348d8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b317fa700 0 client.26815 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 shutdown_connections 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 1 -- 172.21.15.87:0/1998800458 wait complete. 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b5a955b80 20 client.26815 trim_cache size 0 max 16384 2024-06-06T08:03:04.650 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 46, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 46, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "24644966": { 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 46, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:03:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:04.652 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:03:04.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 msgr2=0x7f9b340a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:04.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 0x7f9b340a5750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009f40 tx=0x7f9b400094d0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 msgr2=0x7f9b4810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:04.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.642+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c920 tx=0x7f9b3800cce0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 shutdown_connections 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480769c0 0x7f9b48112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f9b340a3290 0x7f9b340a5750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b48113340 0x7f9b4810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 --2- 172.21.15.87:0/3683415156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b480772e0 0x7f9b4810c9d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 >> 172.21.15.87:0/3683415156 conn(0x7f9b48100170 msgr2=0x7f9b48102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 shutdown_connections 2024-06-06T08:03:04.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:04.643+0000 7f9b500d2700 1 -- 172.21.15.87:0/3683415156 wait complete. 2024-06-06T08:03:05.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 13.60 MB/sec warmup 67 sec latency 1305.500 ms 2024-06-06T08:03:05.430 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '24644966', quiesce took 7.6 seconds. Will release after: 17.65 2024-06-06T08:03:06.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 13.40 MB/sec warmup 68 sec latency 2305.549 ms 2024-06-06T08:03:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:05 smithi195 ceph-mon[37040]: pgmap v476: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-06T08:03:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:05 smithi181 ceph-mon[36790]: pgmap v476: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-06T08:03:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:05 smithi087 ceph-mon[32276]: pgmap v476: 129 pgs: 129 active+clean; 291 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-06-06T08:03:07.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 13.21 MB/sec warmup 69 sec latency 3305.593 ms 2024-06-06T08:03:07.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:06 smithi195 ceph-mon[37040]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T08:03:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:06 smithi181 ceph-mon[36790]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T08:03:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:06 smithi087 ceph-mon[32276]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T08:03:08.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 13.02 MB/sec warmup 70 sec latency 4305.638 ms 2024-06-06T08:03:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:07 smithi195 ceph-mon[37040]: pgmap v478: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 295 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-06-06T08:03:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:07 smithi181 ceph-mon[36790]: pgmap v478: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 295 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-06-06T08:03:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:07 smithi087 ceph-mon[32276]: pgmap v478: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 295 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-06-06T08:03:09.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 12.83 MB/sec warmup 71 sec latency 5305.731 ms 2024-06-06T08:03:10.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 12.65 MB/sec warmup 72 sec latency 6305.848 ms 2024-06-06T08:03:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:09 smithi195 ceph-mon[37040]: pgmap v479: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:09 smithi181 ceph-mon[36790]: pgmap v479: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:09 smithi087 ceph-mon[32276]: pgmap v479: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:11.159 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 12.48 MB/sec warmup 73 sec latency 7305.980 ms 2024-06-06T08:03:12.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 12.31 MB/sec warmup 74 sec latency 8306.091 ms 2024-06-06T08:03:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:11 smithi195 ceph-mon[37040]: pgmap v480: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:11 smithi181 ceph-mon[36790]: pgmap v480: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:11 smithi087 ceph-mon[32276]: pgmap v480: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 294 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 480 op/s 2024-06-06T08:03:13.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 12.15 MB/sec warmup 75 sec latency 9306.130 ms 2024-06-06T08:03:14.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.99 MB/sec warmup 76 sec latency 10306.253 ms 2024-06-06T08:03:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:13 smithi195 ceph-mon[37040]: pgmap v481: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:13 smithi181 ceph-mon[36790]: pgmap v481: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:13 smithi087 ceph-mon[32276]: pgmap v481: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 -- 172.21.15.87:0/4244215947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 msgr2=0x7f49780ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 --2- 172.21.15.87:0/4244215947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f49780ffea0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402a400 comp rx=0 tx=0).stop 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 -- 172.21.15.87:0/4244215947 shutdown_connections 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 --2- 172.21.15.87:0/4244215947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 0x7f4978110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 --2- 172.21.15.87:0/4244215947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 0x7f49781003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 --2- 172.21.15.87:0/4244215947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f49780ffea0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 -- 172.21.15.87:0/4244215947 >> 172.21.15.87:0/4244215947 conn(0x7f4978076e00 msgr2=0x7f49780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 -- 172.21.15.87:0/4244215947 shutdown_connections 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 -- 172.21.15.87:0/4244215947 wait complete. 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.456+0000 7f497f9f9640 1 Processor -- start 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 -- start start 2024-06-06T08:03:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 0x7f497819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 0x7f497819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4978114000 con 0x7f4978102970 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4978113e80 con 0x7f4978102de0 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497f9f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4978114180 con 0x7f497810e620 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41536/0 (socket says 172.21.15.87:41536) 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 -- 172.21.15.87:0/2302864959 learned_addr learned my addr 172.21.15.87:0/2302864959 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497df96640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 0x7f497819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497cf94640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 0x7f497819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 -- 172.21.15.87:0/2302864959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 msgr2=0x7f497819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 0x7f497819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 -- 172.21.15.87:0/2302864959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 msgr2=0x7f497819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 0x7f497819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 -- 172.21.15.87:0/2302864959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497819f390 con 0x7f4978102970 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f497d795640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4974005ab0 tx=0x7f49740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.457+0000 7f496e7fc640 1 -- 172.21.15.87:0/2302864959 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497403f030 con 0x7f4978102970 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.458+0000 7f496e7fc640 1 -- 172.21.15.87:0/2302864959 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4974003ce0 con 0x7f4978102970 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.458+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497819f620 con 0x7f4978102970 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.458+0000 7f496e7fc640 1 -- 172.21.15.87:0/2302864959 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974035070 con 0x7f4978102970 2024-06-06T08:03:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.458+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49781a6f10 con 0x7f4978102970 2024-06-06T08:03:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.460+0000 7f496e7fc640 1 -- 172.21.15.87:0/2302864959 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4974042030 con 0x7f4978102970 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.460+0000 7f49827e9740 1 Processor -- start 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 -- start start 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f496e7fc640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 0x7f49640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed34a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f496e7fc640 1 -- 172.21.15.87:0/2302864959 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9671+0+0 (secure 0 0 0) 0x7f49740f0290 con 0x7f4978102970 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fed33fd4f0 con 0x55fed3499170 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fed339b870 con 0x55fed34a4350 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f49827e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fed33f94b0 con 0x55fed3499a90 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497df96640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 0x7f49640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41552/0 (socket says 172.21.15.87:41552) 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 -- 172.21.15.87:0/2660164897 learned_addr learned my addr 172.21.15.87:0/2660164897 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497cf94640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497df96640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed34a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 -- 172.21.15.87:0/2660164897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 msgr2=0x55fed34a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 -- 172.21.15.87:0/2660164897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 msgr2=0x55fed34a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed34a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 -- 172.21.15.87:0/2660164897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed33b79b0 con 0x55fed3499170 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497df96640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed34a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497df96640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 0x7f49640a5260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4978073e90 tx=0x7f497000e720 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.461+0000 7f497d795640 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f49740bc2c0 tx=0x7f49740bc1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8b8c27df1166a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49637fe640 1 -- 172.21.15.87:0/2660164897 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974049040 con 0x55fed3499170 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49637fe640 1 -- 172.21.15.87:0/2660164897 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740bc330 con 0x55fed3499170 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49637fe640 1 -- 172.21.15.87:0/2660164897 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740b89a0 con 0x55fed3499170 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 -- 172.21.15.87:0/2660164897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 msgr2=0x55fed3499550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f49740bc2c0 tx=0x7f49740bc1c0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 -- 172.21.15.87:0/2660164897 shutdown_connections 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed34a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 --2- 172.21.15.87:0/2660164897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed3499550 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 -- 172.21.15.87:0/2660164897 >> 172.21.15.87:0/2660164897 conn(0x55fed3495680 msgr2=0x55fed3495a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 -- 172.21.15.87:0/2660164897 shutdown_connections 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.462+0000 7f49827e9740 1 -- 172.21.15.87:0/2660164897 wait complete. 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 -- start start 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 Processor -- start 2024-06-06T08:03:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 0x55fed34a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 0x55fed34a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55fed34a6d40 con 0x55fed3499170 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fed3386ae0 con 0x55fed3499a90 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fed34a6ec0 con 0x55fed34a4350 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f49827e9740 10 client.0 _ll_register_callbacks cb 0x55fed33bf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f497d795640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41558/0 (socket says 172.21.15.87:41558) 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.463+0000 7f497d795640 1 -- 172.21.15.87:0/3769955973 learned_addr learned my addr 172.21.15.87:0/3769955973 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497df96640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 0x55fed34a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 -- 172.21.15.87:0/3769955973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 msgr2=0x55fed34a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 0x55fed34a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 -- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 msgr2=0x55fed34a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497cf94640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 0x55fed34a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 0x55fed34a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 -- 172.21.15.87:0/3769955973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed3111210 con 0x55fed3499170 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497df96640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 0x55fed34a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497cf94640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 0x55fed34a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f497d795640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f49740bc030 tx=0x7f497402a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e4309d38da2254 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f4960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740d4070 con 0x55fed3499170 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9671+0+0 (secure 0 0 0) 0x7f49740d5260 con 0x55fed3499170 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740bc330 con 0x55fed3499170 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4974049050 con 0x55fed3499170 2024-06-06T08:03:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f49827e9740 10 client.17679 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fed31dd620 con 0x55fed3499170 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.464+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740bc330 con 0x55fed3499170 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f49740871a0 con 0x55fed3499170 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 10 client.17679 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 20 client.17679 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed34b4110 0x55fed34b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 4 client.17679 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed3410f10 con 0x55fed34b4110 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f497cf94640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed34b4110 0x55fed34b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f4960ff9640 10 client.17679 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.465+0000 7f497cf94640 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed34b4110 0x55fed34b6500 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6311435878756ef2 server_cookie=9cf5f3973ac15cd7 in_seq=0 out_seq=0 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f4960ff9640 1 -- 172.21.15.87:0/3769955973 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fed3410f10 con 0x55fed34b4110 2024-06-06T08:03:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f4960ff9640 10 client.17679 handle_command_reply: tid=0 2024-06-06T08:03:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 client.17679 shutdown 2024-06-06T08:03:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 msgr2=0x55fed34a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f49740bc030 tx=0x7f497402a400 comp rx=0 tx=0).stop 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 shutdown_connections 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed34b4110 0x55fed34b6500 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed34a4350 0x55fed34a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499a90 0x55fed34a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f4960ff9640 0 client.17679 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.466+0000 7f49827e9740 1 --2- 172.21.15.87:0/3769955973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499170 0x55fed34a13f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.467+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 >> 172.21.15.87:0/3769955973 conn(0x55fed3495680 msgr2=0x55fed349b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.467+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 shutdown_connections 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.467+0000 7f49827e9740 1 -- 172.21.15.87:0/3769955973 wait complete. 2024-06-06T08:03:14.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.467+0000 7f49827e9740 20 client.17679 trim_cache size 0 max 16384 2024-06-06T08:03:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 Processor -- start 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 -- start start 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499170 0x55fed34bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed3493f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49781b3bd0 con 0x55fed3499a90 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49781b1440 con 0x55fed34a4350 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f49827e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49781af2b0 con 0x55fed3499170 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41560/0 (socket says 172.21.15.87:41560) 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 -- 172.21.15.87:0/3096617942 learned_addr learned my addr 172.21.15.87:0/3096617942 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497d795640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499170 0x55fed34bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 -- 172.21.15.87:0/3096617942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499170 msgr2=0x55fed34bbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499170 0x55fed34bbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497df96640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed3493f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 -- 172.21.15.87:0/3096617942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 msgr2=0x55fed3493f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed3493f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497cf94640 1 -- 172.21.15.87:0/3096617942 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed34afec0 con 0x55fed3499a90 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.482+0000 7f497df96640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed3493f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f497cf94640 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f497806a200 tx=0x7f496800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ad1a2b44703619 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f4962ffd640 1 -- 172.21.15.87:0/3096617942 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496801e070 con 0x55fed3499a90 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f4962ffd640 1 -- 172.21.15.87:0/3096617942 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4968005050 con 0x55fed3499a90 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f4962ffd640 1 -- 172.21.15.87:0/3096617942 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968004bc0 con 0x55fed3499a90 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 -- 172.21.15.87:0/3096617942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 msgr2=0x55fed34bc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f497806a200 tx=0x7f496800c430 comp rx=0 tx=0).stop 2024-06-06T08:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 -- 172.21.15.87:0/3096617942 shutdown_connections 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed34a4350 0x55fed3493f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed3499a90 0x55fed34bc260 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 --2- 172.21.15.87:0/3096617942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499170 0x55fed34bbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 -- 172.21.15.87:0/3096617942 >> 172.21.15.87:0/3096617942 conn(0x55fed3495680 msgr2=0x55fed349c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 -- 172.21.15.87:0/3096617942 shutdown_connections 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.483+0000 7f49827e9740 1 -- 172.21.15.87:0/3096617942 wait complete. 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 -- start start 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 Processor -- start 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499170 0x55fed34a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49781af900 con 0x55fed34a4350 2024-06-06T08:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49781b0d70 con 0x55fed3499170 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49781affd0 con 0x55fed3499a90 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f49827e9740 10 client.0 _ll_register_callbacks cb 0x7f49740c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f497df96640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f497df96640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41568/0 (socket says 172.21.15.87:41568) 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f497df96640 1 -- 172.21.15.87:0/1530225661 learned_addr learned my addr 172.21.15.87:0/1530225661 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.484+0000 7f497cf94640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497d795640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499170 0x55fed34a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 -- 172.21.15.87:0/1530225661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 msgr2=0x55fed34a8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 -- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499170 msgr2=0x55fed34a7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499170 0x55fed34a7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 -- 172.21.15.87:0/1530225661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed351dcf0 con 0x55fed34a4350 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f497df96640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4970010e10 tx=0x7f4970004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56e751a77de370c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970023070 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9671+0+0 (secure 0 0 0) 0x7f4970003b70 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4968005050 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f497002e050 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 10 client.17685 fetch_fsmap learned FSMap version 46 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fed351eee0 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4968005050 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f497001f540 con 0x55fed34a4350 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 10 client.17685 fetch_fsmap finished waiting for FSMap version 46 2024-06-06T08:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 20 client.17685 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed351f6f0 0x55fed3521b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 4 client.17685 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.485+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fed340ac20 con 0x55fed351f6f0 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.486+0000 7f497d795640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed351f6f0 0x55fed3521b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.486+0000 7f4960ff9640 10 client.17685 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.486+0000 7f497d795640 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed351f6f0 0x55fed3521b10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f8eee4ca3596da server_cookie=868e54aaba7a0e59 in_seq=0 out_seq=0 2024-06-06T08:03:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f4960ff9640 1 -- 172.21.15.87:0/1530225661 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fed340ac20 con 0x55fed351f6f0 2024-06-06T08:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f4960ff9640 10 client.17685 handle_command_reply: tid=0 2024-06-06T08:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 client.17685 shutdown 2024-06-06T08:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 msgr2=0x55fed34a8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4970010e10 tx=0x7f4970004ab0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 shutdown_connections 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55fed351f6f0 0x55fed3521b10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f4960ff9640 0 client.17685 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fed34a4350 0x55fed34a8790 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fed3499a90 0x55fed34a8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 --2- 172.21.15.87:0/1530225661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fed3499170 0x55fed34a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 >> 172.21.15.87:0/1530225661 conn(0x55fed3495680 msgr2=0x55fed34a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 shutdown_connections 2024-06-06T08:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 1 -- 172.21.15.87:0/1530225661 wait complete. 2024-06-06T08:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.686+0000 7f49827e9740 20 client.17685 trim_cache size 0 max 16384 2024-06-06T08:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 msgr2=0x7f49640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 0x7f49640a5260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4978073e90 tx=0x7f497000e720 comp rx=0 tx=0).stop 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 msgr2=0x7f497819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4974005ab0 tx=0x7f49740042f0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 shutdown_connections 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49640a2da0 0x7f49640a5260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497810e620 0x7f497819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978102de0 0x7f497819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 --2- 172.21.15.87:0/2302864959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4978102970 0x7f497819a3a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 >> 172.21.15.87:0/2302864959 conn(0x7f4978076e00 msgr2=0x7f497810f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 shutdown_connections 2024-06-06T08:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:14.687+0000 7f497f9f9640 1 -- 172.21.15.87:0/2302864959 wait complete. 2024-06-06T08:03:15.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.83 MB/sec warmup 77 sec latency 11306.314 ms 2024-06-06T08:03:15.509 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T08:03:15.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T08:03:15.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/3612055915 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f5bd0ba40 con 0x563f5bcde800 2024-06-06T08:03:15.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T08:03:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:15 smithi087 ceph-mon[32276]: pgmap v482: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:15 smithi087 ceph-mon[32276]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T08:03:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:15 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T08:03:16.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.68 MB/sec warmup 78 sec latency 12306.407 ms 2024-06-06T08:03:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-mon[37040]: pgmap v482: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:16.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-mon[37040]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T08:03:16.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:15 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T08:03:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:15 smithi181 ceph-mon[36790]: pgmap v482: 129 pgs: 129 active+clean; 292 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-06-06T08:03:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:15 smithi181 ceph-mon[36790]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T08:03:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:15 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T08:03:17.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:16 smithi087 ceph-mon[32276]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T08:03:17.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:16 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/3997704113,v1:172.21.15.195:6837/3997704113] up:boot 2024-06-06T08:03:17.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:16 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:03:17.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:16 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:03:17.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:16 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:03:17.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.53 MB/sec warmup 79 sec latency 13306.520 ms 2024-06-06T08:03:17.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:16 smithi195 ceph-mon[37040]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T08:03:17.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:16 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/3997704113,v1:172.21.15.195:6837/3997704113] up:boot 2024-06-06T08:03:17.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:16 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:03:17.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:16 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:03:17.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:16 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:03:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:16 smithi181 ceph-mon[36790]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T08:03:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:16 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/3997704113,v1:172.21.15.195:6837/3997704113] up:boot 2024-06-06T08:03:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:16 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:03:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:16 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:03:17.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:16 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:03:18.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.39 MB/sec warmup 80 sec latency 14306.641 ms 2024-06-06T08:03:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:17 smithi195 ceph-mon[37040]: pgmap v485: 129 pgs: 129 active+clean; 288 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 905 KiB/s wr, 58 op/s 2024-06-06T08:03:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:17 smithi181 ceph-mon[36790]: pgmap v485: 129 pgs: 129 active+clean; 288 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 905 KiB/s wr, 58 op/s 2024-06-06T08:03:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:17 smithi087 ceph-mon[32276]: pgmap v485: 129 pgs: 129 active+clean; 288 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 905 KiB/s wr, 58 op/s 2024-06-06T08:03:19.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.25 MB/sec warmup 81 sec latency 15306.766 ms 2024-06-06T08:03:20.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 11.11 MB/sec warmup 82 sec latency 16306.823 ms 2024-06-06T08:03:20.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:19 smithi195 ceph-mon[37040]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-06-06T08:03:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:19 smithi181 ceph-mon[36790]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-06-06T08:03:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:19 smithi087 ceph-mon[32276]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-06-06T08:03:21.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.98 MB/sec warmup 83 sec latency 17306.910 ms 2024-06-06T08:03:22.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.85 MB/sec warmup 84 sec latency 18307.018 ms 2024-06-06T08:03:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:21 smithi195 ceph-mon[37040]: pgmap v487: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 3.1 MiB/s wr, 169 op/s 2024-06-06T08:03:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:21 smithi181 ceph-mon[36790]: pgmap v487: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 3.1 MiB/s wr, 169 op/s 2024-06-06T08:03:22.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:21 smithi087 ceph-mon[32276]: pgmap v487: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 3.1 MiB/s wr, 169 op/s 2024-06-06T08:03:23.082 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:03:23.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.72 MB/sec warmup 85 sec latency 19307.092 ms 2024-06-06T08:03:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:03:24.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.59 MB/sec warmup 86 sec latency 20307.128 ms 2024-06-06T08:03:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:23 smithi195 ceph-mon[37040]: pgmap v488: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.1 MiB/s wr, 179 op/s 2024-06-06T08:03:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:23 smithi181 ceph-mon[36790]: pgmap v488: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.1 MiB/s wr, 179 op/s 2024-06-06T08:03:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:23 smithi087 ceph-mon[32276]: pgmap v488: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 3.1 MiB/s wr, 179 op/s 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.095+0000 7ffbe7a35700 1 -- 172.21.15.87:0/1444628856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0107840 msgr2=0x7ffbe010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.095+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/1444628856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0107840 0x7ffbe010f3c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800b0d0 tx=0x7ffbd800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- 172.21.15.87:0/1444628856 shutdown_connections 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/1444628856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0102740 0x7ffbe0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/1444628856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0107840 0x7ffbe010f3c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/1444628856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- 172.21.15.87:0/1444628856 >> 172.21.15.87:0/1444628856 conn(0x7ffbe00795f0 msgr2=0x7ffbe0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- 172.21.15.87:0/1444628856 shutdown_connections 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- 172.21.15.87:0/1444628856 wait complete. 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 Processor -- start 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- start start 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0107840 0x7ffbe0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01127d0 con 0x7ffbe0102740 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112650 con 0x7ffbe0107840 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.096+0000 7ffbe7a35700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112950 con 0x7ffbe01030f0 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:25.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57854/0 (socket says 172.21.15.87:57854) 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 -- 172.21.15.87:0/3032452328 learned_addr learned my addr 172.21.15.87:0/3032452328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe5fd2700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0107840 0x7ffbe0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe4fd0700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 -- 172.21.15.87:0/3032452328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 msgr2=0x7ffbe019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 -- 172.21.15.87:0/3032452328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0107840 msgr2=0x7ffbe0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0107840 0x7ffbe0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 -- 172.21.15.87:0/3032452328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe01990d0 con 0x7ffbe0102740 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe4fd0700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe57d1700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00d950 tx=0x7ffbdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc016070 con 0x7ffbe0102740 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe01993c0 con 0x7ffbe0102740 2024-06-06T08:03:25.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc009e20 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00f620 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.097+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe01a5680 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.098+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe0069180 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.100+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ffbdc019070 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.101+0000 7ffbd67fc700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 0x7ffbcc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.101+0000 7ffbe4fd0700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 0x7ffbcc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.101+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7ffbdc0cf3e0 con 0x7ffbe0102740 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.101+0000 7ffbe4fd0700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 0x7ffbcc0a5250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0108c90 tx=0x7ffbd0008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:25.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.105+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffbdc09bc30 con 0x7ffbe0102740 2024-06-06T08:03:25.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.47 MB/sec warmup 87 sec latency 21307.175 ms 2024-06-06T08:03:25.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.345+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe01a58e0 con 0x7ffbe0102740 2024-06-06T08:03:25.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.345+0000 7ffbd67fc700 1 -- 172.21.15.87:0/3032452328 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11881 (secure 0 0 0) 0x7ffbdc09b530 con 0x7ffbe0102740 2024-06-06T08:03:25.345 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:03:25.346 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:03:15.948875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26833":{"gid":26833,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/3997704113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3997704113},{"type":"v1","addr":"172.21.15.195:6837","nonce":3997704113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 msgr2=0x7ffbcc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 0x7ffbcc0a5250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0108c90 tx=0x7ffbd0008040 comp rx=0 tx=0).stop 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 msgr2=0x7ffbe019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00d950 tx=0x7ffbdc00dc60 comp rx=0 tx=0).stop 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 shutdown_connections 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe0107840 0x7ffbe0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ffbcc0a2d90 0x7ffbcc0a5250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe0102740 0x7ffbe019e7b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 --2- 172.21.15.87:0/3032452328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe01030f0 0x7ffbe019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 >> 172.21.15.87:0/3032452328 conn(0x7ffbe00795f0 msgr2=0x7ffbe0100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 shutdown_connections 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:25.348+0000 7ffbe7a35700 1 -- 172.21.15.87:0/3032452328 wait complete. 2024-06-06T08:03:25.349 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 48 2024-06-06T08:03:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:25 smithi087 ceph-mon[32276]: pgmap v489: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 2.6 MiB/s wr, 146 op/s 2024-06-06T08:03:26.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:25 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3032452328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:26.153 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 24644966 --release --await' 2024-06-06T08:03:26.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 24644966 --release --await 2024-06-06T08:03:26.160 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.35 MB/sec warmup 88 sec latency 22307.233 ms 2024-06-06T08:03:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:25 smithi195 ceph-mon[37040]: pgmap v489: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 2.6 MiB/s wr, 146 op/s 2024-06-06T08:03:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:25 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3032452328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:25 smithi181 ceph-mon[36790]: pgmap v489: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 2.6 MiB/s wr, 146 op/s 2024-06-06T08:03:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:25 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3032452328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:03:26.398 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:03:27.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.24 MB/sec warmup 89 sec latency 23307.345 ms 2024-06-06T08:03:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 -- 172.21.15.87:0/2589712693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c105370 msgr2=0x7fe23c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 --2- 172.21.15.87:0/2589712693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c105370 0x7fe23c10cf40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe234009ca0 tx=0x7fe23402a8c0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 -- 172.21.15.87:0/2589712693 shutdown_connections 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 --2- 172.21.15.87:0/2589712693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c105370 0x7fe23c10cf40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 --2- 172.21.15.87:0/2589712693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c100c50 0x7fe23c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 --2- 172.21.15.87:0/2589712693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 -- 172.21.15.87:0/2589712693 >> 172.21.15.87:0/2589712693 conn(0x7fe23c076e00 msgr2=0x7fe23c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 -- 172.21.15.87:0/2589712693 shutdown_connections 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.777+0000 7fe240c9a640 1 -- 172.21.15.87:0/2589712693 wait complete. 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 Processor -- start 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 -- start start 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c105370 0x7fe23c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c110d80 con 0x7fe23c19bb20 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c110c00 con 0x7fe23c1002a0 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.778+0000 7fe240c9a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c110f00 con 0x7fe23c105370 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47922/0 (socket says 172.21.15.87:47922) 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 -- 172.21.15.87:0/602153330 learned_addr learned my addr 172.21.15.87:0/602153330 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23a59c640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe239d9b640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c105370 0x7fe23c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 -- 172.21.15.87:0/602153330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c105370 msgr2=0x7fe23c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c105370 0x7fe23c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 -- 172.21.15.87:0/602153330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 msgr2=0x7fe23c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 -- 172.21.15.87:0/602153330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c19c610 con 0x7fe23c19bb20 2024-06-06T08:03:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23a59c640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe23ad9d640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe234031a60 tx=0x7fe234031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe22b7fe640 1 -- 172.21.15.87:0/602153330 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234003c30 con 0x7fe23c19bb20 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c19c8a0 con 0x7fe23c19bb20 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe22b7fe640 1 -- 172.21.15.87:0/602153330 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe234003dd0 con 0x7fe23c19bb20 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe22b7fe640 1 -- 172.21.15.87:0/602153330 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340047e0 con 0x7fe23c19bb20 2024-06-06T08:03:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.779+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c1a40f0 con 0x7fe23c19bb20 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.781+0000 7fe22b7fe640 1 -- 172.21.15.87:0/602153330 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe23402f790 con 0x7fe23c19bb20 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 Processor -- start 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 -- start start 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f0ab1e4f0 con 0x556f0abbcca0 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f0aabc870 con 0x556f0abc7560 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe2439a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f0ab1a4b0 con 0x556f0abbc380 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe22b7fe640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 0x7fe2240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47938/0 (socket says 172.21.15.87:47938) 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe23a59c640 1 -- 172.21.15.87:0/3884613985 learned_addr learned my addr 172.21.15.87:0/3884613985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.782+0000 7fe23ad9d640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 0x7fe2240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe22b7fe640 1 -- 172.21.15.87:0/602153330 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fe2340f0bc0 con 0x7fe23c19bb20 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 -- 172.21.15.87:0/3884613985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 msgr2=0x556f0abbc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 -- 172.21.15.87:0/3884613985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 msgr2=0x556f0abc9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 -- 172.21.15.87:0/3884613985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0aad89b0 con 0x556f0abbcca0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007850 tx=0x7fe22c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1f068a15533146 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe23ad9d640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 0x7fe2240a5260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2340bcdb0 tx=0x7fe2340095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe228ff9640 1 -- 172.21.15.87:0/3884613985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00d950 con 0x556f0abbcca0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe228ff9640 1 -- 172.21.15.87:0/3884613985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c018070 con 0x556f0abbcca0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.783+0000 7fe228ff9640 1 -- 172.21.15.87:0/3884613985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c013e60 con 0x556f0abbcca0 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- 172.21.15.87:0/3884613985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 msgr2=0x556f0abc7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007850 tx=0x7fe22c007d10 comp rx=0 tx=0).stop 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- 172.21.15.87:0/3884613985 shutdown_connections 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc7020 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3884613985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- 172.21.15.87:0/3884613985 >> 172.21.15.87:0/3884613985 conn(0x556f0abb8890 msgr2=0x556f0abb8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- 172.21.15.87:0/3884613985 shutdown_connections 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- 172.21.15.87:0/3884613985 wait complete. 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- start start 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 Processor -- start 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbcca0 0x556f0abc4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f0abca190 con 0x556f0abbc380 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f0aaa7ae0 con 0x556f0abc7560 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.784+0000 7fe2439a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f0abca310 con 0x556f0abbcca0 2024-06-06T08:03:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe2439a1740 10 client.0 _ll_register_callbacks cb 0x556f0aae09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47944/0 (socket says 172.21.15.87:47944) 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 -- 172.21.15.87:0/3686876566 learned_addr learned my addr 172.21.15.87:0/3686876566 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbcca0 0x556f0abc4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 -- 172.21.15.87:0/3686876566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbcca0 msgr2=0x556f0abc4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbcca0 0x556f0abc4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 -- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 msgr2=0x556f0abc4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 -- 172.21.15.87:0/3686876566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0a832210 con 0x556f0abbc380 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe239d9b640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe23c102b30 tx=0x7fe2300047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d321c1014d9f477 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230017e50 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fe230014a30 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe22c018070 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe230014c30 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe2439a1740 10 client.17709 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556f0a8fe620 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.785+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c018070 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe230012070 con 0x556f0abbc380 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 10 client.17709 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 10 client.17709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 20 client.17709 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0abd77e0 0x556f0abd9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 4 client.17709 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f0ab31f10 con 0x556f0abd77e0 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0abd77e0 0x556f0abd9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe23a59c640 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0abd77e0 0x556f0abd9bd0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c54c1720f80b38 server_cookie=9d98842c99bba935 in_seq=0 out_seq=0 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.786+0000 7fe221ffb640 10 client.17709 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe221ffb640 1 -- 172.21.15.87:0/3686876566 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f0ab31f10 con 0x556f0abd77e0 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe221ffb640 10 client.17709 handle_command_reply: tid=0 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 client.17709 shutdown 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 msgr2=0x556f0abc4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe23c102b30 tx=0x7fe2300047d0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 shutdown_connections 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0abd77e0 0x556f0abd9bd0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abc4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe221ffb640 0 client.17709 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbcca0 0x556f0abc4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 --2- 172.21.15.87:0/3686876566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbc380 0x556f0abc4530 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 >> 172.21.15.87:0/3686876566 conn(0x556f0abb8890 msgr2=0x556f0abb57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 shutdown_connections 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 1 -- 172.21.15.87:0/3686876566 wait complete. 2024-06-06T08:03:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.787+0000 7fe2439a1740 20 client.17709 trim_cache size 0 max 16384 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 Processor -- start 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 -- start start 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abcf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1af6c0 con 0x556f0abbcca0 2024-06-06T08:03:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1acf30 con 0x556f0abc7560 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.802+0000 7fe2439a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1aad80 con 0x556f0abbc380 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47960/0 (socket says 172.21.15.87:47960) 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 -- 172.21.15.87:0/1027536194 learned_addr learned my addr 172.21.15.87:0/1027536194 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 -- 172.21.15.87:0/1027536194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 msgr2=0x556f0abbed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 -- 172.21.15.87:0/1027536194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 msgr2=0x556f0abcf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abcf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 -- 172.21.15.87:0/1027536194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0abbcad0 con 0x556f0abbcca0 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe23a59c640 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe22c013d80 tx=0x7fe22c013a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635eb55b302fab7a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe223fff640 1 -- 172.21.15.87:0/1027536194 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c013680 con 0x556f0abbcca0 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe223fff640 1 -- 172.21.15.87:0/1027536194 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c018040 con 0x556f0abbcca0 2024-06-06T08:03:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.803+0000 7fe223fff640 1 -- 172.21.15.87:0/1027536194 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c026710 con 0x556f0abbcca0 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- 172.21.15.87:0/1027536194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 msgr2=0x556f0abc6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe22c013d80 tx=0x7fe22c013a40 comp rx=0 tx=0).stop 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- 172.21.15.87:0/1027536194 shutdown_connections 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abcf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abc6470 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- 172.21.15.87:0/1027536194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abbed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- 172.21.15.87:0/1027536194 >> 172.21.15.87:0/1027536194 conn(0x556f0abb8890 msgr2=0x556f0abc5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- 172.21.15.87:0/1027536194 shutdown_connections 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- 172.21.15.87:0/1027536194 wait complete. 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- start start 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 Processor -- start 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1ab3f0 con 0x556f0abbcca0 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1ac860 con 0x556f0abc7560 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1abac0 con 0x556f0abbc380 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe2439a1740 10 client.0 _ll_register_callbacks cb 0x7fe230017400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe23a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47962/0 (socket says 172.21.15.87:47962) 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe23a59c640 1 -- 172.21.15.87:0/2911888320 learned_addr learned my addr 172.21.15.87:0/2911888320 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.804+0000 7fe239d9b640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 -- 172.21.15.87:0/2911888320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 msgr2=0x556f0abb7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abb7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 -- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 msgr2=0x556f0abb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 -- 172.21.15.87:0/2911888320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0abb5ce0 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe23a59c640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007850 tx=0x7fe22c013ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86259fae07ce200 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00db50 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fe22c00dcf0 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe22c018040 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe22c026490 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 10 client.17715 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556f0ac44570 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c018040 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe22c02f720 con 0x556f0abbcca0 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 10 client.17715 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 10 client.17715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 20 client.17715 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0ac44b60 0x556f0ac46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 4 client.17715 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.805+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f0ab2bc20 con 0x556f0ac44b60 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.806+0000 7fe239d9b640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0ac44b60 0x556f0ac46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.806+0000 7fe221ffb640 10 client.17715 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.806+0000 7fe239d9b640 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0ac44b60 0x556f0ac46f80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fb0a38e8ccd60b server_cookie=34d8ab1da6324888 in_seq=0 out_seq=0 2024-06-06T08:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe221ffb640 1 -- 172.21.15.87:0/2911888320 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f0ab2bc20 con 0x556f0ac44b60 2024-06-06T08:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe221ffb640 10 client.17715 handle_command_reply: tid=0 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 client.17715 shutdown 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 msgr2=0x556f0abb4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007850 tx=0x7fe22c013ab0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 shutdown_connections 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f0ac44b60 0x556f0ac46f80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f0abc7560 0x556f0abb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f0abbcca0 0x556f0abb4f50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe221ffb640 0 client.17715 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 --2- 172.21.15.87:0/2911888320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f0abbc380 0x556f0abb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 >> 172.21.15.87:0/2911888320 conn(0x556f0abb8890 msgr2=0x556f0abc65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 shutdown_connections 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 1 -- 172.21.15.87:0/2911888320 wait complete. 2024-06-06T08:03:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.850+0000 7fe2439a1740 20 client.17715 trim_cache size 0 max 16384 2024-06-06T08:03:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:03:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:03:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 msgr2=0x7fe2240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 0x7fe2240a5260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2340bcdb0 tx=0x7fe2340095c0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 msgr2=0x7fe23c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe234031a60 tx=0x7fe234031a90 comp rx=0 tx=0).stop 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 shutdown_connections 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2240a2da0 0x7fe2240a5260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe23c19bb20 0x7fe23c19bfd0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c105370 0x7fe23c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 --2- 172.21.15.87:0/602153330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe23c1002a0 0x7fe23c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 >> 172.21.15.87:0/602153330 conn(0x7fe23c076e00 msgr2=0x7fe23c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 shutdown_connections 2024-06-06T08:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:27.851+0000 7fe240c9a640 1 -- 172.21.15.87:0/602153330 wait complete. 2024-06-06T08:03:28.136 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:27 smithi087 ceph-mon[32276]: pgmap v490: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 -- 172.21.15.87:0/3914426177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f081030f0 msgr2=0x7f5f08107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/3914426177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f081030f0 0x7f5f08107110 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00099c0 tx=0x7f5ef0009cd0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 -- 172.21.15.87:0/3914426177 shutdown_connections 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/3914426177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08107840 0x7f5f0810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/3914426177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f08102740 0x7f5f08102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/3914426177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f081030f0 0x7f5f08107110 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 -- 172.21.15.87:0/3914426177 >> 172.21.15.87:0/3914426177 conn(0x7f5f080795f0 msgr2=0x7f5f08079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 -- 172.21.15.87:0/3914426177 shutdown_connections 2024-06-06T08:03:28.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.143+0000 7f5f0c98e700 1 -- 172.21.15.87:0/3914426177 wait complete. 2024-06-06T08:03:28.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 Processor -- start 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 -- start start 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08102740 0x7f5f0819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f081030f0 0x7f5f0819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08112770 con 0x7f5f08107840 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081125f0 con 0x7f5f08102740 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.144+0000 7f5f0c98e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081128f0 con 0x7f5f081030f0 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47976/0 (socket says 172.21.15.87:47976) 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 -- 172.21.15.87:0/1365616189 learned_addr learned my addr 172.21.15.87:0/1365616189 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f0659c700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08102740 0x7f5f0819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f05d9b700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f081030f0 0x7f5f0819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 -- 172.21.15.87:0/1365616189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f081030f0 msgr2=0x7f5f0819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f081030f0 0x7f5f0819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 -- 172.21.15.87:0/1365616189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08102740 msgr2=0x7f5f0819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08102740 0x7f5f0819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 -- 172.21.15.87:0/1365616189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f081990d0 con 0x7f5f08107840 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c900 tx=0x7f5f0000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5eff7fe700 1 -- 172.21.15.87:0/1365616189 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00007b00 con 0x7f5f08107840 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f081993c0 con 0x7f5f08107840 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5eff7fe700 1 -- 172.21.15.87:0/1365616189 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0000ce50 con 0x7f5f08107840 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5eff7fe700 1 -- 172.21.15.87:0/1365616189 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00018660 con 0x7f5f08107840 2024-06-06T08:03:28.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.145+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f081a5630 con 0x7f5f08107840 2024-06-06T08:03:28.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.147+0000 7f5eff7fe700 1 -- 172.21.15.87:0/1365616189 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5f0000f450 con 0x7f5f08107840 2024-06-06T08:03:28.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.148+0000 7f5eff7fe700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 0x7f5ef40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.148+0000 7f5f0659c700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 0x7f5ef40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.148+0000 7f5eff7fe700 1 -- 172.21.15.87:0/1365616189 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7f5f000cf7e0 con 0x7f5f08107840 2024-06-06T08:03:28.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.148+0000 7f5f0659c700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 0x7f5ef40a5250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f081073f0 tx=0x7f5ef8009180 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 Processor -- start 2024-06-06T08:03:28.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 -- start start 2024-06-06T08:03:28.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca16560 0x55576ca0fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55576c961910 con 0x55576ca15c40 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55576c909820 con 0x55576ca103d0 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f17211b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55576c95d8d0 con 0x55576ca16560 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47990/0 (socket says 172.21.15.87:47990) 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.149+0000 7f5f0659c700 1 -- 172.21.15.87:0/2524580915 learned_addr learned my addr 172.21.15.87:0/2524580915 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f05d9b700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca16560 0x55576ca0fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 -- 172.21.15.87:0/2524580915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca16560 msgr2=0x55576ca0fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca16560 0x55576ca0fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 -- 172.21.15.87:0/2524580915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 msgr2=0x55576ca10810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca10810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 -- 172.21.15.87:0/2524580915 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576c900050 con 0x55576ca15c40 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5f0659c700 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009550 tx=0x7f5ef8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36e0ed8ef6cbf17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5efcff9700 1 -- 172.21.15.87:0/2524580915 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef801c050 con 0x55576ca15c40 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5efcff9700 1 -- 172.21.15.87:0/2524580915 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef8026070 con 0x55576ca15c40 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.150+0000 7f5efcff9700 1 -- 172.21.15.87:0/2524580915 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80156a0 con 0x55576ca15c40 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- 172.21.15.87:0/2524580915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 msgr2=0x55576ca16020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009550 tx=0x7f5ef8013000 comp rx=0 tx=0).stop 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- 172.21.15.87:0/2524580915 shutdown_connections 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca15c40 0x55576ca16020 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 --2- 172.21.15.87:0/2524580915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca16560 0x55576ca0fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- 172.21.15.87:0/2524580915 >> 172.21.15.87:0/2524580915 conn(0x55576c93adc0 msgr2=0x55576ca0b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- 172.21.15.87:0/2524580915 shutdown_connections 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- 172.21.15.87:0/2524580915 wait complete. 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 -- start start 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.151+0000 7f5f17211b80 1 Processor -- start 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55576ca213e0 con 0x55576ca16560 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55576c901cf0 con 0x55576ca103d0 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55576ca21560 con 0x55576ca15c40 2024-06-06T08:03:28.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f17211b80 10 client.0 _ll_register_callbacks cb 0x55576c8fba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47998/0 (socket says 172.21.15.87:47998) 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 -- 172.21.15.87:0/327222533 learned_addr learned my addr 172.21.15.87:0/327222533 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f05d9b700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f0659c700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 -- 172.21.15.87:0/327222533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 msgr2=0x55576ca195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 -- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 msgr2=0x55576ca1c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca1c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 -- 172.21.15.87:0/327222533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576c8b09d0 con 0x55576ca16560 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5f00097d90 tx=0x7f5f0009c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8070e24ce4f73f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5eedffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:28.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.152+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000f950 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7f5f000cf3f0 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef8026070 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5f000a45a0 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 10 client.17727 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55576ca2bce0 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f000ba020 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5f0009bb90 con 0x55576ca16560 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 10 client.17727 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 10 client.17727 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 20 client.17727 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 4 client.17727 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55576c96ee10 con 0x55576ca2e320 2024-06-06T08:03:28.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.153+0000 7f5f0659c700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.154+0000 7f5eedffb700 10 client.17727 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:28.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.154+0000 7f5f0659c700 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca30710 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36240f3a1a37cf99 server_cookie=495d60507d485a9f in_seq=0 out_seq=0 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5eedffb700 1 -- 172.21.15.87:0/327222533 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55576c96ee10 con 0x55576ca2e320 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5eedffb700 10 client.17727 handle_command_reply: tid=0 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 client.17727 shutdown 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 msgr2=0x55576ca19b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5f00097d90 tx=0x7f5f0009c120 comp rx=0 tx=0).stop 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 shutdown_connections 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca30710 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5eedffb700 0 client.17727 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca19b30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 --2- 172.21.15.87:0/327222533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 >> 172.21.15.87:0/327222533 conn(0x55576c93adc0 msgr2=0x55576ca0ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 shutdown_connections 2024-06-06T08:03:28.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 1 -- 172.21.15.87:0/327222533 wait complete. 2024-06-06T08:03:28.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.155+0000 7f5f17211b80 20 client.17727 trim_cache size 0 max 16384 2024-06-06T08:03:28.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 13722 10.12 MB/sec warmup 90 sec latency 24307.401 ms 2024-06-06T08:03:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 Processor -- start 2024-06-06T08:03:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 -- start start 2024-06-06T08:03:28.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca103d0 0x55576ca17050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca15c40 0x55576ca35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081aec50 con 0x55576ca16560 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081ac480 con 0x55576ca15c40 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f17211b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081aa2d0 con 0x55576ca103d0 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48014/0 (socket says 172.21.15.87:48014) 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.175+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3552089799 learned_addr learned my addr 172.21.15.87:0/3552089799 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f0659c700 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca103d0 0x55576ca17050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f05d9b700 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca15c40 0x55576ca35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3552089799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca103d0 msgr2=0x55576ca17050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca103d0 0x55576ca17050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3552089799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca15c40 msgr2=0x55576ca35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca15c40 0x55576ca35fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3552089799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576ca17590 con 0x55576ca16560 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5f000a4760 tx=0x7f5f00086460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df51ec62f9a8732d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5eeffff700 1 -- 172.21.15.87:0/3552089799 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000f950 con 0x55576ca16560 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5eeffff700 1 -- 172.21.15.87:0/3552089799 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f000ba040 con 0x55576ca16560 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.176+0000 7f5eeffff700 1 -- 172.21.15.87:0/3552089799 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000cf3f0 con 0x55576ca16560 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- 172.21.15.87:0/3552089799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 msgr2=0x55576ca36520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5f000a4760 tx=0x7f5f00086460 comp rx=0 tx=0).stop 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- 172.21.15.87:0/3552089799 shutdown_connections 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca36520 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca103d0 0x55576ca17050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3552089799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca15c40 0x55576ca35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- 172.21.15.87:0/3552089799 >> 172.21.15.87:0/3552089799 conn(0x55576c93adc0 msgr2=0x55576ca16940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- 172.21.15.87:0/3552089799 shutdown_connections 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- 172.21.15.87:0/3552089799 wait complete. 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- start start 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 Processor -- start 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081aa940 con 0x55576ca16560 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081abdb0 con 0x55576ca103d0 2024-06-06T08:03:28.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081ab010 con 0x55576ca15c40 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.177+0000 7f5f17211b80 10 client.0 _ll_register_callbacks cb 0x55576c8fba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f05d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48018/0 (socket says 172.21.15.87:48018) 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3709621830 learned_addr learned my addr 172.21.15.87:0/3709621830 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f0659c700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3709621830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 msgr2=0x55576ca23d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca23d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 msgr2=0x55576ca237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:28.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 -- 172.21.15.87:0/3709621830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576ca98fc0 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f06d9d700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f00099be0 tx=0x7f5f0009bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a7228012f8d87b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5eedffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000f950 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7f5f00018800 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f000ba040 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5f00081050 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f17211b80 10 client.17733 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.178+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55576ca98fc0 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f000ba040 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5f00016b90 con 0x55576ca16560 2024-06-06T08:03:28.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 10 client.17733 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 10 client.17733 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 20 client.17733 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca9c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 4 client.17733 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "24644966", "release": true, "await": true}] 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "24644966", "release": true, "await": true}) -- 0x55576c969390 con 0x55576ca2e320 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f0659c700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca9c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5eedffb700 10 client.17733 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:28.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.179+0000 7f5f0659c700 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca9c600 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8836b4e85e655a53 server_cookie=ef46e2b6f084f579 in_seq=0 out_seq=0 2024-06-06T08:03:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:27 smithi195 ceph-mon[37040]: pgmap v490: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.261+0000 7f5eedffb700 1 -- 172.21.15.87:0/3709621830 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55576c969390 con 0x55576ca2e320 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.261+0000 7f5eedffb700 10 client.17733 handle_command_reply: tid=0 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.261+0000 7f5f17211b80 1 client.17733 shutdown 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 msgr2=0x55576ca24270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f00099be0 tx=0x7f5f0009bb90 comp rx=0 tx=0).stop 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 shutdown_connections 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55576ca2e320 0x55576ca9c600 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55576ca103d0 0x55576ca237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55576ca16560 0x55576ca24270 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5eedffb700 0 client.17733 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 --2- 172.21.15.87:0/3709621830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55576ca15c40 0x55576ca23d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 >> 172.21.15.87:0/3709621830 conn(0x55576c93adc0 msgr2=0x55576ca18770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 shutdown_connections 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 1 -- 172.21.15.87:0/3709621830 wait complete. 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.262+0000 7f5f17211b80 20 client.17733 trim_cache size 0 max 16384 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 48, 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 48, 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "24644966": { 2024-06-06T08:03:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 48, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 145.2, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:03:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:03:28.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 msgr2=0x7f5ef40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 0x7f5ef40a5250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f081073f0 tx=0x7f5ef8009180 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 msgr2=0x7f5f08198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c900 tx=0x7f5f0000cc10 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 shutdown_connections 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f08102740 0x7f5f0819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5ef40a2d90 0x7f5ef40a5250 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f08107840 0x7f5f08198970 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 --2- 172.21.15.87:0/1365616189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f081030f0 0x7f5f0819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 >> 172.21.15.87:0/1365616189 conn(0x7f5f080795f0 msgr2=0x7f5f08101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 shutdown_connections 2024-06-06T08:03:28.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:03:28.263+0000 7f5f0c98e700 1 -- 172.21.15.87:0/1365616189 wait complete. 2024-06-06T08:03:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:27 smithi181 ceph-mon[36790]: pgmap v490: 129 pgs: 129 active+clean; 300 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-06-06T08:03:28.966 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '24644966', total seconds elapsed: 31.2 2024-06-06T08:03:29.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 14104 10.31 MB/sec warmup 91 sec latency 24334.629 ms 2024-06-06T08:03:30.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 14429 10.44 MB/sec warmup 92 sec latency 177.460 ms 2024-06-06T08:03:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:29 smithi195 ceph-mon[37040]: pgmap v491: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-06-06T08:03:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:29 smithi181 ceph-mon[36790]: pgmap v491: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-06-06T08:03:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:29 smithi087 ceph-mon[32276]: pgmap v491: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.6 MiB/s wr, 91 op/s 2024-06-06T08:03:31.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 14734 10.51 MB/sec warmup 93 sec latency 227.313 ms 2024-06-06T08:03:32.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 14976 10.47 MB/sec warmup 94 sec latency 190.979 ms 2024-06-06T08:03:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:31 smithi195 ceph-mon[37040]: pgmap v492: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-06-06T08:03:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:31 smithi181 ceph-mon[36790]: pgmap v492: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-06-06T08:03:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:31 smithi087 ceph-mon[32276]: pgmap v492: 129 pgs: 129 active+clean; 301 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-06-06T08:03:33.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 15309 10.50 MB/sec warmup 95 sec latency 224.812 ms 2024-06-06T08:03:33.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:32 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:03:33.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:32 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:03:33.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:32 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:03:34.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 15741 10.56 MB/sec warmup 96 sec latency 275.414 ms 2024-06-06T08:03:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:33 smithi195 ceph-mon[37040]: pgmap v493: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 943 KiB/s wr, 116 op/s 2024-06-06T08:03:34.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:33 smithi181 ceph-mon[36790]: pgmap v493: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 943 KiB/s wr, 116 op/s 2024-06-06T08:03:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:33 smithi087 ceph-mon[32276]: pgmap v493: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 943 KiB/s wr, 116 op/s 2024-06-06T08:03:35.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 16179 10.70 MB/sec warmup 97 sec latency 199.063 ms 2024-06-06T08:03:36.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 16778 10.88 MB/sec warmup 98 sec latency 73.097 ms 2024-06-06T08:03:36.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:35 smithi195 ceph-mon[37040]: pgmap v494: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 926 KiB/s wr, 109 op/s 2024-06-06T08:03:36.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:35 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:03:36.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:35 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:03:36.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:35 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:03:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:35 smithi181 ceph-mon[36790]: pgmap v494: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 926 KiB/s wr, 109 op/s 2024-06-06T08:03:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:35 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:03:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:35 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:03:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:35 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:03:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:35 smithi087 ceph-mon[32276]: pgmap v494: 129 pgs: 129 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 926 KiB/s wr, 109 op/s 2024-06-06T08:03:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:35 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:03:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:35 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:03:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:35 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:03:37.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 17349 11.23 MB/sec warmup 99 sec latency 98.046 ms 2024-06-06T08:03:38.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 17841 11.40 MB/sec warmup 100 sec latency 103.067 ms 2024-06-06T08:03:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:37 smithi195 ceph-mon[37040]: pgmap v495: 129 pgs: 129 active+clean; 313 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-06T08:03:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:37 smithi181 ceph-mon[36790]: pgmap v495: 129 pgs: 129 active+clean; 313 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-06T08:03:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:37 smithi087 ceph-mon[32276]: pgmap v495: 129 pgs: 129 active+clean; 313 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-06T08:03:39.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 18173 11.51 MB/sec warmup 101 sec latency 50.055 ms 2024-06-06T08:03:40.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 18474 11.53 MB/sec warmup 102 sec latency 75.559 ms 2024-06-06T08:03:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:39 smithi195 ceph-mon[37040]: pgmap v496: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-06-06T08:03:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:39 smithi181 ceph-mon[36790]: pgmap v496: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-06-06T08:03:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:39 smithi087 ceph-mon[32276]: pgmap v496: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 -- 172.21.15.87:0/287150810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd81008d0 msgr2=0x7fdbd8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/287150810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd81008d0 0x7fdbd8100d50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009930 tx=0x7fdbc002a420 comp rx=0 tx=0).stop 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 -- 172.21.15.87:0/287150810 shutdown_connections 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/287150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd8101290 0x7fdbd810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/287150810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd81008d0 0x7fdbd8100d50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/287150810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd8106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 -- 172.21.15.87:0/287150810 >> 172.21.15.87:0/287150810 conn(0x7fdbd8077a20 msgr2=0x7fdbd8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 -- 172.21.15.87:0/287150810 shutdown_connections 2024-06-06T08:03:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.945+0000 7fdbdd86b640 1 -- 172.21.15.87:0/287150810 wait complete. 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 Processor -- start 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 -- start start 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 0x7fdbd8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd81a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8110c60 con 0x7fdbd8101290 2024-06-06T08:03:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8110ae0 con 0x7fdbd8106890 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.946+0000 7fdbdd86b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8110de0 con 0x7fdbd81008d0 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 0x7fdbd8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 0x7fdbd8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:50974/0 (socket says 172.21.15.87:50974) 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/3190614341 learned_addr learned my addr 172.21.15.87:0/3190614341 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46278/0 (socket says 172.21.15.87:46278) 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd81a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 -- 172.21.15.87:0/3190614341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 msgr2=0x7fdbd8199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 0x7fdbd8199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 -- 172.21.15.87:0/3190614341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 msgr2=0x7fdbd81a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd81a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 -- 172.21.15.87:0/3190614341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81a4640 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd81a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005940 tx=0x7fdbc00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.947+0000 7fdbdc869640 1 -- 172.21.15.87:0/3190614341 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc003f070 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.948+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81a4810 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.948+0000 7fdbdc869640 1 -- 172.21.15.87:0/3190614341 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc00044c0 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.948+0000 7fdbdc869640 1 -- 172.21.15.87:0/3190614341 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0035070 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.948+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81a4c70 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.950+0000 7fdbdc869640 1 -- 172.21.15.87:0/3190614341 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdbc0043420 con 0x7fdbd8101290 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.950+0000 7fdbe0579740 1 Processor -- start 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.950+0000 7fdbe0579740 1 -- start start 2024-06-06T08:03:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc514550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5561fc4784f0 con 0x5561fc51f350 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561fc416870 con 0x5561fc514170 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbe0579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561fc4744b0 con 0x5561fc514a90 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:50988/0 (socket says 172.21.15.87:50988) 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46284/0 (socket says 172.21.15.87:46284) 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc514550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/759777578 learned_addr learned my addr 172.21.15.87:0/759777578 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 -- 172.21.15.87:0/759777578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 msgr2=0x5561fc51ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 -- 172.21.15.87:0/759777578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 msgr2=0x5561fc514550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc514550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 -- 172.21.15.87:0/759777578 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fc4329b0 con 0x5561fc51f350 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbdc869640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 0x7fdbc40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.951+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ed000 tx=0x7fdbc00f3f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1348f5396893a48e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.952+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 0x7fdbc40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.952+0000 7fdbdc869640 1 -- 172.21.15.87:0/3190614341 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fdbc00efc80 con 0x7fdbd8101290 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.952+0000 7fdbcdffb640 1 -- 172.21.15.87:0/759777578 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00b8120 con 0x5561fc51f350 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.952+0000 7fdbcdffb640 1 -- 172.21.15.87:0/759777578 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc00bbe80 con 0x5561fc51f350 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.952+0000 7fdbcdffb640 1 -- 172.21.15.87:0/759777578 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00bb7e0 con 0x5561fc51f350 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- 172.21.15.87:0/759777578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 msgr2=0x5561fc521740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 0x7fdbc40a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005fd0 tx=0x7fdbd00096a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ed000 tx=0x7fdbc00f3f50 comp rx=0 tx=0).stop 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- 172.21.15.87:0/759777578 shutdown_connections 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc521740 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 --2- 172.21.15.87:0/759777578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc514550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- 172.21.15.87:0/759777578 >> 172.21.15.87:0/759777578 conn(0x5561fc510680 msgr2=0x5561fc510a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- 172.21.15.87:0/759777578 shutdown_connections 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- 172.21.15.87:0/759777578 wait complete. 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.953+0000 7fdbe0579740 1 -- start start 2024-06-06T08:03:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 Processor -- start 2024-06-06T08:03:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5561fc521d40 con 0x5561fc514170 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561fc401ae0 con 0x5561fc51f350 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561fc521ec0 con 0x5561fc514a90 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46298/0 (socket says 172.21.15.87:46298) 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/140013288 learned_addr learned my addr 172.21.15.87:0/140013288 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbe0579740 10 client.0 _ll_register_callbacks cb 0x5561fc43a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 -- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 msgr2=0x5561fc51ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 -- 172.21.15.87:0/140013288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 msgr2=0x5561fc51c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbce7fc640 1 -- 172.21.15.87:0/140013288 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fc18c210 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.954+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51c8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5561fc511810 tx=0x7fdbc00ecf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d67f754139ed047 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0048070 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fdbc003b460 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc00bbe80 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdbc00bb9c0 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbe0579740 10 client.26878 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5561fc258620 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.955+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc00bbe80 con 0x5561fc514a90 2024-06-06T08:03:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fdbc00d1070 con 0x5561fc514a90 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 10 client.26878 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 20 client.26878 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc52ef20 0x5561fc531310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 4 client.26878 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561fc48bf10 con 0x5561fc52ef20 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc52ef20 0x5561fc531310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbb6ffd640 10 client.26878 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.956+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc52ef20 0x5561fc531310 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3b633156f0743a server_cookie=2cf8dc4ad8aac01c in_seq=0 out_seq=0 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.959+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/140013288 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561fc48bf10 con 0x5561fc52ef20 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.959+0000 7fdbb6ffd640 10 client.26878 handle_command_reply: tid=0 2024-06-06T08:03:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.959+0000 7fdbe0579740 1 client.26878 shutdown 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 msgr2=0x5561fc51c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51c8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5561fc511810 tx=0x7fdbc00ecf70 comp rx=0 tx=0).stop 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 shutdown_connections 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc52ef20 0x5561fc531310 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc51c8d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbb6ffd640 0 client.26878 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 --2- 172.21.15.87:0/140013288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc51c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 >> 172.21.15.87:0/140013288 conn(0x5561fc510680 msgr2=0x5561fc5163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 shutdown_connections 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 1 -- 172.21.15.87:0/140013288 wait complete. 2024-06-06T08:03:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.960+0000 7fdbe0579740 20 client.26878 trim_cache size 0 max 16384 2024-06-06T08:03:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 Processor -- start 2024-06-06T08:03:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 -- start start 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc510a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b1750 con 0x5561fc514170 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81aefc0 con 0x5561fc51f350 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.980+0000 7fdbe0579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ace10 con 0x5561fc514a90 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46306/0 (socket says 172.21.15.87:46306) 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/2318456279 learned_addr learned my addr 172.21.15.87:0/2318456279 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc510a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/2318456279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 msgr2=0x5561fc510a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc510a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/2318456279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 msgr2=0x5561fc51cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 -- 172.21.15.87:0/2318456279 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fc51b340 con 0x5561fc514170 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005e00 tx=0x7fdbc8012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc599ebbce24e4bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbcd7fa640 1 -- 172.21.15.87:0/2318456279 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8004020 con 0x5561fc514170 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbcd7fa640 1 -- 172.21.15.87:0/2318456279 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc800fd70 con 0x5561fc514170 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.981+0000 7fdbcd7fa640 1 -- 172.21.15.87:0/2318456279 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8003a10 con 0x5561fc514170 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- 172.21.15.87:0/2318456279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 msgr2=0x5561fc50ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005e00 tx=0x7fdbc8012c40 comp rx=0 tx=0).stop 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- 172.21.15.87:0/2318456279 shutdown_connections 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc51f350 0x5561fc51cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc510a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2318456279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc514170 0x5561fc50ff60 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- 172.21.15.87:0/2318456279 >> 172.21.15.87:0/2318456279 conn(0x5561fc510680 msgr2=0x5561fc5163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- 172.21.15.87:0/2318456279 shutdown_connections 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- 172.21.15.87:0/2318456279 wait complete. 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.982+0000 7fdbe0579740 1 -- start start 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 Processor -- start 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc522540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc522a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ad480 con 0x5561fc51f350 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ae8f0 con 0x5561fc514170 2024-06-06T08:03:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81adb50 con 0x5561fc514a90 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbe0579740 10 client.0 _ll_register_callbacks cb 0x7fdbd0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46322/0 (socket says 172.21.15.87:46322) 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 -- 172.21.15.87:0/2529053277 learned_addr learned my addr 172.21.15.87:0/2529053277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbce7fc640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc522a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc522540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 -- 172.21.15.87:0/2529053277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 msgr2=0x5561fc522a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc522a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 -- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 msgr2=0x5561fc522540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc522540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.983+0000 7fdbd77fe640 1 -- 172.21.15.87:0/2529053277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fc597ea0 con 0x5561fc51f350 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbd77fe640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001e420 tx=0x7fdbd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd6dca25305be64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd002e060 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc800fd70 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fdbd0003970 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdbd0035050 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 10 client.17751 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5561fc599190 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc800fd70 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fdbd001d4a0 con 0x5561fc51f350 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 10 client.17751 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 20 client.17751 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc599950 0x5561fc59bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 4 client.17751 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:03:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.984+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561fc485c20 con 0x5561fc599950 2024-06-06T08:03:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.985+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc599950 0x5561fc59bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.985+0000 7fdbb6ffd640 10 client.17751 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:40.985+0000 7fdbd6ffd640 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc599950 0x5561fc59bd70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d8033bb21b463 server_cookie=af44fa2575f44050 in_seq=0 out_seq=0 2024-06-06T08:03:41.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 18762 11.55 MB/sec warmup 103 sec latency 160.233 ms 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.634+0000 7fdbb6ffd640 1 -- 172.21.15.87:0/2529053277 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561fc485c20 con 0x5561fc599950 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.634+0000 7fdbb6ffd640 10 client.17751 handle_command_reply: tid=0 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.634+0000 7fdbe0579740 1 client.17751 shutdown 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 msgr2=0x5561fc597960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdbd001e420 tx=0x7fdbd0004a40 comp rx=0 tx=0).stop 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 shutdown_connections 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561fc599950 0x5561fc59bd70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561fc51f350 0x5561fc597960 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561fc514a90 0x5561fc522a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 --2- 172.21.15.87:0/2529053277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561fc514170 0x5561fc522540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 >> 172.21.15.87:0/2529053277 conn(0x5561fc510680 msgr2=0x5561fc5163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbb6ffd640 0 client.17751 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 shutdown_connections 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 1 -- 172.21.15.87:0/2529053277 wait complete. 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.635+0000 7fdbe0579740 20 client.17751 trim_cache size 0 max 16384 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:03:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 msgr2=0x7fdbc40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 0x7fdbc40a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005fd0 tx=0x7fdbd00096a0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 msgr2=0x7fdbd81995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005940 tx=0x7fdbc00040c0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 shutdown_connections 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.636+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbc40a2da0 0x7fdbc40a5260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8106890 0x7fdbd81a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd8101290 0x7fdbd81995c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 --2- 172.21.15.87:0/3190614341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbd81008d0 0x7fdbd8199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 >> 172.21.15.87:0/3190614341 conn(0x7fdbd8077a20 msgr2=0x7fdbd8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 shutdown_connections 2024-06-06T08:03:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:41.637+0000 7fdbdd86b640 1 -- 172.21.15.87:0/3190614341 wait complete. 2024-06-06T08:03:42.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 18971 11.54 MB/sec warmup 104 sec latency 437.369 ms 2024-06-06T08:03:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:41 smithi195 ceph-mon[37040]: pgmap v497: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 263 op/s 2024-06-06T08:03:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:41 smithi181 ceph-mon[36790]: pgmap v497: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 263 op/s 2024-06-06T08:03:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:41 smithi087 ceph-mon[32276]: pgmap v497: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.5 MiB/s wr, 263 op/s 2024-06-06T08:03:43.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 19294 11.60 MB/sec warmup 105 sec latency 105.028 ms 2024-06-06T08:03:44.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 19757 11.65 MB/sec warmup 106 sec latency 183.333 ms 2024-06-06T08:03:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:43 smithi195 ceph-mon[37040]: pgmap v498: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-06-06T08:03:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:43 smithi181 ceph-mon[36790]: pgmap v498: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-06-06T08:03:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:43 smithi087 ceph-mon[32276]: pgmap v498: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-06-06T08:03:45.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 20254 11.76 MB/sec warmup 107 sec latency 201.830 ms 2024-06-06T08:03:46.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 20813 11.89 MB/sec warmup 108 sec latency 239.250 ms 2024-06-06T08:03:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:45 smithi195 ceph-mon[37040]: pgmap v499: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-06T08:03:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:45 smithi181 ceph-mon[36790]: pgmap v499: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-06T08:03:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:45 smithi087 ceph-mon[32276]: pgmap v499: 129 pgs: 129 active+clean; 317 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-06-06T08:03:47.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 21330 12.16 MB/sec warmup 109 sec latency 190.380 ms 2024-06-06T08:03:48.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 21798 12.26 MB/sec warmup 110 sec latency 218.575 ms 2024-06-06T08:03:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:47 smithi195 ceph-mon[37040]: pgmap v500: 129 pgs: 129 active+clean; 318 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-06T08:03:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:47 smithi181 ceph-mon[36790]: pgmap v500: 129 pgs: 129 active+clean; 318 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-06T08:03:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:47 smithi087 ceph-mon[32276]: pgmap v500: 129 pgs: 129 active+clean; 318 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-06-06T08:03:49.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 22175 12.36 MB/sec warmup 111 sec latency 156.445 ms 2024-06-06T08:03:50.161 INFO:tasks.workunit.client.0.smithi087.stdout: 10 22507 12.42 MB/sec warmup 112 sec latency 63.336 ms 2024-06-06T08:03:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:49 smithi195 ceph-mon[37040]: pgmap v501: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 537 op/s 2024-06-06T08:03:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:49 smithi181 ceph-mon[36790]: pgmap v501: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 537 op/s 2024-06-06T08:03:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:49 smithi087 ceph-mon[32276]: pgmap v501: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.9 MiB/s wr, 537 op/s 2024-06-06T08:03:51.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 22875 12.51 MB/sec warmup 113 sec latency 109.839 ms 2024-06-06T08:03:52.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 23156 12.51 MB/sec warmup 114 sec latency 1037.430 ms 2024-06-06T08:03:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:52 smithi195 ceph-mon[37040]: pgmap v502: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-06-06T08:03:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:52 smithi181 ceph-mon[36790]: pgmap v502: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-06-06T08:03:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:52 smithi087 ceph-mon[32276]: pgmap v502: 129 pgs: 129 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-06-06T08:03:53.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 23658 12.57 MB/sec warmup 115 sec latency 1780.477 ms 2024-06-06T08:03:54.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 24155 12.65 MB/sec warmup 116 sec latency 378.285 ms 2024-06-06T08:03:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:54 smithi195 ceph-mon[37040]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-06-06T08:03:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:54 smithi181 ceph-mon[36790]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-06-06T08:03:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:54 smithi087 ceph-mon[32276]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-06-06T08:03:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 -- 172.21.15.87:0/1342728841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c106890 msgr2=0x7fd71c106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 --2- 172.21.15.87:0/1342728841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c106890 0x7fd71c106c70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd710009930 tx=0x7fd71002a420 comp rx=0 tx=0).stop 2024-06-06T08:03:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 -- 172.21.15.87:0/1342728841 shutdown_connections 2024-06-06T08:03:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 --2- 172.21.15.87:0/1342728841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c101290 0x7fd71c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 --2- 172.21.15.87:0/1342728841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c1008d0 0x7fd71c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 --2- 172.21.15.87:0/1342728841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c106890 0x7fd71c106c70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 -- 172.21.15.87:0/1342728841 >> 172.21.15.87:0/1342728841 conn(0x7fd71c077a20 msgr2=0x7fd71c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 -- 172.21.15.87:0/1342728841 shutdown_connections 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.742+0000 7fd723109640 1 -- 172.21.15.87:0/1342728841 wait complete. 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 Processor -- start 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 -- start start 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c101290 0x7fd71c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c106890 0x7fd71c1a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c110ca0 con 0x7fd71c1008d0 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c110b20 con 0x7fd71c101290 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd723109640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c110e20 con 0x7fd71c106890 2024-06-06T08:03:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37574/0 (socket says 172.21.15.87:37574) 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd720ea5640 1 -- 172.21.15.87:0/1602757100 learned_addr learned my addr 172.21.15.87:0/1602757100 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd71bfff640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c101290 0x7fd71c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.743+0000 7fd7216a6640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c106890 0x7fd71c1a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 -- 172.21.15.87:0/1602757100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c106890 msgr2=0x7fd71c1a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c106890 0x7fd71c1a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 -- 172.21.15.87:0/1602757100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c101290 msgr2=0x7fd71c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c101290 0x7fd71c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 -- 172.21.15.87:0/1602757100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c1a45c0 con 0x7fd71c1008d0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd710009230 tx=0x7fd710004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd719ffb640 1 -- 172.21.15.87:0/1602757100 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71003f030 con 0x7fd71c1008d0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd719ffb640 1 -- 172.21.15.87:0/1602757100 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd710004440 con 0x7fd71c1008d0 2024-06-06T08:03:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c1a4790 con 0x7fd71c1008d0 2024-06-06T08:03:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd719ffb640 1 -- 172.21.15.87:0/1602757100 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710035070 con 0x7fd71c1008d0 2024-06-06T08:03:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.744+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c1a4b90 con 0x7fd71c1008d0 2024-06-06T08:03:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.747+0000 7fd719ffb640 1 -- 172.21.15.87:0/1602757100 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd710042030 con 0x7fd71c1008d0 2024-06-06T08:03:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 Processor -- start 2024-06-06T08:03:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 -- start start 2024-06-06T08:03:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd719ffb640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 0x7fd70c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630170 0x55e6dd630550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd5944f0 con 0x55e6dd63b350 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd532870 con 0x55e6dd630a90 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd725e10740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd5904b0 con 0x55e6dd630170 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54882/0 (socket says 172.21.15.87:54882) 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd71bfff640 1 -- 172.21.15.87:0/2012681713 learned_addr learned my addr 172.21.15.87:0/2012681713 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.748+0000 7fd719ffb640 1 -- 172.21.15.87:0/1602757100 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fd7100efb40 con 0x7fd71c1008d0 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd7216a6640 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd7216a6640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 0x7fd70c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd720ea5640 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630170 0x55e6dd630550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 -- 172.21.15.87:0/2012681713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630170 msgr2=0x55e6dd630550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630170 0x55e6dd630550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 -- 172.21.15.87:0/2012681713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 msgr2=0x55e6dd63d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 -- 172.21.15.87:0/2012681713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd54e9b0 con 0x55e6dd630a90 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd71bfff640 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd708009c90 tx=0x7fd708011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b04b714d90e6117 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.749+0000 7fd7216a6640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 0x7fd70c0a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55e6dd62daa0 tx=0x7fd714009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd706ffd640 1 -- 172.21.15.87:0/2012681713 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708023070 con 0x55e6dd630a90 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd706ffd640 1 -- 172.21.15.87:0/2012681713 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd708009e40 con 0x55e6dd630a90 2024-06-06T08:03:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd706ffd640 1 -- 172.21.15.87:0/2012681713 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70801eb00 con 0x55e6dd630a90 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 -- 172.21.15.87:0/2012681713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 msgr2=0x55e6dd63ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd708009c90 tx=0x7fd708011810 comp rx=0 tx=0).stop 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 -- 172.21.15.87:0/2012681713 shutdown_connections 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630a90 0x55e6dd63ae10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 --2- 172.21.15.87:0/2012681713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630170 0x55e6dd630550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.750+0000 7fd725e10740 1 -- 172.21.15.87:0/2012681713 >> 172.21.15.87:0/2012681713 conn(0x55e6dd62c680 msgr2=0x55e6dd62ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.751+0000 7fd725e10740 1 -- 172.21.15.87:0/2012681713 shutdown_connections 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.751+0000 7fd725e10740 1 -- 172.21.15.87:0/2012681713 wait complete. 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.751+0000 7fd725e10740 1 -- start start 2024-06-06T08:03:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.751+0000 7fd725e10740 1 Processor -- start 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd638900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd638e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd63df80 con 0x55e6dd63b350 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd51dae0 con 0x55e6dd630170 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd63e100 con 0x55e6dd630a90 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd725e10740 10 client.0 _ll_register_callbacks cb 0x55e6dd5569b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd7216a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd638e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54890/0 (socket says 172.21.15.87:54890) 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 -- 172.21.15.87:0/1966575021 learned_addr learned my addr 172.21.15.87:0/1966575021 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd71bfff640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd638900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 -- 172.21.15.87:0/1966575021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 msgr2=0x55e6dd638900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd638900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 -- 172.21.15.87:0/1966575021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 msgr2=0x55e6dd638e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd638e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 -- 172.21.15.87:0/1966575021 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd2a8210 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd7216a6640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd638e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd720ea5640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd71004a000 tx=0x7fd7100bb940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74dae55ccaa6a3fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.752+0000 7fd6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710049040 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fd7100b83b0 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e6dd5a7f10 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd7100c0040 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd725e10740 10 client.26756 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.753+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e6dd374620 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7100c4900 con 0x55e6dd630170 2024-06-06T08:03:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fd7100bb190 con 0x55e6dd630170 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 10 client.26756 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 10 client.26756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 20 client.26756 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd64b5a0 0x55e6dd64d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 4 client.26756 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6dd5a7f10 con 0x55e6dd64b5a0 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.754+0000 7fd71bfff640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd64b5a0 0x55e6dd64d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.755+0000 7fd71bfff640 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd64b5a0 0x55e6dd64d9b0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec9a69cf68dd7cc server_cookie=d1a5ac1260f49c62 in_seq=0 out_seq=0 2024-06-06T08:03:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.755+0000 7fd6fbfff640 10 client.26756 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.756+0000 7fd6fbfff640 1 -- 172.21.15.87:0/1966575021 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e6dd5a7f10 con 0x55e6dd64b5a0 2024-06-06T08:03:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.756+0000 7fd6fbfff640 10 client.26756 handle_command_reply: tid=0 2024-06-06T08:03:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.756+0000 7fd725e10740 1 client.26756 shutdown 2024-06-06T08:03:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 msgr2=0x55e6dd6383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd71004a000 tx=0x7fd7100bb940 comp rx=0 tx=0).stop 2024-06-06T08:03:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 shutdown_connections 2024-06-06T08:03:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd64b5a0 0x55e6dd64d9b0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd638e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd638900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 --2- 172.21.15.87:0/1966575021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6383c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd6fbfff640 0 client.26756 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 >> 172.21.15.87:0/1966575021 conn(0x55e6dd62c680 msgr2=0x55e6dd6323b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 shutdown_connections 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 1 -- 172.21.15.87:0/1966575021 wait complete. 2024-06-06T08:03:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.757+0000 7fd725e10740 20 client.26756 trim_cache size 0 max 16384 2024-06-06T08:03:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.775+0000 7fd725e10740 1 Processor -- start 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 -- start start 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd653800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd63b350 0x55e6dd631b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1af500 con 0x55e6dd630170 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1acd70 con 0x55e6dd63b350 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd725e10740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1aabc0 con 0x55e6dd630a90 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37596/0 (socket says 172.21.15.87:37596) 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 -- 172.21.15.87:0/3881267493 learned_addr learned my addr 172.21.15.87:0/3881267493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd71bfff640 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd653800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd7216a6640 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd63b350 0x55e6dd631b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 -- 172.21.15.87:0/3881267493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 msgr2=0x55e6dd653800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd653800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 -- 172.21.15.87:0/3881267493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd63b350 msgr2=0x55e6dd631b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd63b350 0x55e6dd631b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.776+0000 7fd720ea5640 1 -- 172.21.15.87:0/3881267493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd6471e0 con 0x55e6dd630170 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.777+0000 7fd720ea5640 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd710009a50 tx=0x7fd71003b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e29a9c409ec077d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.777+0000 7fd7067fc640 1 -- 172.21.15.87:0/3881267493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710049070 con 0x55e6dd630170 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.777+0000 7fd7067fc640 1 -- 172.21.15.87:0/3881267493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7100c4900 con 0x55e6dd630170 2024-06-06T08:03:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.777+0000 7fd7067fc640 1 -- 172.21.15.87:0/3881267493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100b83b0 con 0x55e6dd630170 2024-06-06T08:03:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- 172.21.15.87:0/3881267493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 msgr2=0x55e6dd6532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd710009a50 tx=0x7fd71003b560 comp rx=0 tx=0).stop 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- 172.21.15.87:0/3881267493 shutdown_connections 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd63b350 0x55e6dd631b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd653800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 --2- 172.21.15.87:0/3881267493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd630170 0x55e6dd6532c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- 172.21.15.87:0/3881267493 >> 172.21.15.87:0/3881267493 conn(0x55e6dd62c680 msgr2=0x55e6dd633670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- 172.21.15.87:0/3881267493 shutdown_connections 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- 172.21.15.87:0/3881267493 wait complete. 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 -- start start 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.778+0000 7fd725e10740 1 Processor -- start 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd6aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1ab230 con 0x55e6dd63b350 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1ac6a0 con 0x55e6dd630170 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c1ab900 con 0x55e6dd630a90 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd725e10740 10 client.0 _ll_register_callbacks cb 0x7fd710009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd6aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37610/0 (socket says 172.21.15.87:37610) 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd720ea5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54920/0 (socket says 172.21.15.87:54920) 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd6aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:45496/0 (socket says 172.21.15.87:45496) 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 -- 172.21.15.87:0/562029394 learned_addr learned my addr 172.21.15.87:0/562029394 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 -- 172.21.15.87:0/562029394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 msgr2=0x55e6dd6aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd6aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 -- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 msgr2=0x55e6dd6ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 -- 172.21.15.87:0/562029394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd63f050 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd720ea5640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd7216a6640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd714009fd0 tx=0x7fd714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258baaced2c7b500 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.779+0000 7fd6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714004d20 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9495+0+0 (secure 0 0 0) 0x7fd71400e990 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7100c4900 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd714036050 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 10 client.17769 fetch_fsmap learned FSMap version 48 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e6dd6b5de0 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7100c4900 con 0x55e6dd63b350 2024-06-06T08:03:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fd71401d070 con 0x55e6dd63b350 2024-06-06T08:03:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 10 client.17769 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:03:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:03:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 20 client.17769 populate_metadata read hostname 'smithi087' 2024-06-06T08:03:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd6b6610 0x55e6dd6b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:03:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.780+0000 7fd725e10740 4 client.17769 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:03:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.781+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6dd5a1c20 con 0x55e6dd6b6610 2024-06-06T08:03:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.781+0000 7fd720ea5640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd6b6610 0x55e6dd6b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:03:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.781+0000 7fd720ea5640 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd6b6610 0x55e6dd6b8a30 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31cb6966fbbce326 server_cookie=784440f0b0f7dc46 in_seq=0 out_seq=0 2024-06-06T08:03:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:54.781+0000 7fd6fbfff640 10 client.17769 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:55.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 24504 12.72 MB/sec warmup 117 sec latency 372.568 ms 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd6fbfff640 1 -- 172.21.15.87:0/562029394 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e6dd5a1c20 con 0x55e6dd6b6610 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd6fbfff640 10 client.17769 handle_command_reply: tid=0 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd725e10740 1 client.17769 shutdown 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 msgr2=0x55e6dd63e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd714009fd0 tx=0x7fd714004a40 comp rx=0 tx=0).stop 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.478+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 shutdown_connections 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e6dd6b6610 0x55e6dd6b8a30 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd6fbfff640 0 client.17769 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e6dd63b350 0x55e6dd63e910 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6dd630a90 0x55e6dd6aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 --2- 172.21.15.87:0/562029394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6dd630170 0x55e6dd6ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 >> 172.21.15.87:0/562029394 conn(0x55e6dd62c680 msgr2=0x55e6dd6461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 shutdown_connections 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 1 -- 172.21.15.87:0/562029394 wait complete. 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.479+0000 7fd725e10740 20 client.17769 trim_cache size 0 max 16384 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:03:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:03:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 msgr2=0x7fd70c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 0x7fd70c0a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55e6dd62daa0 tx=0x7fd714009500 comp rx=0 tx=0).stop 2024-06-06T08:03:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 msgr2=0x7fd71c199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd710009230 tx=0x7fd710004040 comp rx=0 tx=0).stop 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 shutdown_connections 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd70c0a2da0 0x7fd70c0a5260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71c106890 0x7fd71c1a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd71c101290 0x7fd71c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 --2- 172.21.15.87:0/1602757100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd71c1008d0 0x7fd71c199000 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 >> 172.21.15.87:0/1602757100 conn(0x7fd71c077a20 msgr2=0x7fd71c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 shutdown_connections 2024-06-06T08:03:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:03:55.480+0000 7fd723109640 1 -- 172.21.15.87:0/1602757100 wait complete. 2024-06-06T08:03:56.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 24903 12.85 MB/sec warmup 118 sec latency 668.995 ms 2024-06-06T08:03:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:56 smithi195 ceph-mon[37040]: pgmap v504: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-06-06T08:03:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:56 smithi181 ceph-mon[36790]: pgmap v504: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-06-06T08:03:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:56 smithi087 ceph-mon[32276]: pgmap v504: 129 pgs: 129 active+clean; 340 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-06-06T08:03:57.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 25357 13.01 MB/sec warmup 119 sec latency 151.854 ms 2024-06-06T08:03:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:03:58 smithi181 ceph-mon[36790]: pgmap v505: 129 pgs: 129 active+clean; 333 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-06-06T08:03:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:03:58 smithi087 ceph-mon[32276]: pgmap v505: 129 pgs: 129 active+clean; 333 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-06-06T08:03:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:03:58 smithi195 ceph-mon[37040]: pgmap v505: 129 pgs: 129 active+clean; 333 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-06-06T08:03:59.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 26118 19.96 MB/sec execute 1 sec latency 83.407 ms 2024-06-06T08:04:00.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 26407 17.10 MB/sec execute 2 sec latency 206.546 ms 2024-06-06T08:04:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:00 smithi181 ceph-mon[36790]: pgmap v506: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.6 MiB/s wr, 542 op/s 2024-06-06T08:04:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:00 smithi087 ceph-mon[32276]: pgmap v506: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.6 MiB/s wr, 542 op/s 2024-06-06T08:04:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:00 smithi195 ceph-mon[37040]: pgmap v506: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.6 MiB/s wr, 542 op/s 2024-06-06T08:04:01.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 26680 15.37 MB/sec execute 3 sec latency 249.692 ms 2024-06-06T08:04:02.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 26862 13.42 MB/sec execute 4 sec latency 302.978 ms 2024-06-06T08:04:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:02 smithi181 ceph-mon[36790]: pgmap v507: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-06-06T08:04:02.311 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:02 smithi195 ceph-mon[37040]: pgmap v507: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-06-06T08:04:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:02 smithi087 ceph-mon[32276]: pgmap v507: 129 pgs: 129 active+clean; 334 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.6 MiB/s wr, 447 op/s 2024-06-06T08:04:03.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 27127 12.26 MB/sec execute 5 sec latency 356.743 ms 2024-06-06T08:04:04.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 27471 13.28 MB/sec execute 6 sec latency 468.990 ms 2024-06-06T08:04:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:04 smithi181 ceph-mon[36790]: pgmap v508: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-06-06T08:04:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:04 smithi087 ceph-mon[32276]: pgmap v508: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-06-06T08:04:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:04 smithi195 ceph-mon[37040]: pgmap v508: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-06-06T08:04:05.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 27885 14.82 MB/sec execute 7 sec latency 452.067 ms 2024-06-06T08:04:06.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28329 16.53 MB/sec execute 8 sec latency 107.553 ms 2024-06-06T08:04:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:06 smithi181 ceph-mon[36790]: pgmap v509: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 498 op/s 2024-06-06T08:04:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:06 smithi087 ceph-mon[32276]: pgmap v509: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 498 op/s 2024-06-06T08:04:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:06 smithi195 ceph-mon[37040]: pgmap v509: 129 pgs: 129 active+clean; 348 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 498 op/s 2024-06-06T08:04:07.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28503 15.70 MB/sec execute 9 sec latency 180.087 ms 2024-06-06T08:04:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:07 smithi181 ceph-mon[36790]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T08:04:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:07 smithi087 ceph-mon[32276]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T08:04:07.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:07 smithi195 ceph-mon[37040]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T08:04:08.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28754 16.01 MB/sec execute 10 sec latency 118.990 ms 2024-06-06T08:04:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:08 smithi181 ceph-mon[36790]: pgmap v511: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 353 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 547 op/s 2024-06-06T08:04:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:08 smithi087 ceph-mon[32276]: pgmap v511: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 353 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 547 op/s 2024-06-06T08:04:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:08 smithi195 ceph-mon[37040]: pgmap v511: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 353 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 547 op/s 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 -- 172.21.15.87:0/1345844539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b8105330 msgr2=0x7f08b810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/1345844539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b8105330 0x7f08b810cf00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009be0 tx=0x7f08b002a6c0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 -- 172.21.15.87:0/1345844539 shutdown_connections 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/1345844539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b8105330 0x7f08b810cf00 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/1345844539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8100c10 0x7f08b8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/1345844539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 -- 172.21.15.87:0/1345844539 >> 172.21.15.87:0/1345844539 conn(0x7f08b8076e30 msgr2=0x7f08b80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 -- 172.21.15.87:0/1345844539 shutdown_connections 2024-06-06T08:04:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 -- 172.21.15.87:0/1345844539 wait complete. 2024-06-06T08:04:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.591+0000 7f08bf3ee640 1 Processor -- start 2024-06-06T08:04:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 -- start start 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8105330 0x7f08b8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 0x7f08b819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd98b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 0x7f08b819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:44278/0 (socket says 172.21.15.87:44278) 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 -- 172.21.15.87:0/3634942018 learned_addr learned my addr 172.21.15.87:0/3634942018 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110c10 con 0x7f08b819ca90 2024-06-06T08:04:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110a90 con 0x7f08b8105330 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110d90 con 0x7f08b8100260 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bc989640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8105330 0x7f08b8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 -- 172.21.15.87:0/3634942018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8105330 msgr2=0x7f08b8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8105330 0x7f08b8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 -- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 msgr2=0x7f08b819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 0x7f08b819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.592+0000 7f08bd18a640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b819d580 con 0x7f08b8100260 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 0x7f08b819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08b400e9c0 tx=0x7f08b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08ae7fc640 1 -- 172.21.15.87:0/3634942018 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b400cde0 con 0x7f08b8100260 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b819d870 con 0x7f08b8100260 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81a50b0 con 0x7f08b8100260 2024-06-06T08:04:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08ae7fc640 1 -- 172.21.15.87:0/3634942018 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b40035f0 con 0x7f08b8100260 2024-06-06T08:04:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.593+0000 7f08ae7fc640 1 -- 172.21.15.87:0/3634942018 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4010690 con 0x7f08b8100260 2024-06-06T08:04:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.596+0000 7f08ae7fc640 1 -- 172.21.15.87:0/3634942018 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f08b4004020 con 0x7f08b8100260 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 Processor -- start 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 -- start start 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd399b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558fd38ff4f0 con 0x558fd399ba90 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558fd389d870 con 0x558fd399b170 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.597+0000 7f08c21db740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558fd38fb4b0 con 0x558fd39a6350 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39274/0 (socket says 172.21.15.87:39274) 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bd18a640 1 -- 172.21.15.87:0/925374158 learned_addr learned my addr 172.21.15.87:0/925374158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08ae7fc640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 0x7f08a40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 -- 172.21.15.87:0/925374158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 msgr2=0x558fd399b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd399b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 -- 172.21.15.87:0/925374158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 msgr2=0x558fd39a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 -- 172.21.15.87:0/925374158 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd38b99b0 con 0x558fd39a6350 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bd18a640 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 0x7f08a40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.598+0000 7f08bc989640 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a8740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08b8074120 tx=0x7f08a8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f05723121377b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.599+0000 7f08ae7fc640 1 -- 172.21.15.87:0/3634942018 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7f08b40cfbb0 con 0x7f08b8100260 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.599+0000 7f08a37fe640 1 -- 172.21.15.87:0/925374158 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8011da0 con 0x558fd39a6350 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.599+0000 7f08a37fe640 1 -- 172.21.15.87:0/925374158 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8016070 con 0x558fd39a6350 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.599+0000 7f08a37fe640 1 -- 172.21.15.87:0/925374158 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a801b470 con 0x558fd39a6350 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- 172.21.15.87:0/925374158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 msgr2=0x558fd39a8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 0x7f08a40a52e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08b00092f0 tx=0x7f08b0009280 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a8740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08b8074120 tx=0x7f08a8009f50 comp rx=0 tx=0).stop 2024-06-06T08:04:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- 172.21.15.87:0/925374158 shutdown_connections 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a8740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 --2- 172.21.15.87:0/925374158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd399b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- 172.21.15.87:0/925374158 >> 172.21.15.87:0/925374158 conn(0x558fd3997680 msgr2=0x558fd3997a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- 172.21.15.87:0/925374158 shutdown_connections 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- 172.21.15.87:0/925374158 wait complete. 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 -- start start 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.600+0000 7f08c21db740 1 Processor -- start 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd39a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558fd39a8f80 con 0x558fd399ba90 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558fd3888ae0 con 0x558fd399b170 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558fd39a9100 con 0x558fd39a6350 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39290/0 (socket says 172.21.15.87:39290) 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 -- 172.21.15.87:0/2160582626 learned_addr learned my addr 172.21.15.87:0/2160582626 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bd98b640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08c21db740 10 client.0 _ll_register_callbacks cb 0x558fd38c19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bd18a640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd39a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 -- 172.21.15.87:0/2160582626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 msgr2=0x558fd39a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 -- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 msgr2=0x558fd39a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd39a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bc989640 1 -- 172.21.15.87:0/2160582626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd3613210 con 0x558fd399ba90 2024-06-06T08:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.601+0000 7f08bd18a640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd39a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08bc989640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005760 tx=0x7f08a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceb7d049d0d7ad4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8004250 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7f08a80043f0 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a8016070 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f08a8031050 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08c21db740 10 client.17781 fetch_fsmap learned FSMap version 48 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.602+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558fd36df620 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8016070 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f08a8012450 con 0x558fd399ba90 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 10 client.17781 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:04:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 10 client.17781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 20 client.17781 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd39b8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 4 client.17781 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fd3912f10 con 0x558fd39b6460 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.603+0000 7f08bd18a640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd39b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.604+0000 7f08bd18a640 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd39b8850 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a304368ec7256518 server_cookie=266eed1a5f65823c in_seq=0 out_seq=0 2024-06-06T08:04:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.604+0000 7f08a0ff9640 10 client.17781 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08a0ff9640 1 -- 172.21.15.87:0/2160582626 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558fd3912f10 con 0x558fd39b6460 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08a0ff9640 10 client.17781 handle_command_reply: tid=0 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 client.17781 shutdown 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 msgr2=0x558fd39a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005760 tx=0x7f08a8005850 comp rx=0 tx=0).stop 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 shutdown_connections 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd39b8850 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd39a6350 0x558fd39a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399ba90 0x558fd39a3830 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08a0ff9640 0 client.17781 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 --2- 172.21.15.87:0/2160582626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399b170 0x558fd39a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 >> 172.21.15.87:0/2160582626 conn(0x558fd3997680 msgr2=0x558fd399d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 shutdown_connections 2024-06-06T08:04:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 1 -- 172.21.15.87:0/2160582626 wait complete. 2024-06-06T08:04:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.609+0000 7f08c21db740 20 client.17781 trim_cache size 0 max 16384 2024-06-06T08:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.626+0000 7f08c21db740 1 Processor -- start 2024-06-06T08:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 -- start start 2024-06-06T08:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399ba90 0x558fd39bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd39a6350 0x558fd39be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f089c012c50 con 0x558fd399b170 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f089c0104c0 con 0x558fd39a6350 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08c21db740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f089c00e330 con 0x558fd399ba90 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39300/0 (socket says 172.21.15.87:39300) 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 -- 172.21.15.87:0/1533494552 learned_addr learned my addr 172.21.15.87:0/1533494552 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bc989640 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399ba90 0x558fd39bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd98b640 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd39a6350 0x558fd39be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 -- 172.21.15.87:0/1533494552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399ba90 msgr2=0x558fd39bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399ba90 0x558fd39bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 -- 172.21.15.87:0/1533494552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd39a6350 msgr2=0x558fd39be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd39a6350 0x558fd39be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 -- 172.21.15.87:0/1533494552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd3997a60 con 0x558fd399b170 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.627+0000 7f08bd18a640 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f08b4098050 tx=0x7f08b4097fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad052c2da95264e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.628+0000 7f08a2ffd640 1 -- 172.21.15.87:0/1533494552 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b40b2070 con 0x558fd399b170 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.628+0000 7f08a2ffd640 1 -- 172.21.15.87:0/1533494552 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b409c2f0 con 0x558fd399b170 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.628+0000 7f08a2ffd640 1 -- 172.21.15.87:0/1533494552 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4098260 con 0x558fd399b170 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.628+0000 7f08c21db740 1 -- 172.21.15.87:0/1533494552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 msgr2=0x558fd39a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.628+0000 7f08c21db740 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f08b4098050 tx=0x7f08b4097fb0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 -- 172.21.15.87:0/1533494552 shutdown_connections 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd39a6350 0x558fd39be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399ba90 0x558fd39bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 --2- 172.21.15.87:0/1533494552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd399b170 0x558fd39a39c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 -- 172.21.15.87:0/1533494552 >> 172.21.15.87:0/1533494552 conn(0x558fd3997680 msgr2=0x558fd39a32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 -- 172.21.15.87:0/1533494552 shutdown_connections 2024-06-06T08:04:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 -- 172.21.15.87:0/1533494552 wait complete. 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 -- start start 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.629+0000 7f08c21db740 1 Processor -- start 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399ba90 0x558fd39aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 0x558fd399fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd18a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:44310/0 (socket says 172.21.15.87:44310) 2024-06-06T08:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd18a640 1 -- 172.21.15.87:0/3429339739 learned_addr learned my addr 172.21.15.87:0/3429339739 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f089c00e980 con 0x558fd39a6350 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f089c00fdf0 con 0x558fd399ba90 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f089c00f050 con 0x558fd399b170 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08c21db740 10 client.0 _ll_register_callbacks cb 0x7f08b40d3530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 0x558fd399fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 -- 172.21.15.87:0/3429339739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 msgr2=0x558fd39ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 -- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399ba90 msgr2=0x558fd39aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399ba90 0x558fd39aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd98b640 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd39a06a0 con 0x558fd39a6350 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.630+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08bd98b640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 0x558fd399fe70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b0003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2180cd809864ce4f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0044070 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b409c2f0 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7f08b0004790 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f08b0036050 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08c21db740 10 client.17787 fetch_fsmap learned FSMap version 48 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558fd3a20c50 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0051070 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f08b003c040 con 0x558fd39a6350 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08c21db740 10 client.17787 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.631+0000 7f08c21db740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08c21db740 20 client.17787 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd3a233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08c21db740 4 client.17787 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f08b409c2f0 con 0x558fd39b6460 2024-06-06T08:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd3a233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08bd18a640 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd3a233c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0eabd6bc2d7521 server_cookie=72779039455edac8 in_seq=0 out_seq=0 2024-06-06T08:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:08.632+0000 7f08a0ff9640 10 client.17787 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:09.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28949 15.59 MB/sec execute 11 sec latency 381.360 ms 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.951+0000 7f08a0ff9640 1 -- 172.21.15.87:0/3429339739 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f08b409c2f0 con 0x558fd39b6460 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.951+0000 7f08a0ff9640 10 client.17787 handle_command_reply: tid=0 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.951+0000 7f08c21db740 1 client.17787 shutdown 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 msgr2=0x558fd399fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 0x558fd399fe70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b0003bf0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 shutdown_connections 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558fd39b6460 0x558fd3a233c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558fd39a6350 0x558fd399fe70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fd399ba90 0x558fd39aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 --2- 172.21.15.87:0/3429339739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558fd399b170 0x558fd39ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 >> 172.21.15.87:0/3429339739 conn(0x558fd3997680 msgr2=0x558fd39b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08a0ff9640 0 client.17787 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 shutdown_connections 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 1 -- 172.21.15.87:0/3429339739 wait complete. 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.952+0000 7f08c21db740 20 client.17787 trim_cache size 0 max 16384 2024-06-06T08:04:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:04:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:04:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:04:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 msgr2=0x7f08a40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 0x7f08a40a52e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08b00092f0 tx=0x7f08b0009280 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 msgr2=0x7f08b81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08b400e9c0 tx=0x7f08b400ee80 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 shutdown_connections 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f08a40a2e20 0x7f08a40a52e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b819ca90 0x7f08b819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08b8105330 0x7f08b8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.953+0000 7f08bf3ee640 1 --2- 172.21.15.87:0/3634942018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8100260 0x7f08b81981d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.954+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 >> 172.21.15.87:0/3634942018 conn(0x7f08b8076e30 msgr2=0x7f08b810daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.954+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 shutdown_connections 2024-06-06T08:04:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:09.954+0000 7f08bf3ee640 1 -- 172.21.15.87:0/3634942018 wait complete. 2024-06-06T08:04:10.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28963 14.32 MB/sec execute 12 sec latency 1029.369 ms 2024-06-06T08:04:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:10 smithi181 ceph-mon[36790]: pgmap v512: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:10 smithi087 ceph-mon[32276]: pgmap v512: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:10 smithi195 ceph-mon[37040]: pgmap v512: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:11.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28968 13.22 MB/sec execute 13 sec latency 1473.555 ms 2024-06-06T08:04:12.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 28994 12.38 MB/sec execute 14 sec latency 2114.033 ms 2024-06-06T08:04:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:12 smithi181 ceph-mon[36790]: pgmap v513: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:12 smithi087 ceph-mon[32276]: pgmap v513: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:12.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:12 smithi195 ceph-mon[37040]: pgmap v513: 129 pgs: 21 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 53 active+clean; 358 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 474 op/s 2024-06-06T08:04:13.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 29304 12.87 MB/sec execute 15 sec latency 480.075 ms 2024-06-06T08:04:14.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 29515 12.68 MB/sec execute 16 sec latency 97.401 ms 2024-06-06T08:04:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:14 smithi087 ceph-mon[32276]: pgmap v514: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:14 smithi195 ceph-mon[37040]: pgmap v514: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:14 smithi181 ceph-mon[36790]: pgmap v514: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:15.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 29873 13.13 MB/sec execute 17 sec latency 137.517 ms 2024-06-06T08:04:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:15 smithi087 ceph-mon[32276]: pgmap v515: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:16.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:15 smithi087 ceph-mon[32276]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T08:04:16.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 30141 12.71 MB/sec execute 18 sec latency 173.366 ms 2024-06-06T08:04:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:15 smithi195 ceph-mon[37040]: pgmap v515: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:15 smithi195 ceph-mon[37040]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T08:04:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:15 smithi181 ceph-mon[36790]: pgmap v515: 129 pgs: 129 active+clean; 423 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-06-06T08:04:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:15 smithi181 ceph-mon[36790]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T08:04:17.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 30528 13.02 MB/sec execute 19 sec latency 467.457 ms 2024-06-06T08:04:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:17 smithi087 ceph-mon[32276]: pgmap v517: 129 pgs: 129 active+clean; 410 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 877 op/s 2024-06-06T08:04:18.162 INFO:tasks.workunit.client.0.smithi087.stdout: 10 31016 13.76 MB/sec execute 20 sec latency 1467.484 ms 2024-06-06T08:04:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:17 smithi195 ceph-mon[37040]: pgmap v517: 129 pgs: 129 active+clean; 410 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 877 op/s 2024-06-06T08:04:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:17 smithi181 ceph-mon[36790]: pgmap v517: 129 pgs: 129 active+clean; 410 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 877 op/s 2024-06-06T08:04:19.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 31535 14.58 MB/sec execute 21 sec latency 2467.511 ms 2024-06-06T08:04:20.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 32012 15.10 MB/sec execute 22 sec latency 3467.541 ms 2024-06-06T08:04:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:19 smithi195 ceph-mon[37040]: pgmap v518: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:19 smithi181 ceph-mon[36790]: pgmap v518: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:19 smithi087 ceph-mon[32276]: pgmap v518: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:21.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 32507 15.63 MB/sec execute 23 sec latency 4467.569 ms 2024-06-06T08:04:22.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:21 smithi087 ceph-mon[32276]: pgmap v519: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:22.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 32936 16.31 MB/sec execute 24 sec latency 5467.605 ms 2024-06-06T08:04:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:21 smithi195 ceph-mon[37040]: pgmap v519: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:21 smithi181 ceph-mon[36790]: pgmap v519: 129 pgs: 129 active+clean; 367 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 15 MiB/s wr, 928 op/s 2024-06-06T08:04:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 -- 172.21.15.87:0/2362407767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4106890 msgr2=0x7fc5d4106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 --2- 172.21.15.87:0/2362407767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4106890 0x7fc5d4106c70 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c402a420 comp rx=0 tx=0).stop 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 -- 172.21.15.87:0/2362407767 shutdown_connections 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 --2- 172.21.15.87:0/2362407767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4101290 0x7fc5d410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 --2- 172.21.15.87:0/2362407767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 0x7fc5d4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 --2- 172.21.15.87:0/2362407767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4106890 0x7fc5d4106c70 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 -- 172.21.15.87:0/2362407767 >> 172.21.15.87:0/2362407767 conn(0x7fc5d4077a20 msgr2=0x7fc5d4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 -- 172.21.15.87:0/2362407767 shutdown_connections 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.058+0000 7fc5da831640 1 -- 172.21.15.87:0/2362407767 wait complete. 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 Processor -- start 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 -- start start 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 0x7fc5d41970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4106890 0x7fc5d419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110cf0 con 0x7fc5d4101290 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110b70 con 0x7fc5d41008d0 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5da831640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110e70 con 0x7fc5d4106890 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56186/0 (socket says 172.21.15.87:56186) 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1007251708 learned_addr learned my addr 172.21.15.87:0/1007251708 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 0x7fc5d41970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.059+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4106890 0x7fc5d419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1007251708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4106890 msgr2=0x7fc5d419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4106890 0x7fc5d419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1007251708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 msgr2=0x7fc5d41970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 0x7fc5d41970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1007251708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d419c0a0 con 0x7fc5d4101290 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d17fa640 1 -- 172.21.15.87:0/1007251708 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800cd80 con 0x7fc5d4101290 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d17fa640 1 -- 172.21.15.87:0/1007251708 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8010910 con 0x7fc5d4101290 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d419c390 con 0x7fc5d4101290 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5d17fa640 1 -- 172.21.15.87:0/1007251708 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8018990 con 0x7fc5d4101290 2024-06-06T08:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.060+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a3c20 con 0x7fc5d4101290 2024-06-06T08:04:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.062+0000 7fc5d17fa640 1 -- 172.21.15.87:0/1007251708 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc5c8010ac0 con 0x7fc5d4101290 2024-06-06T08:04:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5d17fa640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 0x7fc5bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 Processor -- start 2024-06-06T08:04:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 -- start start 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 0x7fc5bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5d17fa640 1 -- 172.21.15.87:0/1007251708 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc5c80cfad0 con 0x7fc5d4101290 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc22a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc239950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5560dc18e4f0 con 0x5560dc22a9b0 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5560dc12c870 con 0x5560dc22a090 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5560dc18a4b0 con 0x5560dc237560 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.063+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56192/0 (socket says 172.21.15.87:56192) 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc22a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1320143936 learned_addr learned my addr 172.21.15.87:0/1320143936 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc239950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 0x7fc5bc0a52e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c4000bc0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1320143936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 msgr2=0x5560dc239950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc239950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1320143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 msgr2=0x5560dc22a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc22a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 -- 172.21.15.87:0/1320143936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dc1489b0 con 0x5560dc22a9b0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc22a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8098650 tx=0x7fc5c809bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0c1f8f59aa5d16 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5ba7fc640 1 -- 172.21.15.87:0/1320143936 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80cd080 con 0x5560dc22a9b0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5ba7fc640 1 -- 172.21.15.87:0/1320143936 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c80982f0 con 0x5560dc22a9b0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.064+0000 7fc5ba7fc640 1 -- 172.21.15.87:0/1320143936 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c809b330 con 0x5560dc22a9b0 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1320143936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 msgr2=0x5560dc237020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8098650 tx=0x7fc5c809bd40 comp rx=0 tx=0).stop 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1320143936 shutdown_connections 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc239950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc237020 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1320143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc22a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1320143936 >> 172.21.15.87:0/1320143936 conn(0x5560dc226680 msgr2=0x5560dc226a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1320143936 shutdown_connections 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1320143936 wait complete. 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 -- start start 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.065+0000 7fc5dd61c740 1 Processor -- start 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc234590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 0x5560dc234ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d8dce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d8dce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56196/0 (socket says 172.21.15.87:56196) 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d8dce640 1 -- 172.21.15.87:0/3422849878 learned_addr learned my addr 172.21.15.87:0/3422849878 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5560dc23a190 con 0x5560dc237560 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 0x5560dc234ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5560dc117ae0 con 0x5560dc22a090 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5560dc23a310 con 0x5560dc22a9b0 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc234590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 msgr2=0x5560dc234590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc234590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 10 client.0 _ll_register_callbacks cb 0x5560dc1509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3422849878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 msgr2=0x5560dc235010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dbea2210 con 0x5560dc22a9b0 2024-06-06T08:04:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c80982f0 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 0x5560dc234ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80cceb0 tx=0x7fc5c80a0100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d360cd57a31456b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5b37fe640 10 client.26932 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3422849878 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80ba070 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.066+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3422849878 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc5c809b2e0 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3422849878 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc5c8010430 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 10 client.26932 fetch_fsmap learned FSMap version 48 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=48}) -- 0x5560dbf6e620 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3422849878 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc5c80105d0 con 0x5560dc22a9b0 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 10 client.26932 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 10 client.26932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 20 client.26932 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc247480 0x5560dc249870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 4 client.26932 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:04:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560dc1a1f10 con 0x5560dc247480 2024-06-06T08:04:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.067+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc247480 0x5560dc249870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.068+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc247480 0x5560dc249870 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a717c6007734363b server_cookie=e14a1cf18c3fab89 in_seq=0 out_seq=0 2024-06-06T08:04:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.068+0000 7fc5b37fe640 10 client.26932 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3422849878 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5560dc1a1f10 con 0x5560dc247480 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5b37fe640 10 client.26932 handle_command_reply: tid=0 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 client.26932 shutdown 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 msgr2=0x5560dc234ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 0x5560dc234ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80cceb0 tx=0x7fc5c80a0100 comp rx=0 tx=0).stop 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 shutdown_connections 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc247480 0x5560dc249870 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc235010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a9b0 0x5560dc234ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3422849878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc234590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5b37fe640 0 client.26932 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 >> 172.21.15.87:0/3422849878 conn(0x5560dc226680 msgr2=0x5560dc22c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 shutdown_connections 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3422849878 wait complete. 2024-06-06T08:04:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.071+0000 7fc5dd61c740 20 client.26932 trim_cache size 0 max 16384 2024-06-06T08:04:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.088+0000 7fc5dd61c740 1 Processor -- start 2024-06-06T08:04:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 -- start start 2024-06-06T08:04:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a090 0x5560dc234e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a9b0 0x5560dc225660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41af710 con 0x5560dc237560 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41acf80 con 0x5560dc22a9b0 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41aadf0 con 0x5560dc22a090 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56212/0 (socket says 172.21.15.87:56212) 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 -- 172.21.15.87:0/1341327332 learned_addr learned my addr 172.21.15.87:0/1341327332 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a090 0x5560dc234e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a9b0 0x5560dc225660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 -- 172.21.15.87:0/1341327332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a090 msgr2=0x5560dc234e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a090 0x5560dc234e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 -- 172.21.15.87:0/1341327332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a9b0 msgr2=0x5560dc225660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a9b0 0x5560dc225660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 -- 172.21.15.87:0/1341327332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dc243f30 con 0x5560dc237560 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc003e20 tx=0x7fc5cc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9f09b39df1b115 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.089+0000 7fc5b9ffb640 1 -- 172.21.15.87:0/1341327332 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc021070 con 0x5560dc237560 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5b9ffb640 1 -- 172.21.15.87:0/1341327332 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc011c40 con 0x5560dc237560 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5b9ffb640 1 -- 172.21.15.87:0/1341327332 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc01ce40 con 0x5560dc237560 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1341327332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 msgr2=0x5560dc24c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc003e20 tx=0x7fc5cc004410 comp rx=0 tx=0).stop 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1341327332 shutdown_connections 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc237560 0x5560dc24c890 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a9b0 0x5560dc225660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/1341327332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc22a090 0x5560dc234e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1341327332 >> 172.21.15.87:0/1341327332 conn(0x5560dc226680 msgr2=0x5560dc224f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1341327332 shutdown_connections 2024-06-06T08:04:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.090+0000 7fc5dd61c740 1 -- 172.21.15.87:0/1341327332 wait complete. 2024-06-06T08:04:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 -- start start 2024-06-06T08:04:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 Processor -- start 2024-06-06T08:04:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc24cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc24a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41ab440 con 0x5560dc22a9b0 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41ac8b0 con 0x5560dc22a090 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41abb10 con 0x5560dc237560 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5dd61c740 10 client.0 _ll_register_callbacks cb 0x7fc5c4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56226/0 (socket says 172.21.15.87:56226) 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3335483981 learned_addr learned my addr 172.21.15.87:0/3335483981 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc24cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.091+0000 7fc5d8dce640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc24a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3335483981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 msgr2=0x5560dc24a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc24a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 msgr2=0x5560dc24cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc24cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 -- 172.21.15.87:0/3335483981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dc24ac30 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5d37fe640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8098600 tx=0x7fc5c809bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9c10172b5c8392 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80ba040 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc5c8099f80 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc011c40 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc5c809ab40 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 10 client.17805 fetch_fsmap learned FSMap version 48 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5560dc2b1b60 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc011c40 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc5c80a11a0 con 0x5560dc22a9b0 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 10 client.17805 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 10 client.17805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 20 client.17805 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc2b2030 0x5560dc2b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 4 client.17805 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:04:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.092+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560dc19bc20 con 0x5560dc2b2030 2024-06-06T08:04:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.093+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc2b2030 0x5560dc2b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.093+0000 7fc5d3fff640 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc2b2030 0x5560dc2b4420 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a7d7bb9bed1bca server_cookie=82145f4406a5a6a1 in_seq=0 out_seq=0 2024-06-06T08:04:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:23.093+0000 7fc5b37fe640 10 client.17805 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:23.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 33196 16.04 MB/sec execute 25 sec latency 6467.680 ms 2024-06-06T08:04:24.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 33379 15.70 MB/sec execute 26 sec latency 7467.721 ms 2024-06-06T08:04:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:23 smithi195 ceph-mon[37040]: pgmap v520: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:23 smithi181 ceph-mon[36790]: pgmap v520: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:23 smithi087 ceph-mon[32276]: pgmap v520: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:25.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 33735 15.61 MB/sec execute 27 sec latency 8467.807 ms 2024-06-06T08:04:26.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 34182 15.87 MB/sec execute 28 sec latency 9467.847 ms 2024-06-06T08:04:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:25 smithi195 ceph-mon[37040]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:25 smithi181 ceph-mon[36790]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:25 smithi087 ceph-mon[32276]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-06-06T08:04:27.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 34648 16.20 MB/sec execute 29 sec latency 10467.898 ms 2024-06-06T08:04:28.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 35113 16.54 MB/sec execute 30 sec latency 11467.945 ms 2024-06-06T08:04:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:27 smithi195 ceph-mon[37040]: pgmap v522: 129 pgs: 129 active+clean; 377 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2024-06-06T08:04:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:27 smithi181 ceph-mon[36790]: pgmap v522: 129 pgs: 129 active+clean; 377 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2024-06-06T08:04:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:27 smithi087 ceph-mon[32276]: pgmap v522: 129 pgs: 129 active+clean; 377 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2024-06-06T08:04:29.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 35692 16.95 MB/sec execute 31 sec latency 12467.995 ms 2024-06-06T08:04:30.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 36152 17.55 MB/sec execute 32 sec latency 13468.030 ms 2024-06-06T08:04:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:29 smithi195 ceph-mon[37040]: pgmap v523: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.4 MiB/s wr, 663 op/s 2024-06-06T08:04:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:29 smithi181 ceph-mon[36790]: pgmap v523: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.4 MiB/s wr, 663 op/s 2024-06-06T08:04:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:29 smithi087 ceph-mon[32276]: pgmap v523: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.4 MiB/s wr, 663 op/s 2024-06-06T08:04:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5b37fe640 1 -- 172.21.15.87:0/3335483981 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560dc19bc20 con 0x5560dc2b2030 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5b37fe640 10 client.17805 handle_command_reply: tid=0 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 client.17805 shutdown 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 msgr2=0x5560dc24a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8098600 tx=0x7fc5c809bd40 comp rx=0 tx=0).stop 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 shutdown_connections 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5560dc2b2030 0x5560dc2b4420 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560dc237560 0x5560dc24a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5b37fe640 0 client.17805 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5560dc22a9b0 0x5560dc24a130 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 --2- 172.21.15.87:0/3335483981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5560dc22a090 0x5560dc24cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 >> 172.21.15.87:0/3335483981 conn(0x5560dc226680 msgr2=0x5560dc232a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 shutdown_connections 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.708+0000 7fc5dd61c740 1 -- 172.21.15.87:0/3335483981 wait complete. 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5dd61c740 20 client.17805 trim_cache size 0 max 16384 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:04:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 msgr2=0x7fc5bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 0x7fc5bc0a52e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c4000bc0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 msgr2=0x7fc5d41975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 shutdown_connections 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc5bc0a2e20 0x7fc5bc0a52e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4106890 0x7fc5d419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5d4101290 0x7fc5d41975f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 --2- 172.21.15.87:0/1007251708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d41008d0 0x7fc5d41970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 >> 172.21.15.87:0/1007251708 conn(0x7fc5d4077a20 msgr2=0x7fc5d4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 shutdown_connections 2024-06-06T08:04:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:30.709+0000 7fc5da831640 1 -- 172.21.15.87:0/1007251708 wait complete. 2024-06-06T08:04:31.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 36465 17.48 MB/sec execute 33 sec latency 13996.075 ms 2024-06-06T08:04:32.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 36761 17.45 MB/sec execute 34 sec latency 206.348 ms 2024-06-06T08:04:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:31 smithi195 ceph-mon[37040]: pgmap v524: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-06-06T08:04:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:31 smithi181 ceph-mon[36790]: pgmap v524: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-06-06T08:04:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:31 smithi087 ceph-mon[32276]: pgmap v524: 129 pgs: 129 active+clean; 388 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-06-06T08:04:33.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 37117 17.33 MB/sec execute 35 sec latency 242.103 ms 2024-06-06T08:04:34.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 37499 17.40 MB/sec execute 36 sec latency 59.695 ms 2024-06-06T08:04:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:33 smithi195 ceph-mon[37040]: pgmap v525: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-06-06T08:04:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:33 smithi181 ceph-mon[36790]: pgmap v525: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-06-06T08:04:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:33 smithi087 ceph-mon[32276]: pgmap v525: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-06-06T08:04:35.163 INFO:tasks.workunit.client.0.smithi087.stdout: 10 37876 17.53 MB/sec execute 37 sec latency 241.431 ms 2024-06-06T08:04:36.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 37913 17.08 MB/sec execute 38 sec latency 901.336 ms 2024-06-06T08:04:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:35 smithi195 ceph-mon[37040]: pgmap v526: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-06-06T08:04:36.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:35 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:04:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:35 smithi181 ceph-mon[36790]: pgmap v526: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-06-06T08:04:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:35 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:04:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:35 smithi087 ceph-mon[32276]: pgmap v526: 129 pgs: 129 active+clean; 378 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-06-06T08:04:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:35 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:04:37.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 38001 16.74 MB/sec execute 39 sec latency 1148.336 ms 2024-06-06T08:04:38.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 38256 16.52 MB/sec execute 40 sec latency 255.047 ms 2024-06-06T08:04:38.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:37 smithi195 ceph-mon[37040]: pgmap v527: 129 pgs: 129 active+clean; 408 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-06-06T08:04:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:37 smithi181 ceph-mon[36790]: pgmap v527: 129 pgs: 129 active+clean; 408 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-06-06T08:04:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:37 smithi087 ceph-mon[32276]: pgmap v527: 129 pgs: 129 active+clean; 408 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-06-06T08:04:39.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 38685 16.66 MB/sec execute 41 sec latency 270.638 ms 2024-06-06T08:04:39.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:38 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:39.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:38 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:04:39.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:38 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:04:39.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:38 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:04:39.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:38 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:39.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:38 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:04:39.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:38 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:04:39.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:38 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:04:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:38 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:38 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:04:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:38 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:04:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:38 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:04:40.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 39126 16.96 MB/sec execute 42 sec latency 205.589 ms 2024-06-06T08:04:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:39 smithi195 ceph-mon[37040]: pgmap v528: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-06T08:04:40.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:39 smithi181 ceph-mon[36790]: pgmap v528: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-06T08:04:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:39 smithi087 ceph-mon[32276]: pgmap v528: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-06T08:04:41.164 INFO:tasks.workunit.client.0.smithi087.stdout: 10 39618 17.28 MB/sec execute 43 sec latency 266.201 ms 2024-06-06T08:04:42.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 39973 17.34 MB/sec execute 44 sec latency 172.515 ms 2024-06-06T08:04:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:41 smithi195 ceph-mon[37040]: pgmap v529: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 332 op/s 2024-06-06T08:04:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:41 smithi181 ceph-mon[36790]: pgmap v529: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 332 op/s 2024-06-06T08:04:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:41 smithi087 ceph-mon[32276]: pgmap v529: 129 pgs: 129 active+clean; 450 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 332 op/s 2024-06-06T08:04:43.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 40408 17.53 MB/sec execute 45 sec latency 94.878 ms 2024-06-06T08:04:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 -- 172.21.15.87:0/4077161544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 msgr2=0x7f86c410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/4077161544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c410e410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009be0 tx=0x7f86bc02a7e0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 -- 172.21.15.87:0/4077161544 shutdown_connections 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/4077161544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c410e410 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/4077161544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4103550 0x7f86c410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/4077161544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4102c30 0x7f86c4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 -- 172.21.15.87:0/4077161544 >> 172.21.15.87:0/4077161544 conn(0x7f86c4077a20 msgr2=0x7f86c4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 -- 172.21.15.87:0/4077161544 shutdown_connections 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 -- 172.21.15.87:0/4077161544 wait complete. 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.804+0000 7f86cbd5a640 1 Processor -- start 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 -- start start 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 0x7f86c4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4111d20 con 0x7f86c4102c30 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4111ba0 con 0x7f86c4103550 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86cbd5a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4111ea0 con 0x7f86c410bf70 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56160/0 (socket says 172.21.15.87:56160) 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 -- 172.21.15.87:0/2207731898 learned_addr learned my addr 172.21.15.87:0/2207731898 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c92f5640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 0x7f86c4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 -- 172.21.15.87:0/2207731898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 msgr2=0x7f86c419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 -- 172.21.15.87:0/2207731898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 msgr2=0x7f86c4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 0x7f86c4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 -- 172.21.15.87:0/2207731898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c419d1d0 con 0x7f86c4102c30 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c92f5640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 0x7f86c4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e9f0 tx=0x7f86c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86baffd640 1 -- 172.21.15.87:0/2207731898 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c000cd80 con 0x7f86c4102c30 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.805+0000 7f86baffd640 1 -- 172.21.15.87:0/2207731898 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c0005500 con 0x7f86c4102c30 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.806+0000 7f86baffd640 1 -- 172.21.15.87:0/2207731898 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0010690 con 0x7f86c4102c30 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.806+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c419d4c0 con 0x7f86c4102c30 2024-06-06T08:04:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.806+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41a4d50 con 0x7f86c4102c30 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86baffd640 1 -- 172.21.15.87:0/2207731898 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f86c00046c0 con 0x7f86c4102c30 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 Processor -- start 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 -- start start 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 0x555f4638b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f4638db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555f462e04f0 con 0x555f4637c990 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f4627e870 con 0x555f4637c070 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86cea65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f462dc4b0 con 0x555f4638b760 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86ca2f7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 0x555f4638b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86ca2f7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41950/0 (socket says 172.21.15.87:41950) 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86ca2f7640 1 -- 172.21.15.87:0/4248830517 learned_addr learned my addr 172.21.15.87:0/4248830517 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.809+0000 7f86c92f5640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f4638db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86baffd640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 0x7f86b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 -- 172.21.15.87:0/4248830517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 msgr2=0x555f4637c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 -- 172.21.15.87:0/4248830517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 msgr2=0x555f4638b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 0x555f4638b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 -- 172.21.15.87:0/4248830517 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4629a9b0 con 0x555f4638b760 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c9af6640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 0x555f4638b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86baffd640 1 -- 172.21.15.87:0/2207731898 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f86c00cfc90 con 0x7f86c4102c30 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 0x7f86b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86c92f5640 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f4638db50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009530 tx=0x7f86b40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb275f55df58836 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86affff640 1 -- 172.21.15.87:0/4248830517 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4004020 con 0x555f4638b760 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86affff640 1 -- 172.21.15.87:0/4248830517 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b40041c0 con 0x555f4638b760 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 0x7f86b00a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009be0 tx=0x7f86bc009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.810+0000 7f86affff640 1 -- 172.21.15.87:0/4248830517 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b400f9a0 con 0x555f4638b760 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- 172.21.15.87:0/4248830517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 msgr2=0x555f4638db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f4638db50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009530 tx=0x7f86b40118d0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- 172.21.15.87:0/4248830517 shutdown_connections 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f4638db50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c990 0x555f4638b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 --2- 172.21.15.87:0/4248830517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4637c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- 172.21.15.87:0/4248830517 >> 172.21.15.87:0/4248830517 conn(0x555f46378680 msgr2=0x555f46378a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- 172.21.15.87:0/4248830517 shutdown_connections 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- 172.21.15.87:0/4248830517 wait complete. 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 -- start start 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.811+0000 7f86cea65740 1 Processor -- start 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f46388bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555f4638e390 con 0x555f4638b760 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f46269ae0 con 0x555f4637c990 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f4638e510 con 0x555f4637c070 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86cea65740 10 client.0 _ll_register_callbacks cb 0x555f462a29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86c92f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f46388bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56170/0 (socket says 172.21.15.87:56170) 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 -- 172.21.15.87:0/3184844178 learned_addr learned my addr 172.21.15.87:0/3184844178 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:35842/0 (socket says 172.21.15.87:35842) 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 -- 172.21.15.87:0/3184844178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 msgr2=0x555f46388690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 -- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 msgr2=0x555f46388bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f46388bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 -- 172.21.15.87:0/3184844178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f45ff4210 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86c92f5640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f46388bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86c9af6640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f86bc030650 tx=0x7f86bc02a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8495058e94f0ee1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.812+0000 7f86ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc04f070 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f86bc004240 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b40041c0 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f86bc036040 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 10 client.17817 fetch_fsmap learned FSMap version 49 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555f460c0620 con 0x555f4638b760 2024-06-06T08:04:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b40041c0 con 0x555f4638b760 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f86bc03c040 con 0x555f4638b760 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 10 client.17817 fetch_fsmap finished waiting for FSMap version 49 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 10 client.17817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 20 client.17817 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f4639b900 0x555f4639dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 4 client.17817 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f462f3f10 con 0x555f4639b900 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.813+0000 7f86c9af6640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f4639b900 0x555f4639dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.814+0000 7f86ad7fa640 10 client.17817 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.814+0000 7f86c9af6640 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f4639b900 0x555f4639dd50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf6332cb15b40c1 server_cookie=1af3dd9a94e70c0 in_seq=0 out_seq=0 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86ad7fa640 1 -- 172.21.15.87:0/3184844178 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f462f3f10 con 0x555f4639b900 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86ad7fa640 10 client.17817 handle_command_reply: tid=0 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 client.17817 shutdown 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 msgr2=0x555f46389110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f86bc030650 tx=0x7f86bc02a7e0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 shutdown_connections 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f4639b900 0x555f4639dd50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f46389110 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f46388bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86ad7fa640 0 client.17817 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 --2- 172.21.15.87:0/3184844178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c070 0x555f46388690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 >> 172.21.15.87:0/3184844178 conn(0x555f46378680 msgr2=0x555f4637e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 shutdown_connections 2024-06-06T08:04:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.816+0000 7f86cea65740 1 -- 172.21.15.87:0/3184844178 wait complete. 2024-06-06T08:04:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.817+0000 7f86cea65740 20 client.17817 trim_cache size 0 max 16384 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.832+0000 7f86cea65740 1 Processor -- start 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.832+0000 7f86cea65740 1 -- start start 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f463874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f463a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41af640 con 0x555f4637c070 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41aceb0 con 0x555f4637c990 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86cea65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41aad20 con 0x555f4638b760 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56180/0 (socket says 172.21.15.87:56180) 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 -- 172.21.15.87:0/2558516081 learned_addr learned my addr 172.21.15.87:0/2558516081 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f463a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c92f5640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f463874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 -- 172.21.15.87:0/2558516081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 msgr2=0x555f463a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f463a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 -- 172.21.15.87:0/2558516081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 msgr2=0x555f463874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f463874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 -- 172.21.15.87:0/2558516081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4637c7c0 con 0x555f4637c070 2024-06-06T08:04:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c92f5640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f463874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f86c00cd020 tx=0x7f86c00d37e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369df06b1baefb40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86af7fe640 1 -- 172.21.15.87:0/2558516081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c009bb60 con 0x555f4637c070 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86af7fe640 1 -- 172.21.15.87:0/2558516081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c009c490 con 0x555f4637c070 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.833+0000 7f86af7fe640 1 -- 172.21.15.87:0/2558516081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c009b6f0 con 0x555f4637c070 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- 172.21.15.87:0/2558516081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 msgr2=0x555f46386f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f86c00cd020 tx=0x7f86c00d37e0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- 172.21.15.87:0/2558516081 shutdown_connections 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4638b760 0x555f463a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c990 0x555f463874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 --2- 172.21.15.87:0/2558516081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4637c070 0x555f46386f60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- 172.21.15.87:0/2558516081 >> 172.21.15.87:0/2558516081 conn(0x555f46378680 msgr2=0x555f4637d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- 172.21.15.87:0/2558516081 shutdown_connections 2024-06-06T08:04:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- 172.21.15.87:0/2558516081 wait complete. 2024-06-06T08:04:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.834+0000 7f86cea65740 1 -- start start 2024-06-06T08:04:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 Processor -- start 2024-06-06T08:04:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4638e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f4638f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41ab370 con 0x555f4638b760 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41ac7e0 con 0x555f4637c070 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41aba40 con 0x555f4637c990 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86cea65740 10 client.0 _ll_register_callbacks cb 0x7f86c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c92f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4638e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f4638f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c92f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:35866/0 (socket says 172.21.15.87:35866) 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c9af6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4638e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:41974/0 (socket says 172.21.15.87:41974) 2024-06-06T08:04:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c92f5640 1 -- 172.21.15.87:0/2510677912 learned_addr learned my addr 172.21.15.87:0/2510677912 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 -- 172.21.15.87:0/2510677912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 msgr2=0x555f4638ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 -- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 msgr2=0x555f4638e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4638e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86ca2f7640 1 -- 172.21.15.87:0/2510677912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f46404870 con 0x555f4638b760 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.835+0000 7f86c92f5640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ca2f7640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f4638f1a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f86bc031c00 tx=0x7f86bc02af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c85f6250baed713 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc04f040 con 0x555f4638b760 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f86bc031930 con 0x555f4638b760 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c009c490 con 0x555f4638b760 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f86bc003d40 con 0x555f4638b760 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 10 client.17823 fetch_fsmap learned FSMap version 49 2024-06-06T08:04:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555f46405b80 con 0x555f4638b760 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c009c490 con 0x555f4638b760 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 49) ==== 8354+0+0 (secure 0 0 0) 0x7f86bc0545e0 con 0x555f4638b760 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 10 client.17823 fetch_fsmap finished waiting for FSMap version 49 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 20 client.17823 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f464062d0 0x555f464086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 4 client.17823 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f462edc20 con 0x555f464062d0 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.836+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f464062d0 0x555f464086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.837+0000 7f86c9af6640 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f464062d0 0x555f464086f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd82f7cecce8599 server_cookie=a335bef92e455394 in_seq=0 out_seq=0 2024-06-06T08:04:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:43.837+0000 7f86ad7fa640 10 client.17823 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:44.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 40634 17.38 MB/sec execute 46 sec latency 219.723 ms 2024-06-06T08:04:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:43 smithi195 ceph-mon[37040]: pgmap v530: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-06-06T08:04:44.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:43 smithi181 ceph-mon[36790]: pgmap v530: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-06-06T08:04:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.334+0000 7f86ad7fa640 1 -- 172.21.15.87:0/2510677912 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555f462edc20 con 0x555f464062d0 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.334+0000 7f86ad7fa640 10 client.17823 handle_command_reply: tid=0 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 client.17823 shutdown 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 msgr2=0x555f4638f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f4638f1a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f86bc031c00 tx=0x7f86bc02af60 comp rx=0 tx=0).stop 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 shutdown_connections 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f464062d0 0x555f464086f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f4638b760 0x555f4638f1a0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f4637c990 0x555f4638ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 --2- 172.21.15.87:0/2510677912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f4637c070 0x555f4638e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86ad7fa640 0 client.17823 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 >> 172.21.15.87:0/2510677912 conn(0x555f46378680 msgr2=0x555f46386ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 shutdown_connections 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 1 -- 172.21.15.87:0/2510677912 wait complete. 2024-06-06T08:04:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.335+0000 7f86cea65740 20 client.17823 trim_cache size 0 max 16384 2024-06-06T08:04:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:04:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:04:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 msgr2=0x7f86b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 0x7f86b00a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009be0 tx=0x7f86bc009500 comp rx=0 tx=0).stop 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 msgr2=0x7f86c41981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e9f0 tx=0x7f86c000edf0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 shutdown_connections 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f86b00a2da0 0x7f86b00a5260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86c410bf70 0x7f86c419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4103550 0x7f86c4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 --2- 172.21.15.87:0/2207731898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86c4102c30 0x7f86c41981e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 >> 172.21.15.87:0/2207731898 conn(0x7f86c4077a20 msgr2=0x7f86c40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 shutdown_connections 2024-06-06T08:04:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:44.336+0000 7f86cbd5a640 1 -- 172.21.15.87:0/2207731898 wait complete. 2024-06-06T08:04:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:43 smithi087 ceph-mon[32276]: pgmap v530: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-06-06T08:04:45.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 40974 17.47 MB/sec execute 47 sec latency 295.629 ms 2024-06-06T08:04:46.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 41277 17.34 MB/sec execute 48 sec latency 171.854 ms 2024-06-06T08:04:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:45 smithi195 ceph-mon[37040]: pgmap v531: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.9 MiB/s wr, 450 op/s 2024-06-06T08:04:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:45 smithi181 ceph-mon[36790]: pgmap v531: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.9 MiB/s wr, 450 op/s 2024-06-06T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:45 smithi087 ceph-mon[32276]: pgmap v531: 129 pgs: 129 active+clean; 393 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.9 MiB/s wr, 450 op/s 2024-06-06T08:04:47.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 41615 17.36 MB/sec execute 49 sec latency 71.441 ms 2024-06-06T08:04:48.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 42127 17.41 MB/sec execute 50 sec latency 76.043 ms 2024-06-06T08:04:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:47 smithi195 ceph-mon[37040]: pgmap v532: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 564 op/s 2024-06-06T08:04:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:47 smithi181 ceph-mon[36790]: pgmap v532: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 564 op/s 2024-06-06T08:04:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:47 smithi087 ceph-mon[32276]: pgmap v532: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 564 op/s 2024-06-06T08:04:49.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 42602 17.63 MB/sec execute 51 sec latency 185.436 ms 2024-06-06T08:04:49.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:49 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:49.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:49 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:49.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:49 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:04:50.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 43115 17.87 MB/sec execute 52 sec latency 167.871 ms 2024-06-06T08:04:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:50 smithi195 ceph-mon[37040]: pgmap v533: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-06-06T08:04:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:50 smithi181 ceph-mon[36790]: pgmap v533: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-06-06T08:04:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:50 smithi087 ceph-mon[32276]: pgmap v533: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-06-06T08:04:51.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 43490 17.97 MB/sec execute 53 sec latency 143.531 ms 2024-06-06T08:04:52.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 43930 18.09 MB/sec execute 54 sec latency 190.709 ms 2024-06-06T08:04:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:52 smithi195 ceph-mon[37040]: pgmap v534: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 494 op/s 2024-06-06T08:04:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:52 smithi181 ceph-mon[36790]: pgmap v534: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 494 op/s 2024-06-06T08:04:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:52 smithi087 ceph-mon[32276]: pgmap v534: 129 pgs: 129 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 494 op/s 2024-06-06T08:04:53.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 44279 18.13 MB/sec execute 55 sec latency 153.624 ms 2024-06-06T08:04:54.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 44541 18.12 MB/sec execute 56 sec latency 247.363 ms 2024-06-06T08:04:54.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:54 smithi195 ceph-mon[37040]: pgmap v535: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2024-06-06T08:04:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:54 smithi181 ceph-mon[36790]: pgmap v535: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2024-06-06T08:04:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:54 smithi087 ceph-mon[32276]: pgmap v535: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2024-06-06T08:04:55.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 44848 18.01 MB/sec execute 57 sec latency 215.663 ms 2024-06-06T08:04:56.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 45279 17.98 MB/sec execute 58 sec latency 214.547 ms 2024-06-06T08:04:56.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:56 smithi195 ceph-mon[37040]: pgmap v536: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2024-06-06T08:04:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:56 smithi181 ceph-mon[36790]: pgmap v536: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2024-06-06T08:04:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:56 smithi087 ceph-mon[32276]: pgmap v536: 129 pgs: 129 active+clean; 390 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2024-06-06T08:04:57.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 45748 18.10 MB/sec execute 59 sec latency 211.823 ms 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 -- 172.21.15.87:0/3209182578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72441008d0 msgr2=0x7f7244100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 --2- 172.21.15.87:0/3209182578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72441008d0 0x7f7244100d50 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7234009890 tx=0x7f723402a380 comp rx=0 tx=0).stop 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 -- 172.21.15.87:0/3209182578 shutdown_connections 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 --2- 172.21.15.87:0/3209182578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f724410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 --2- 172.21.15.87:0/3209182578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72441008d0 0x7f7244100d50 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 --2- 172.21.15.87:0/3209182578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7244106890 0x7f7244106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 -- 172.21.15.87:0/3209182578 >> 172.21.15.87:0/3209182578 conn(0x7f7244077a20 msgr2=0x7f7244077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 -- 172.21.15.87:0/3209182578 shutdown_connections 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.434+0000 7f724c114640 1 -- 172.21.15.87:0/3209182578 wait complete. 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 Processor -- start 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 -- start start 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72441008d0 0x7f7244199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f7244199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110ca0 con 0x7f7244106890 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110b20 con 0x7f72441008d0 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724c114640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110e20 con 0x7f7244101290 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f7244199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46690/0 (socket says 172.21.15.87:46690) 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f724a6b1640 1 -- 172.21.15.87:0/3265718772 learned_addr learned my addr 172.21.15.87:0/3265718772 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f7249eb0640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72441008d0 0x7f7244199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 -- 172.21.15.87:0/3265718772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72441008d0 msgr2=0x7f7244199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72441008d0 0x7f7244199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 -- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 msgr2=0x7f72441a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.435+0000 7f72496af640 1 -- 172.21.15.87:0/3265718772 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72441a45e0 con 0x7f7244101290 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f72496af640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f7244199560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f723402a8a0 tx=0x7f7234033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f723affd640 1 -- 172.21.15.87:0/3265718772 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723403f020 con 0x7f7244101290 2024-06-06T08:04:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72441a47b0 con 0x7f7244101290 2024-06-06T08:04:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f723affd640 1 -- 172.21.15.87:0/3265718772 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7234030550 con 0x7f7244101290 2024-06-06T08:04:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f723affd640 1 -- 172.21.15.87:0/3265718772 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234037040 con 0x7f7244101290 2024-06-06T08:04:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.436+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72441a4c70 con 0x7f7244101290 2024-06-06T08:04:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f723affd640 1 -- 172.21.15.87:0/3265718772 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7234042020 con 0x7f7244101290 2024-06-06T08:04:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 Processor -- start 2024-06-06T08:04:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 -- start start 2024-06-06T08:04:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 0x556f3f3df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f3f3434f0 con 0x556f3f3df050 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f3f2e1870 con 0x556f3f3ee760 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.439+0000 7f724ee23740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f3f33f4b0 con 0x556f3f3df970 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 0x556f3f3df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f7249eb0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 0x556f3f3df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46702/0 (socket says 172.21.15.87:46702) 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f724a6b1640 1 -- 172.21.15.87:0/1394077096 learned_addr learned my addr 172.21.15.87:0/1394077096 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f723affd640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 0x7f72300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 -- 172.21.15.87:0/1394077096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 msgr2=0x556f3f3ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 -- 172.21.15.87:0/1394077096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 msgr2=0x556f3f3df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 0x556f3f3df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 -- 172.21.15.87:0/1394077096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3f2fd9b0 con 0x556f3f3ee760 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 0x7f72300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f723affd640 1 -- 172.21.15.87:0/3265718772 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f72340ef6a0 con 0x7f7244101290 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f72496af640 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3f0b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f72340ecae0 tx=0x7f72340b7c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ea729c786e9344 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 0x7f72300a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f723c011ad0 tx=0x7f723c004a40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f722ffff640 1 -- 172.21.15.87:0/1394077096 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340bb680 con 0x556f3f3ee760 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f722ffff640 1 -- 172.21.15.87:0/1394077096 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340bbfb0 con 0x556f3f3ee760 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.440+0000 7f722ffff640 1 -- 172.21.15.87:0/1394077096 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340c3e80 con 0x556f3f3ee760 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 -- 172.21.15.87:0/1394077096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 msgr2=0x556f3f3f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3f0b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f72340ecae0 tx=0x7f72340b7c80 comp rx=0 tx=0).stop 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 -- 172.21.15.87:0/1394077096 shutdown_connections 2024-06-06T08:04:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3f0b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 --2- 172.21.15.87:0/1394077096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df050 0x556f3f3df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 -- 172.21.15.87:0/1394077096 >> 172.21.15.87:0/1394077096 conn(0x556f3f3db680 msgr2=0x556f3f3dba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 -- 172.21.15.87:0/1394077096 shutdown_connections 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.441+0000 7f724ee23740 1 -- 172.21.15.87:0/1394077096 wait complete. 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 -- start start 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 Processor -- start 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df050 0x556f3f3eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f3f3f1150 con 0x556f3f3ee760 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f3f2ccae0 con 0x556f3f3df050 2024-06-06T08:04:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f3f3f12d0 con 0x556f3f3df970 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724ee23740 10 client.0 _ll_register_callbacks cb 0x556f3f3059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46708/0 (socket says 172.21.15.87:46708) 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724a6b1640 1 -- 172.21.15.87:0/3670740521 learned_addr learned my addr 172.21.15.87:0/3670740521 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f72496af640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f7249eb0640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df050 0x556f3f3eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724a6b1640 1 -- 172.21.15.87:0/3670740521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 msgr2=0x556f3f3ebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.442+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ebc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724a6b1640 1 -- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df050 msgr2=0x556f3f3eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df050 0x556f3f3eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724a6b1640 1 -- 172.21.15.87:0/3670740521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3f057210 con 0x556f3f3ee760 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f723c01c6d0 tx=0x7f723c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e67031c605fd2c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c02c060 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f723c00e880 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72340bbfb0 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f723c033050 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724ee23740 10 client.17835 fetch_fsmap learned FSMap version 50 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556f3f123620 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340bbfb0 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.443+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f723c027cb0 con 0x556f3f3ee760 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 10 client.17835 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 10 client.17835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 20 client.17835 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f3fe630 0x556f3f400a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 4 client.17835 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:04:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3f356f10 con 0x556f3f3fe630 2024-06-06T08:04:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f7249eb0640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f3fe630 0x556f3f400a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f722d7fa640 10 client.17835 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.444+0000 7f7249eb0640 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f3fe630 0x556f3f400a40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40b9b909bf942da server_cookie=970e62eb06c93f6b in_seq=0 out_seq=0 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f722d7fa640 1 -- 172.21.15.87:0/3670740521 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f3f356f10 con 0x556f3f3fe630 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f722d7fa640 10 client.17835 handle_command_reply: tid=0 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 client.17835 shutdown 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 msgr2=0x556f3f3ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f723c01c6d0 tx=0x7f723c004ab0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 shutdown_connections 2024-06-06T08:04:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f3fe630 0x556f3f400a40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3ec140 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df970 0x556f3f3ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 --2- 172.21.15.87:0/3670740521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df050 0x556f3f3eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f722d7fa640 0 client.17835 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 >> 172.21.15.87:0/3670740521 conn(0x556f3f3db680 msgr2=0x556f3f3e14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 shutdown_connections 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 1 -- 172.21.15.87:0/3670740521 wait complete. 2024-06-06T08:04:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.448+0000 7f724ee23740 20 client.17835 trim_cache size 0 max 16384 2024-06-06T08:04:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 Processor -- start 2024-06-06T08:04:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 -- start start 2024-06-06T08:04:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f406350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f72441af6d0 con 0x556f3f3df970 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72441acf40 con 0x556f3f3ee760 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.464+0000 7f724ee23740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72441aadb0 con 0x556f3f3df050 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46714/0 (socket says 172.21.15.87:46714) 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 -- 172.21.15.87:0/3335811962 learned_addr learned my addr 172.21.15.87:0/3335811962 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f7249eb0640 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f406350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f724a6b1640 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 -- 172.21.15.87:0/3335811962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 msgr2=0x556f3f406350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f406350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 -- 172.21.15.87:0/3335811962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 msgr2=0x556f3f3ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 -- 172.21.15.87:0/3335811962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3f4010c0 con 0x556f3f3df970 2024-06-06T08:04:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f72496af640 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f72340b8280 tx=0x7f7234033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d3964aad155b39 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f722f7fe640 1 -- 172.21.15.87:0/3335811962 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340b9e70 con 0x556f3f3df970 2024-06-06T08:04:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f722f7fe640 1 -- 172.21.15.87:0/3335811962 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340f5320 con 0x556f3f3df970 2024-06-06T08:04:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.465+0000 7f722f7fe640 1 -- 172.21.15.87:0/3335811962 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340ba060 con 0x556f3f3df970 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 -- 172.21.15.87:0/3335811962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 msgr2=0x556f3f406890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f72340b8280 tx=0x7f7234033f90 comp rx=0 tx=0).stop 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 -- 172.21.15.87:0/3335811962 shutdown_connections 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3ee760 0x556f3f3ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3df970 0x556f3f406890 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 --2- 172.21.15.87:0/3335811962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f406350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 -- 172.21.15.87:0/3335811962 >> 172.21.15.87:0/3335811962 conn(0x556f3f3db680 msgr2=0x556f3f3e9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 -- 172.21.15.87:0/3335811962 shutdown_connections 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.466+0000 7f724ee23740 1 -- 172.21.15.87:0/3335811962 wait complete. 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 -- start start 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 Processor -- start 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f3f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df970 0x556f3f3f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f72441ab400 con 0x556f3f3ee760 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72441ac870 con 0x556f3f3df970 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72441abad0 con 0x556f3f3df050 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724ee23740 10 client.0 _ll_register_callbacks cb 0x7f723c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724a6b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46722/0 (socket says 172.21.15.87:46722) 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724a6b1640 1 -- 172.21.15.87:0/562194199 learned_addr learned my addr 172.21.15.87:0/562194199 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f7249eb0640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f3f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f72496af640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df970 0x556f3f3f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724a6b1640 1 -- 172.21.15.87:0/562194199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 msgr2=0x556f3f3f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.467+0000 7f724a6b1640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f3f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724a6b1640 1 -- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df970 msgr2=0x556f3f3f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724a6b1640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df970 0x556f3f3f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724a6b1640 1 -- 172.21.15.87:0/562194199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3f3f22a0 con 0x556f3f3ee760 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724a6b1640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f723c01c6a0 tx=0x7f723c00e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0343c72aa0bd91f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c03c060 con 0x556f3f3ee760 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f723c026660 con 0x556f3f3ee760 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72340f5320 con 0x556f3f3ee760 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f723c043050 con 0x556f3f3ee760 2024-06-06T08:04:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 10 client.17841 fetch_fsmap learned FSMap version 50 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556f3f468e30 con 0x556f3f3ee760 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340f5320 con 0x556f3f3ee760 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f723c027790 con 0x556f3f3ee760 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 10 client.17841 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 10 client.17841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 20 client.17841 populate_metadata read hostname 'smithi087' 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.468+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f469580 0x556f3f46b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.469+0000 7f724ee23740 4 client.17841 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.469+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f3f350c20 con 0x556f3f469580 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.469+0000 7f7249eb0640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f469580 0x556f3f46b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.469+0000 7f722d7fa640 10 client.17841 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:57.469+0000 7f7249eb0640 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f469580 0x556f3f46b9a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daeed9318248dd5 server_cookie=904b26e6aaa391ae in_seq=0 out_seq=0 2024-06-06T08:04:58.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 45961 17.96 MB/sec execute 60 sec latency 569.791 ms 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.218+0000 7f722d7fa640 1 -- 172.21.15.87:0/562194199 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f3f350c20 con 0x556f3f469580 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.218+0000 7f722d7fa640 10 client.17841 handle_command_reply: tid=0 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.218+0000 7f724ee23740 1 client.17841 shutdown 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 msgr2=0x556f3f3f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f723c01c6a0 tx=0x7f723c00e930 comp rx=0 tx=0).stop 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 shutdown_connections 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f3f469580 0x556f3f46b9a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f722d7fa640 0 client.17841 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f3f3ee760 0x556f3f3f1a70 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f3f3df970 0x556f3f3f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 --2- 172.21.15.87:0/562194199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f3f3df050 0x556f3f3f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 >> 172.21.15.87:0/562194199 conn(0x556f3f3db680 msgr2=0x556f3f3da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 shutdown_connections 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 1 -- 172.21.15.87:0/562194199 wait complete. 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724ee23740 20 client.17841 trim_cache size 0 max 16384 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:04:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:04:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:04:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 msgr2=0x7f72300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 0x7f72300a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f723c011ad0 tx=0x7f723c004a40 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 msgr2=0x7f7244199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.219+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f7244199560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f723402a8a0 tx=0x7f7234033ae0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 shutdown_connections 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f72300a2da0 0x7f72300a5260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7244106890 0x7f72441a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7244101290 0x7f7244199560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 --2- 172.21.15.87:0/3265718772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72441008d0 0x7f7244199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 >> 172.21.15.87:0/3265718772 conn(0x7f7244077a20 msgr2=0x7f7244103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 shutdown_connections 2024-06-06T08:04:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:04:58.220+0000 7f724c114640 1 -- 172.21.15.87:0/3265718772 wait complete. 2024-06-06T08:04:58.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:58 smithi181 ceph-mon[36790]: pgmap v537: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-06T08:04:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:58 smithi087 ceph-mon[32276]: pgmap v537: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-06T08:04:58.412 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:58 smithi195 ceph-mon[37040]: pgmap v537: 129 pgs: 129 active+clean; 392 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 393 op/s 2024-06-06T08:04:58.759 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:04:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T08:04:58.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:04:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T08:04:58.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:04:58 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/3955395734 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c723a3a40 con 0x559c72376c00 2024-06-06T08:04:58.760 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:04:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T08:04:59.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 46381 18.02 MB/sec execute 61 sec latency 570.192 ms 2024-06-06T08:04:59.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:04:59 smithi181 ceph-mon[36790]: daemon mds.c restarted 2024-06-06T08:04:59.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:04:59 smithi087 ceph-mon[32276]: daemon mds.c restarted 2024-06-06T08:04:59.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:04:59 smithi195 ceph-mon[37040]: daemon mds.c restarted 2024-06-06T08:05:00.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 46903 18.23 MB/sec execute 62 sec latency 179.829 ms 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: pgmap v538: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/4097924488,v1:172.21.15.195:6837/4097924488] up:boot 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:05:00.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:00 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: pgmap v538: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/4097924488,v1:172.21.15.195:6837/4097924488] up:boot 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:05:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:00 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:05:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: pgmap v538: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-06-06T08:05:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T08:05:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/4097924488,v1:172.21.15.195:6837/4097924488] up:boot 2024-06-06T08:05:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:05:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:05:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:00 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:05:01.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 47259 18.26 MB/sec execute 63 sec latency 412.501 ms 2024-06-06T08:05:02.165 INFO:tasks.workunit.client.0.smithi087.stdout: 10 47272 17.99 MB/sec execute 64 sec latency 1272.379 ms 2024-06-06T08:05:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:02 smithi181 ceph-mon[36790]: pgmap v540: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-06-06T08:05:02.312 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:02 smithi195 ceph-mon[37040]: pgmap v540: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-06-06T08:05:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:02 smithi087 ceph-mon[32276]: pgmap v540: 129 pgs: 129 active+clean; 404 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-06-06T08:05:02.753 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.7 2024-06-06T08:05:02.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:05:03.155 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:03.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 47345 17.76 MB/sec execute 65 sec latency 1338.702 ms 2024-06-06T08:05:04.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 47619 17.89 MB/sec execute 66 sec latency 80.370 ms 2024-06-06T08:05:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:04 smithi181 ceph-mon[36790]: pgmap v541: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:04 smithi087 ceph-mon[32276]: pgmap v541: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:04 smithi195 ceph-mon[37040]: pgmap v541: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:05.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 -- 172.21.15.87:0/665960303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818106b90 msgr2=0x7fa818106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 --2- 172.21.15.87:0/665960303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818106b90 0x7fa818106f70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa814009990 tx=0x7fa814009ca0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 -- 172.21.15.87:0/665960303 shutdown_connections 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 --2- 172.21.15.87:0/665960303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818101510 0x7fa81810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 --2- 172.21.15.87:0/665960303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818106b90 0x7fa818106f70 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 --2- 172.21.15.87:0/665960303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818100b50 0x7fa818100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.094+0000 7fa8201d0700 1 -- 172.21.15.87:0/665960303 >> 172.21.15.87:0/665960303 conn(0x7fa818077a00 msgr2=0x7fa8180fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 -- 172.21.15.87:0/665960303 shutdown_connections 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 -- 172.21.15.87:0/665960303 wait complete. 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 Processor -- start 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 -- start start 2024-06-06T08:05:05.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.095+0000 7fa8201d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818106b90 0x7fa8181968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa8201d0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181105c0 con 0x7fa818100b50 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa8201d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa818110440 con 0x7fa818106b90 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa8201d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa818110740 con 0x7fa818101510 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81df6c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81df6c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46742/0 (socket says 172.21.15.87:46742) 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81df6c700 1 -- 172.21.15.87:0/1996101230 learned_addr learned my addr 172.21.15.87:0/1996101230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:34996/0 (socket says 172.21.15.87:34996) 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81e76d700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818106b90 0x7fa8181968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 -- 172.21.15.87:0/1996101230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818106b90 msgr2=0x7fa8181968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:05.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818106b90 0x7fa8181968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 -- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 msgr2=0x7fa81819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 -- 172.21.15.87:0/1996101230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa818197020 con 0x7fa818101510 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81df6c700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa81d76b700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa80800ea90 tx=0x7fa80800eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.096+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800ccd0 con 0x7fa818101510 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.097+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa818197310 con 0x7fa818101510 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.097+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa808004500 con 0x7fa818101510 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.097+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8181a57e0 con 0x7fa818101510 2024-06-06T08:05:05.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.097+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808005260 con 0x7fa818101510 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.098+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa818109090 con 0x7fa818101510 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.099+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa808005520 con 0x7fa818101510 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.099+0000 7fa80effd700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 0x7fa8040a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.100+0000 7fa81df6c700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 0x7fa8040a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.100+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fa8080cf570 con 0x7fa818101510 2024-06-06T08:05:05.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.100+0000 7fa81df6c700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 0x7fa8040a52d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa814009750 tx=0x7fa814005e80 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:05.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.105+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa80809be50 con 0x7fa818101510 2024-06-06T08:05:05.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 47968 17.93 MB/sec execute 67 sec latency 103.721 ms 2024-06-06T08:05:05.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.392+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa818197df0 con 0x7fa818101510 2024-06-06T08:05:05.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.393+0000 7fa80effd700 1 -- 172.21.15.87:0/1996101230 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11881 (secure 0 0 0) 0x7fa80809b750 con 0x7fa818101510 2024-06-06T08:05:05.393 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:05:05.393 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:04:59.022587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:05:05.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 msgr2=0x7fa8040a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 0x7fa8040a52d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa814009750 tx=0x7fa814005e80 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 msgr2=0x7fa81819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa80800ea90 tx=0x7fa80800eda0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 shutdown_connections 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa818106b90 0x7fa8181968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa8040a2e10 0x7fa8040a52d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa818100b50 0x7fa81819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 --2- 172.21.15.87:0/1996101230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa818101510 0x7fa81819cc40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.395+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 >> 172.21.15.87:0/1996101230 conn(0x7fa818077a00 msgr2=0x7fa818103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.396+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 shutdown_connections 2024-06-06T08:05:05.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:05.396+0000 7fa8201d0700 1 -- 172.21.15.87:0/1996101230 wait complete. 2024-06-06T08:05:05.397 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 52 2024-06-06T08:05:06.068 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 135.7 --await-for 45.0' 2024-06-06T08:05:06.068 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 135.7 --await-for 45.0 2024-06-06T08:05:06.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48233 17.82 MB/sec execute 68 sec latency 766.239 ms 2024-06-06T08:05:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:06 smithi181 ceph-mon[36790]: pgmap v542: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:06.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:06 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1996101230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:06.346 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:06 smithi087 ceph-mon[32276]: pgmap v542: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:06 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1996101230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:06 smithi195 ceph-mon[37040]: pgmap v542: 129 pgs: 129 active+clean; 448 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.8 MiB/s wr, 625 op/s 2024-06-06T08:05:06.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:06 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1996101230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:07.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48557 17.80 MB/sec execute 69 sec latency 1766.282 ms 2024-06-06T08:05:08.090 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:08 smithi087 ceph-mon[32276]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-06-06T08:05:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 -- 172.21.15.87:0/1878663326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 msgr2=0x7f732810e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 --2- 172.21.15.87:0/1878663326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f732810e290 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f732000b0d0 tx=0x7f732000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 -- 172.21.15.87:0/1878663326 shutdown_connections 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 --2- 172.21.15.87:0/1878663326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73281015f0 0x7f73281019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 --2- 172.21.15.87:0/1878663326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f732810e290 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 --2- 172.21.15.87:0/1878663326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7328101fa0 0x7f7328105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 -- 172.21.15.87:0/1878663326 >> 172.21.15.87:0/1878663326 conn(0x7f7328078330 msgr2=0x7f7328078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 -- 172.21.15.87:0/1878663326 shutdown_connections 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.116+0000 7f732d625700 1 -- 172.21.15.87:0/1878663326 wait complete. 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.117+0000 7f732d625700 1 Processor -- start 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.117+0000 7f732d625700 1 -- start start 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.117+0000 7f732d625700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 0x7f732819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f732d625700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7328101fa0 0x7f732819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f732d625700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f7328198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f732d625700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73281116d0 con 0x7f73281066c0 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f732d625700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7328111550 con 0x7f7328101fa0 2024-06-06T08:05:08.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f732d625700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7328111850 con 0x7f73281015f0 2024-06-06T08:05:08.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f7326ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 0x7f732819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f7328198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f7326ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 0x7f732819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58654/0 (socket says 172.21.15.87:58654) 2024-06-06T08:05:08.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f7326ffd700 1 -- 172.21.15.87:0/439355429 learned_addr learned my addr 172.21.15.87:0/439355429 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:08.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73267fc700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7328101fa0 0x7f732819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 -- 172.21.15.87:0/439355429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 msgr2=0x7f732819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 0x7f732819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 -- 172.21.15.87:0/439355429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7328101fa0 msgr2=0x7f732819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7328101fa0 0x7f732819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 -- 172.21.15.87:0/439355429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73281990c0 con 0x7f73281066c0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f73277fe700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f7328198960 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f732000b770 tx=0x7f7320007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.118+0000 7f730ffff700 1 -- 172.21.15.87:0/439355429 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320021020 con 0x7f73281066c0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.119+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7328199350 con 0x7f73281066c0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.119+0000 7f730ffff700 1 -- 172.21.15.87:0/439355429 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7320004c50 con 0x7f73281066c0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.119+0000 7f730ffff700 1 -- 172.21.15.87:0/439355429 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200052b0 con 0x7f73281066c0 2024-06-06T08:05:08.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.119+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7328102fb0 con 0x7f73281066c0 2024-06-06T08:05:08.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.121+0000 7f730ffff700 1 -- 172.21.15.87:0/439355429 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7320024020 con 0x7f73281066c0 2024-06-06T08:05:08.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.122+0000 7f730ffff700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 0x7f73100a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.122+0000 7f730ffff700 1 -- 172.21.15.87:0/439355429 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f732000f020 con 0x7f73281066c0 2024-06-06T08:05:08.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.123+0000 7f7326ffd700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 0x7f73100a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.123+0000 7f7326ffd700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 0x7f73100a52f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7328106280 tx=0x7f7318009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.123+0000 7f7337ea8b80 1 Processor -- start 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 -- start start 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f729d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f2f71ee9a0 con 0x55f2f72a35f0 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2f71968b0 con 0x55f2f72a2cd0 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2f71ea960 con 0x55f2f729d460 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56676/0 (socket says 172.21.15.87:56676) 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 -- 172.21.15.87:0/2590932493 learned_addr learned my addr 172.21.15.87:0/2590932493 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f7326ffd700 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73277fe700 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f729d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 -- 172.21.15.87:0/2590932493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 msgr2=0x55f2f729d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f729d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 -- 172.21.15.87:0/2590932493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 msgr2=0x55f2f72a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.124+0000 7f73267fc700 1 -- 172.21.15.87:0/2590932493 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f718d0e0 con 0x55f2f72a35f0 2024-06-06T08:05:08.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.125+0000 7f73267fc700 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f731c009c90 tx=0x7f731c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a96468415885844 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.125+0000 7f730d7fa700 1 -- 172.21.15.87:0/2590932493 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c011d20 con 0x55f2f72a35f0 2024-06-06T08:05:08.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.125+0000 7f730d7fa700 1 -- 172.21.15.87:0/2590932493 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c004530 con 0x55f2f72a35f0 2024-06-06T08:05:08.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.125+0000 7f730d7fa700 1 -- 172.21.15.87:0/2590932493 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c019640 con 0x55f2f72a35f0 2024-06-06T08:05:08.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- 172.21.15.87:0/2590932493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 msgr2=0x55f2f729cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f731c009c90 tx=0x7f731c011440 comp rx=0 tx=0).stop 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- 172.21.15.87:0/2590932493 shutdown_connections 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f729d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/2590932493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f729cf20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- 172.21.15.87:0/2590932493 >> 172.21.15.87:0/2590932493 conn(0x55f2f71c7e50 msgr2=0x55f2f72981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- 172.21.15.87:0/2590932493 shutdown_connections 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- 172.21.15.87:0/2590932493 wait complete. 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.126+0000 7f7337ea8b80 1 -- start start 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 Processor -- start 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f2f72aa060 con 0x55f2f72a35f0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2f718ed80 con 0x55f2f72a2cd0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2f72aa1e0 con 0x55f2f729d460 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56678/0 (socket says 172.21.15.87:56678) 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 -- 172.21.15.87:0/3918654801 learned_addr learned my addr 172.21.15.87:0/3918654801 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7337ea8b80 10 client.0 _ll_register_callbacks cb 0x55f2f7188ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f7326ffd700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73267fc700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 -- 172.21.15.87:0/3918654801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 msgr2=0x55f2f72a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.127+0000 7f73277fe700 1 -- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 msgr2=0x55f2f72a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f73277fe700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f73277fe700 1 -- 172.21.15.87:0/3918654801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f713da60 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f73267fc700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f73277fe700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7320009fd0 tx=0x7f732009c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aa9c43a549582e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f730aff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200b5060 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f73200a1e00 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c004530 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f73200a0c70 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f7337ea8b80 10 client.17865 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.128+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f2f72aa060 con 0x55f2f72a35f0 2024-06-06T08:05:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.129+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c004530 con 0x55f2f72a35f0 2024-06-06T08:05:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.129+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f732002e060 con 0x55f2f72a35f0 2024-06-06T08:05:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 10 client.17865 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 10 client.17865 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 20 client.17865 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:08.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f72b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 4 client.17865 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:08.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2f71fbea0 con 0x55f2f72b6e80 2024-06-06T08:05:08.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7326ffd700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f72b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f7326ffd700 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f72b92c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c3d812242e001a server_cookie=4c269fd346f480b3 in_seq=0 out_seq=0 2024-06-06T08:05:08.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.130+0000 7f730aff5700 10 client.17865 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f730aff5700 1 -- 172.21.15.87:0/3918654801 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2f71fbea0 con 0x55f2f72b6e80 2024-06-06T08:05:08.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f730aff5700 10 client.17865 handle_command_reply: tid=0 2024-06-06T08:05:08.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 client.17865 shutdown 2024-06-06T08:05:08.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 msgr2=0x55f2f72a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7320009fd0 tx=0x7f732009c320 comp rx=0 tx=0).stop 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 shutdown_connections 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f72b92c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f730aff5700 0 client.17865 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a35f0 0x55f2f72a6b30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/3918654801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.135+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 >> 172.21.15.87:0/3918654801 conn(0x55f2f71c7e50 msgr2=0x55f2f729bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.136+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 shutdown_connections 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.136+0000 7f7337ea8b80 1 -- 172.21.15.87:0/3918654801 wait complete. 2024-06-06T08:05:08.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.136+0000 7f7337ea8b80 20 client.17865 trim_cache size 0 max 16384 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.161+0000 7f7337ea8b80 1 Processor -- start 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.161+0000 7f7337ea8b80 1 -- start start 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a35f0 0x55f2f72a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73281b2ed0 con 0x55f2f72a2cd0 2024-06-06T08:05:08.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73281b0700 con 0x55f2f72a35f0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73281ae550 con 0x55f2f729d460 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56682/0 (socket says 172.21.15.87:56682) 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f7326ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 -- 172.21.15.87:0/459352872 learned_addr learned my addr 172.21.15.87:0/459352872 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a35f0 0x55f2f72a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 -- 172.21.15.87:0/459352872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 msgr2=0x55f2f72a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 -- 172.21.15.87:0/459352872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a35f0 msgr2=0x55f2f72a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a35f0 0x55f2f72a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 -- 172.21.15.87:0/459352872 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f7299540 con 0x55f2f72a2cd0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f73267fc700 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f731c009c90 tx=0x7f731c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebbd23b1ee1b67a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.162+0000 7f730cff9700 1 -- 172.21.15.87:0/459352872 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c011b20 con 0x55f2f72a2cd0 2024-06-06T08:05:08.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f730cff9700 1 -- 172.21.15.87:0/459352872 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c027070 con 0x55f2f72a2cd0 2024-06-06T08:05:08.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f730cff9700 1 -- 172.21.15.87:0/459352872 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c003be0 con 0x55f2f72a2cd0 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 -- 172.21.15.87:0/459352872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 msgr2=0x55f2f72a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f731c009c90 tx=0x7f731c005d40 comp rx=0 tx=0).stop 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 -- 172.21.15.87:0/459352872 shutdown_connections 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f72a35f0 0x55f2f72a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f729d460 0x55f2f72a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/459352872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72a3a90 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 -- 172.21.15.87:0/459352872 >> 172.21.15.87:0/459352872 conn(0x55f2f71c7e50 msgr2=0x55f2f72a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 -- 172.21.15.87:0/459352872 shutdown_connections 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.163+0000 7f7337ea8b80 1 -- 172.21.15.87:0/459352872 wait complete. 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 -- start start 2024-06-06T08:05:08.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 Processor -- start 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f72a35f0 0x55f2f72aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73281aebc0 con 0x55f2f72a2cd0 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73281b0030 con 0x55f2f729d460 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73281af290 con 0x55f2f72a35f0 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7326ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56696/0 (socket says 172.21.15.87:56696) 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 -- 172.21.15.87:0/151928555 learned_addr learned my addr 172.21.15.87:0/151928555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7337ea8b80 10 client.0 _ll_register_callbacks cb 0x55f2f7188ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f7326ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:45122/0 (socket says 172.21.15.87:45122) 2024-06-06T08:05:08.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73277fe700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f72a35f0 0x55f2f72aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 -- 172.21.15.87:0/151928555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f72a35f0 msgr2=0x55f2f72aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f72a35f0 0x55f2f72aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.164+0000 7f73267fc700 1 -- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 msgr2=0x55f2f72afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f73267fc700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f73267fc700 1 -- 172.21.15.87:0/151928555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f72ad270 con 0x55f2f72a2cd0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f7326ffd700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f73267fc700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f731c000c00 tx=0x7f731c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8258a9c70e1404 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f730aff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c011b20 con 0x55f2f72a2cd0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f731c011cc0 con 0x55f2f72a2cd0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c027070 con 0x55f2f72a2cd0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f731c039050 con 0x55f2f72a2cd0 2024-06-06T08:05:08.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f7337ea8b80 10 client.17871 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f2f7322c90 con 0x55f2f72a2cd0 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.165+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c027070 con 0x55f2f72a2cd0 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f731c018070 con 0x55f2f72a2cd0 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 10 client.17871 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 10 client.17871 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 20 client.17871 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f73251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 4 client.17871 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.7, "await_for": 45.0}] 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.7, "await_for": 45.0}) -- 0x55f2f71f6420 con 0x55f2f72b6e80 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7326ffd700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f73251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f730aff5700 10 client.17871 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:08.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:08.166+0000 7f7326ffd700 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f73251a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3bae3a296a1310 server_cookie=daedd573fcfd8096 in_seq=0 out_seq=0 2024-06-06T08:05:08.171 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48955 17.85 MB/sec execute 70 sec latency 2288.315 ms 2024-06-06T08:05:08.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:08 smithi181 ceph-mon[36790]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-06-06T08:05:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:08 smithi195 ceph-mon[37040]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-06-06T08:05:09.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f730aff5700 1 -- 172.21.15.87:0/151928555 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f2f71f6420 con 0x55f2f72b6e80 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f730aff5700 10 client.17871 handle_command_reply: tid=0 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 client.17871 shutdown 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 msgr2=0x55f2f72ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f731c000c00 tx=0x7f731c0052f0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 shutdown_connections 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2f72b6e80 0x55f2f73251a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f729d460 0x55f2f72afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2f72a35f0 0x55f2f72aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.103+0000 7f730aff5700 0 client.17871 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f7337ea8b80 1 --2- 172.21.15.87:0/151928555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2f72a2cd0 0x55f2f72ac470 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 >> 172.21.15.87:0/151928555 conn(0x55f2f71c7e50 msgr2=0x55f2f72a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:09.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 shutdown_connections 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f7337ea8b80 1 -- 172.21.15.87:0/151928555 wait complete. 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f7337ea8b80 20 client.17871 trim_cache size 0 max 16384 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 52, 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 50, 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "daea42d4": { 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 50, 2024-06-06T08:05:09.105 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 135.7, 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:05:09.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:09.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:09.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:09.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:09.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:09.107 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 msgr2=0x7f73100a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 0x7f73100a52f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7328106280 tx=0x7f7318009260 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 msgr2=0x7f7328198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f7328198960 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f732000b770 tx=0x7f7320007a20 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 shutdown_connections 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f73100a2e30 0x7f73100a52f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73281066c0 0x7f7328198960 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73281015f0 0x7f732819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 --2- 172.21.15.87:0/439355429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7328101fa0 0x7f732819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:09.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.104+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 >> 172.21.15.87:0/439355429 conn(0x7f7328078330 msgr2=0x7f732806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:09.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.105+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 shutdown_connections 2024-06-06T08:05:09.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:09.105+0000 7f732d625700 1 -- 172.21.15.87:0/439355429 wait complete. 2024-06-06T08:05:09.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 17.60 MB/sec execute 71 sec latency 1664.954 ms 2024-06-06T08:05:09.815 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'daea42d4', quiesce took 7.1 seconds. Will release after: 8.6 2024-06-06T08:05:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:10 smithi087 ceph-mon[32276]: pgmap v544: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-06T08:05:10.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 17.36 MB/sec execute 72 sec latency 2665.042 ms 2024-06-06T08:05:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:10 smithi181 ceph-mon[36790]: pgmap v544: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-06T08:05:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:10 smithi195 ceph-mon[37040]: pgmap v544: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-06-06T08:05:11.170 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 17.12 MB/sec execute 73 sec latency 3665.095 ms 2024-06-06T08:05:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 -- 172.21.15.87:0/3720972872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc1008d0 msgr2=0x7fdecc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 --2- 172.21.15.87:0/3720972872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc1008d0 0x7fdecc100d50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009930 tx=0x7fdec002a420 comp rx=0 tx=0).stop 2024-06-06T08:05:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 -- 172.21.15.87:0/3720972872 shutdown_connections 2024-06-06T08:05:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 --2- 172.21.15.87:0/3720972872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc101290 0x7fdecc10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 --2- 172.21.15.87:0/3720972872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc1008d0 0x7fdecc100d50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 --2- 172.21.15.87:0/3720972872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc106890 0x7fdecc106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 -- 172.21.15.87:0/3720972872 >> 172.21.15.87:0/3720972872 conn(0x7fdecc077a20 msgr2=0x7fdecc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 -- 172.21.15.87:0/3720972872 shutdown_connections 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.317+0000 7fded28a5640 1 -- 172.21.15.87:0/3720972872 wait complete. 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 Processor -- start 2024-06-06T08:05:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 -- start start 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 0x7fdecc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc101290 0x7fdecc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc111da0 con 0x7fdecc106890 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc111c20 con 0x7fdecc1008d0 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded28a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc111f20 con 0x7fdecc101290 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded0e42640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded0e42640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56724/0 (socket says 172.21.15.87:56724) 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fded0e42640 1 -- 172.21.15.87:0/1105695505 learned_addr learned my addr 172.21.15.87:0/1105695505 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc101290 0x7fdecc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.318+0000 7fdecbfff640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 0x7fdecc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 -- 172.21.15.87:0/1105695505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc101290 msgr2=0x7fdecc1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc101290 0x7fdecc1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 -- 172.21.15.87:0/1105695505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 msgr2=0x7fdecc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 0x7fdecc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 -- 172.21.15.87:0/1105695505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc19d240 con 0x7fdecc106890 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fdecbfff640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 0x7fdecc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded0e42640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdec400eaa0 tx=0x7fdec400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fdec97fa640 1 -- 172.21.15.87:0/1105695505 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec400cdb0 con 0x7fdecc106890 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fdec97fa640 1 -- 172.21.15.87:0/1105695505 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec4005020 con 0x7fdecc106890 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc19d530 con 0x7fdecc106890 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fdec97fa640 1 -- 172.21.15.87:0/1105695505 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec40116b0 con 0x7fdecc106890 2024-06-06T08:05:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.319+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc1a4dc0 con 0x7fdecc106890 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fdec97fa640 1 -- 172.21.15.87:0/1105695505 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdec4005560 con 0x7fdecc106890 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 Processor -- start 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 -- start start 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 0x560149f48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f48760 0x560149f4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560149e9d4f0 con 0x560149f3d580 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560149e3b870 con 0x560149f3dea0 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.322+0000 7fded55b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560149e994b0 con 0x560149f48760 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56730/0 (socket says 172.21.15.87:56730) 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 -- 172.21.15.87:0/299983437 learned_addr learned my addr 172.21.15.87:0/299983437 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecbfff640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 0x560149f48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f48760 0x560149f4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdec97fa640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 0x7fdeb40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 -- 172.21.15.87:0/299983437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f48760 msgr2=0x560149f4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f48760 0x560149f4ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 -- 172.21.15.87:0/299983437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 msgr2=0x560149f48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 0x560149f48220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 -- 172.21.15.87:0/299983437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560149e579b0 con 0x560149f3d580 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecbfff640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 0x560149f48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fded0e42640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 0x7fdeb40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe17c3f81c2aa61b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdec97fa640 1 -- 172.21.15.87:0/1105695505 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fdec40cf9d0 con 0x7fdecc106890 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdeb27fc640 1 -- 172.21.15.87:0/299983437 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0044030 con 0x560149f3d580 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdeb27fc640 1 -- 172.21.15.87:0/299983437 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec0031a20 con 0x560149f3d580 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.323+0000 7fdeb27fc640 1 -- 172.21.15.87:0/299983437 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0035040 con 0x560149f3d580 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- 172.21.15.87:0/299983437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 msgr2=0x560149f3d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded0e42640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 0x7fdeb40a52e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdec40cd350 tx=0x7fdec40054d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0005fd0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- 172.21.15.87:0/299983437 shutdown_connections 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f48760 0x560149f4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f3dea0 0x560149f48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 --2- 172.21.15.87:0/299983437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f3d960 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- 172.21.15.87:0/299983437 >> 172.21.15.87:0/299983437 conn(0x560149f39a90 msgr2=0x560149f39e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- 172.21.15.87:0/299983437 shutdown_connections 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- 172.21.15.87:0/299983437 wait complete. 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.324+0000 7fded55b2740 1 -- start start 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 Processor -- start 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f45bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f46110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560149f4b330 con 0x560149f3d580 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560149e26ae0 con 0x560149f48760 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560149f4b4b0 con 0x560149f3dea0 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fded55b2740 10 client.0 _ll_register_callbacks cb 0x560149e5f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56734/0 (socket says 172.21.15.87:56734) 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 -- 172.21.15.87:0/1599208024 learned_addr learned my addr 172.21.15.87:0/1599208024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecbfff640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f45bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 -- 172.21.15.87:0/1599208024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 msgr2=0x560149f45bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f45bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 -- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 msgr2=0x560149f46110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f46110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.325+0000 7fdecb7fe640 1 -- 172.21.15.87:0/1599208024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560149bb1210 con 0x560149f3d580 2024-06-06T08:05:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009340 tx=0x7fdec00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170e38b9f123ed11 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec004d020 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fdec0004970 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec0031a20 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdec0035070 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 10 client.17883 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560149c7d620 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec0031a20 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fdec0049bb0 con 0x560149f3d580 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 10 client.17883 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 20 client.17883 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149f586f0 0x560149f5aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 4 client.17883 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.326+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560149eb0f10 con 0x560149f586f0 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.327+0000 7fdecbfff640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149f586f0 0x560149f5aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.327+0000 7fdeab7fe640 10 client.17883 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.327+0000 7fdecbfff640 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149f586f0 0x560149f5aae0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3153f384388f84 server_cookie=bdf2c751242f2217 in_seq=0 out_seq=0 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fdeab7fe640 1 -- 172.21.15.87:0/1599208024 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560149eb0f10 con 0x560149f586f0 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fdeab7fe640 10 client.17883 handle_command_reply: tid=0 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 client.17883 shutdown 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 msgr2=0x560149f45690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009340 tx=0x7fdec00315b0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 shutdown_connections 2024-06-06T08:05:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149f586f0 0x560149f5aae0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f46110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f45bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fdeab7fe640 0 client.17883 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 --2- 172.21.15.87:0/1599208024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f45690 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 >> 172.21.15.87:0/1599208024 conn(0x560149f39a90 msgr2=0x560149f347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 shutdown_connections 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 1 -- 172.21.15.87:0/1599208024 wait complete. 2024-06-06T08:05:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.328+0000 7fded55b2740 20 client.17883 trim_cache size 0 max 16384 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 Processor -- start 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 -- start start 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3d580 0x560149f34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1af5a0 con 0x560149f3dea0 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1ace10 con 0x560149f48760 2024-06-06T08:05:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fded55b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1aac60 con 0x560149f3d580 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56744/0 (socket says 172.21.15.87:56744) 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 -- 172.21.15.87:0/925467815 learned_addr learned my addr 172.21.15.87:0/925467815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3d580 0x560149f34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 -- 172.21.15.87:0/925467815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3d580 msgr2=0x560149f34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3d580 0x560149f34660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 -- 172.21.15.87:0/925467815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 msgr2=0x560149f605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 -- 172.21.15.87:0/925467815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560149f398c0 con 0x560149f3dea0 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdecbfff640 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005fd0 tx=0x7fdebc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ff437f0569010 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.344+0000 7fdeb1ffb640 1 -- 172.21.15.87:0/925467815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc01e070 con 0x560149f3dea0 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fdeb1ffb640 1 -- 172.21.15.87:0/925467815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc012e20 con 0x560149f3dea0 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fdeb1ffb640 1 -- 172.21.15.87:0/925467815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc011ad0 con 0x560149f3dea0 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- 172.21.15.87:0/925467815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 msgr2=0x560149f60080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005fd0 tx=0x7fdebc005cf0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- 172.21.15.87:0/925467815 shutdown_connections 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3dea0 0x560149f60080 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 --2- 172.21.15.87:0/925467815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3d580 0x560149f34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- 172.21.15.87:0/925467815 >> 172.21.15.87:0/925467815 conn(0x560149f39a90 msgr2=0x560149f33f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- 172.21.15.87:0/925467815 shutdown_connections 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- 172.21.15.87:0/925467815 wait complete. 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.345+0000 7fded55b2740 1 -- start start 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 Processor -- start 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f4bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f4c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1ab2d0 con 0x560149f3d580 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1ac740 con 0x560149f48760 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1ab9a0 con 0x560149f3dea0 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fded55b2740 10 client.0 _ll_register_callbacks cb 0x7fdec002aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56756/0 (socket says 172.21.15.87:56756) 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 -- 172.21.15.87:0/2827208111 learned_addr learned my addr 172.21.15.87:0/2827208111 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecbfff640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f4bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f4c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 -- 172.21.15.87:0/2827208111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 msgr2=0x560149f4bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f4bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 -- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 msgr2=0x560149f4c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f4c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.346+0000 7fdecb7fe640 1 -- 172.21.15.87:0/2827208111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560149f4c920 con 0x560149f3d580 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdecb7fe640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdec0004720 tx=0x7fdec00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed67b2cab0909e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec004d040 con 0x560149f3d580 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fdec003adc0 con 0x560149f3d580 2024-06-06T08:05:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdebc012e20 con 0x560149f3d580 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdec0069050 con 0x560149f3d580 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 10 client.17889 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560149fc2d30 con 0x560149f3d580 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc012e20 con 0x560149f3d580 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fdec0049710 con 0x560149f3d580 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 10 client.17889 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 20 client.17889 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149fc3400 0x560149fc5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 4 client.17889 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.347+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560149f40410 con 0x560149fc3400 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.348+0000 7fdecbfff640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149fc3400 0x560149fc5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.348+0000 7fdecbfff640 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149fc3400 0x560149fc5820 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86708211ef1724a6 server_cookie=be5d908e68078526 in_seq=0 out_seq=0 2024-06-06T08:05:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.348+0000 7fdeab7fe640 10 client.17889 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.627+0000 7fdeab7fe640 1 -- 172.21.15.87:0/2827208111 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560149f40410 con 0x560149fc3400 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.627+0000 7fdeab7fe640 10 client.17889 handle_command_reply: tid=0 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.627+0000 7fded55b2740 1 client.17889 shutdown 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.627+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 msgr2=0x560149f50b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.627+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdec0004720 tx=0x7fdec00315b0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 shutdown_connections 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560149fc3400 0x560149fc5820 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fdeab7fe640 0 client.17889 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560149f48760 0x560149f4c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560149f3dea0 0x560149f4bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 --2- 172.21.15.87:0/2827208111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560149f3d580 0x560149f50b30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 >> 172.21.15.87:0/2827208111 conn(0x560149f39a90 msgr2=0x560149f3ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 shutdown_connections 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 1 -- 172.21.15.87:0/2827208111 wait complete. 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.628+0000 7fded55b2740 20 client.17889 trim_cache size 0 max 16384 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:05:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 msgr2=0x7fdeb40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 0x7fdeb40a52e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdec40cd350 tx=0x7fdec40054d0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 msgr2=0x7fdecc19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdec400eaa0 tx=0x7fdec400ef60 comp rx=0 tx=0).stop 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 shutdown_connections 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdeb40a2e20 0x7fdeb40a52e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc106890 0x7fdecc19cb10 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdecc101290 0x7fdecc1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 --2- 172.21.15.87:0/1105695505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc1008d0 0x7fdecc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 >> 172.21.15.87:0/1105695505 conn(0x7fdecc077a20 msgr2=0x7fdecc1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 shutdown_connections 2024-06-06T08:05:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:11.629+0000 7fded28a5640 1 -- 172.21.15.87:0/1105695505 wait complete. 2024-06-06T08:05:12.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 16.89 MB/sec execute 74 sec latency 4665.190 ms 2024-06-06T08:05:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:12 smithi181 ceph-mon[36790]: pgmap v545: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 721 op/s 2024-06-06T08:05:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:12 smithi087 ceph-mon[32276]: pgmap v545: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 721 op/s 2024-06-06T08:05:12.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:12 smithi195 ceph-mon[37040]: pgmap v545: 129 pgs: 129 active+clean; 404 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 721 op/s 2024-06-06T08:05:13.166 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 16.66 MB/sec execute 75 sec latency 5665.308 ms 2024-06-06T08:05:14.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 16.44 MB/sec execute 76 sec latency 6665.428 ms 2024-06-06T08:05:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:14 smithi181 ceph-mon[36790]: pgmap v546: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 880 op/s 2024-06-06T08:05:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:14 smithi087 ceph-mon[32276]: pgmap v546: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 880 op/s 2024-06-06T08:05:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:14 smithi195 ceph-mon[37040]: pgmap v546: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 880 op/s 2024-06-06T08:05:15.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 16.23 MB/sec execute 77 sec latency 7665.558 ms 2024-06-06T08:05:16.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 16.02 MB/sec execute 78 sec latency 8665.646 ms 2024-06-06T08:05:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:16 smithi181 ceph-mon[36790]: pgmap v547: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 466 op/s 2024-06-06T08:05:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:16 smithi087 ceph-mon[32276]: pgmap v547: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 466 op/s 2024-06-06T08:05:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:16 smithi195 ceph-mon[37040]: pgmap v547: 129 pgs: 129 active+clean; 402 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 466 op/s 2024-06-06T08:05:17.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 15.82 MB/sec execute 79 sec latency 9665.730 ms 2024-06-06T08:05:18.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 15.62 MB/sec execute 80 sec latency 10665.852 ms 2024-06-06T08:05:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:18 smithi181 ceph-mon[36790]: pgmap v548: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-06T08:05:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:18 smithi087 ceph-mon[32276]: pgmap v548: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-06T08:05:18.417 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:05:18.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:18 smithi195 ceph-mon[37040]: pgmap v548: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-06-06T08:05:18.669 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:19.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 15.43 MB/sec execute 81 sec latency 11665.896 ms 2024-06-06T08:05:20.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:20 smithi087 ceph-mon[32276]: pgmap v549: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-06-06T08:05:20.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 15.24 MB/sec execute 82 sec latency 12665.948 ms 2024-06-06T08:05:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:20 smithi181 ceph-mon[36790]: pgmap v549: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-06-06T08:05:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:20 smithi195 ceph-mon[37040]: pgmap v549: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 -- 172.21.15.87:0/1829658920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 msgr2=0x7f7658109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 --2- 172.21.15.87:0/1829658920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 0x7f7658109490 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f764c009990 tx=0x7f764c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 -- 172.21.15.87:0/1829658920 shutdown_connections 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 --2- 172.21.15.87:0/1829658920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 0x7f7658109490 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 --2- 172.21.15.87:0/1829658920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 0x7f765811e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 --2- 172.21.15.87:0/1829658920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f7658109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 -- 172.21.15.87:0/1829658920 >> 172.21.15.87:0/1829658920 conn(0x7f765806d600 msgr2=0x7f765806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 -- 172.21.15.87:0/1829658920 shutdown_connections 2024-06-06T08:05:20.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.688+0000 7f76603dd700 1 -- 172.21.15.87:0/1829658920 wait complete. 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.689+0000 7f76603dd700 1 Processor -- start 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.689+0000 7f76603dd700 1 -- start start 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 0x7f76581adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 0x7f76581a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76581216f0 con 0x7f765811bc70 2024-06-06T08:05:20.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7658121570 con 0x7f7658110c70 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f76603dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7658121870 con 0x7f76581099d0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e179700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e179700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:41904/0 (socket says 172.21.15.87:41904) 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e179700 1 -- 172.21.15.87:0/376114622 learned_addr learned my addr 172.21.15.87:0/376114622 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e97a700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 0x7f76581a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e179700 1 -- 172.21.15.87:0/376114622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 msgr2=0x7f76581adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.690+0000 7f765e179700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 0x7f76581adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f765e179700 1 -- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 msgr2=0x7f76581a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f765e179700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 0x7f76581a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f765e179700 1 -- 172.21.15.87:0/376114622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76581a80b0 con 0x7f76581099d0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f765e179700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f764c000c00 tx=0x7f764c003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c027070 con 0x7f76581099d0 2024-06-06T08:05:20.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f764c004050 con 0x7f76581099d0 2024-06-06T08:05:20.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.691+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c022d40 con 0x7f76581099d0 2024-06-06T08:05:20.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.692+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76581a8340 con 0x7f76581099d0 2024-06-06T08:05:20.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.692+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76581b4660 con 0x7f76581099d0 2024-06-06T08:05:20.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.693+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7658069180 con 0x7f76581099d0 2024-06-06T08:05:20.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.695+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f764c022470 con 0x7f76581099d0 2024-06-06T08:05:20.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.695+0000 7f76577fe700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 0x7f76400a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:20.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.695+0000 7f765d978700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 0x7f76400a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:20.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.695+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f764c0d0d80 con 0x7f76581099d0 2024-06-06T08:05:20.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.696+0000 7f765d978700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 0x7f76400a53a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76500060b0 tx=0x7f7650006040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:20.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.701+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f764c013070 con 0x7f76581099d0 2024-06-06T08:05:20.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.951+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76581b4830 con 0x7f76581099d0 2024-06-06T08:05:20.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.952+0000 7f76577fe700 1 -- 172.21.15.87:0/376114622 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11881 (secure 0 0 0) 0x7f764c09d600 con 0x7f76581099d0 2024-06-06T08:05:20.951 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:05:20.951 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:04:59.022587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:05:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 msgr2=0x7f76400a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 0x7f76400a53a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76500060b0 tx=0x7f7650006040 comp rx=0 tx=0).stop 2024-06-06T08:05:20.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 msgr2=0x7f76581ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f764c000c00 tx=0x7f764c003890 comp rx=0 tx=0).stop 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 shutdown_connections 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76400a2ee0 0x7f76400a53a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f765811bc70 0x7f76581a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76581099d0 0x7f76581ad790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 --2- 172.21.15.87:0/376114622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7658110c70 0x7f76581adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 >> 172.21.15.87:0/376114622 conn(0x7f765806d600 msgr2=0x7f7658119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 shutdown_connections 2024-06-06T08:05:20.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:20.954+0000 7f76603dd700 1 -- 172.21.15.87:0/376114622 wait complete. 2024-06-06T08:05:20.955 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 52 2024-06-06T08:05:21.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 15.06 MB/sec execute 83 sec latency 13665.982 ms 2024-06-06T08:05:21.288 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:21 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/376114622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:21.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:21 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/376114622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:21.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:21 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/376114622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:21.631 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id daea42d4 --release --await' 2024-06-06T08:05:21.631 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id daea42d4 --release --await 2024-06-06T08:05:21.872 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:22.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 14.88 MB/sec execute 84 sec latency 14666.047 ms 2024-06-06T08:05:22.245 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:22 smithi087 ceph-mon[32276]: pgmap v550: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-06-06T08:05:22.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:22 smithi195 ceph-mon[37040]: pgmap v550: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-06-06T08:05:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:22 smithi181 ceph-mon[36790]: pgmap v550: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-06-06T08:05:23.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 48982 14.70 MB/sec execute 85 sec latency 15666.080 ms 2024-06-06T08:05:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 -- 172.21.15.87:0/1732356168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 msgr2=0x7f6c3010f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/1732356168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c3010f3f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b0d0 tx=0x7f6c2800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 -- 172.21.15.87:0/1732356168 shutdown_connections 2024-06-06T08:05:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/1732356168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 0x7f6c30102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/1732356168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c3010f3f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/1732356168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 0x7f6c30107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 -- 172.21.15.87:0/1732356168 >> 172.21.15.87:0/1732356168 conn(0x7f6c30079570 msgr2=0x7f6c30079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 -- 172.21.15.87:0/1732356168 shutdown_connections 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.724+0000 7f6c37a1f700 1 -- 172.21.15.87:0/1732356168 wait complete. 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 Processor -- start 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 -- start start 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 0x7f6c3019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 0x7f6c3019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30112810 con 0x7f6c30107870 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30112690 con 0x7f6c301027a0 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.725+0000 7f6c37a1f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30112990 con 0x7f6c30103150 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55758/0 (socket says 172.21.15.87:55758) 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 -- 172.21.15.87:0/2601412612 learned_addr learned my addr 172.21.15.87:0/2601412612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c357bb700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 0x7f6c3019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c34fba700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 0x7f6c3019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 -- 172.21.15.87:0/2601412612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 msgr2=0x7f6c3019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 0x7f6c3019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 -- 172.21.15.87:0/2601412612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 msgr2=0x7f6c3019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 0x7f6c3019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 -- 172.21.15.87:0/2601412612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c301990f0 con 0x7f6c30107870 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f6c28006070 tx=0x7f6c28004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c267fc700 1 -- 172.21.15.87:0/2601412612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28021070 con 0x7f6c30107870 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c30199380 con 0x7f6c30107870 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c267fc700 1 -- 172.21.15.87:0/2601412612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28007d00 con 0x7f6c30107870 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c267fc700 1 -- 172.21.15.87:0/2601412612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28005470 con 0x7f6c30107870 2024-06-06T08:05:23.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.726+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c301a56a0 con 0x7f6c30107870 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.729+0000 7f6c267fc700 1 -- 172.21.15.87:0/2601412612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6c28024070 con 0x7f6c30107870 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.729+0000 7f6c267fc700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 0x7f6c1c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.729+0000 7f6c357bb700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 0x7f6c1c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.729+0000 7f6c267fc700 1 -- 172.21.15.87:0/2601412612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f6c2800f020 con 0x7f6c30107870 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c357bb700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 0x7f6c1c0a5250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c30199ad0 tx=0x7f6c2c009190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c422a2b80 1 Processor -- start 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c422a2b80 1 -- start start 2024-06-06T08:05:23.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cd7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.730+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cd1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eff7c22910 con 0x55eff7cd7560 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eff7bca820 con 0x55eff7cd13d0 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eff7c1e8d0 con 0x55eff7cd6c40 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55774/0 (socket says 172.21.15.87:55774) 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 -- 172.21.15.87:0/408070324 learned_addr learned my addr 172.21.15.87:0/408070324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c357bb700 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cd7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cd1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 -- 172.21.15.87:0/408070324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 msgr2=0x55eff7cd7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cd7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 -- 172.21.15.87:0/408070324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 msgr2=0x55eff7cd1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cd1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 -- 172.21.15.87:0/408070324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff7bc1050 con 0x55eff7cd7560 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c34fba700 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000a2b0 tx=0x7f6c2000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e948bc5da76e0cf4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c1b7fe700 1 -- 172.21.15.87:0/408070324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2001f070 con 0x55eff7cd7560 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c1b7fe700 1 -- 172.21.15.87:0/408070324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2000ae50 con 0x55eff7cd7560 2024-06-06T08:05:23.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.731+0000 7f6c1b7fe700 1 -- 172.21.15.87:0/408070324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2001a6d0 con 0x55eff7cd7560 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- 172.21.15.87:0/408070324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 msgr2=0x55eff7cd0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000a2b0 tx=0x7f6c2000a5c0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- 172.21.15.87:0/408070324 shutdown_connections 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cd1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cd7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/408070324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd0e90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- 172.21.15.87:0/408070324 >> 172.21.15.87:0/408070324 conn(0x55eff7bfbdc0 msgr2=0x55eff7ccc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- 172.21.15.87:0/408070324 shutdown_connections 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- 172.21.15.87:0/408070324 wait complete. 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.732+0000 7f6c422a2b80 1 -- start start 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 Processor -- start 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cdd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55eff7cddfd0 con 0x55eff7cd7560 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eff7bc2cf0 con 0x55eff7cd13d0 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eff7cde150 con 0x55eff7cd6c40 2024-06-06T08:05:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c422a2b80 10 client.0 _ll_register_callbacks cb 0x55eff7bbca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55784/0 (socket says 172.21.15.87:55784) 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c35fbc700 1 -- 172.21.15.87:0/141255237 learned_addr learned my addr 172.21.15.87:0/141255237 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c357bb700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cdd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.733+0000 7f6c34fba700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 -- 172.21.15.87:0/141255237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 msgr2=0x55eff7cda560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cda560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 -- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 msgr2=0x55eff7cdd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cdd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 -- 172.21.15.87:0/141255237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff7b719d0 con 0x55eff7cd7560 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6c2801de40 tx=0x7f6c2809caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8ff64f36ab553d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a14f0 con 0x55eff7cd7560 2024-06-06T08:05:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f6c280074e0 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2000ae50 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6c280a5d20 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 10 client.17904 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55eff7ce8890 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2000ae50 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f6c28011070 con 0x55eff7cd7560 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 10 client.17904 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 10 client.17904 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 20 client.17904 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7ceae50 0x55eff7ced270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.734+0000 7f6c422a2b80 4 client.17904 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.735+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eff7c2fe10 con 0x55eff7ceae50 2024-06-06T08:05:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.735+0000 7f6c357bb700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7ceae50 0x55eff7ced270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.735+0000 7f6c357bb700 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7ceae50 0x55eff7ced270 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b970733425a5de server_cookie=1302d171effe361b in_seq=0 out_seq=0 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.735+0000 7f6c18ff9700 10 client.17904 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c18ff9700 1 -- 172.21.15.87:0/141255237 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eff7c2fe10 con 0x55eff7ceae50 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c18ff9700 10 client.17904 handle_command_reply: tid=0 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 client.17904 shutdown 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 msgr2=0x55eff7cdaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6c2801de40 tx=0x7f6c2809caa0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 shutdown_connections 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7ceae50 0x55eff7ced270 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd13d0 0x55eff7cdd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cdaaa0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/141255237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7cda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c18ff9700 0 client.17904 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:23.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 >> 172.21.15.87:0/141255237 conn(0x55eff7bfbdc0 msgr2=0x55eff7ccfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 shutdown_connections 2024-06-06T08:05:23.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 1 -- 172.21.15.87:0/141255237 wait complete. 2024-06-06T08:05:23.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.736+0000 7f6c422a2b80 20 client.17904 trim_cache size 0 max 16384 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.758+0000 7f6c422a2b80 1 Processor -- start 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.758+0000 7f6c422a2b80 1 -- start start 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.758+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd13d0 0x55eff7cd0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.758+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 0x55eff7cd0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.758+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301aebf0 con 0x55eff7cd7560 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301ac420 con 0x55eff7cd6c40 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301aa270 con 0x55eff7cd13d0 2024-06-06T08:05:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55798/0 (socket says 172.21.15.87:55798) 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 -- 172.21.15.87:0/988953036 learned_addr learned my addr 172.21.15.87:0/988953036 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c34fba700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 0x55eff7cd0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c357bb700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd13d0 0x55eff7cd0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 -- 172.21.15.87:0/988953036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd13d0 msgr2=0x55eff7cd0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd13d0 0x55eff7cd0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 -- 172.21.15.87:0/988953036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 msgr2=0x55eff7cd0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 0x55eff7cd0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 -- 172.21.15.87:0/988953036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff7ced8d0 con 0x55eff7cd7560 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c34fba700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 0x55eff7cd0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6c2809d0d0 tx=0x7f6c2800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f0eae0f4011b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c1affd700 1 -- 172.21.15.87:0/988953036 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a14f0 con 0x55eff7cd7560 2024-06-06T08:05:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c1affd700 1 -- 172.21.15.87:0/988953036 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c280074e0 con 0x55eff7cd7560 2024-06-06T08:05:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.759+0000 7f6c1affd700 1 -- 172.21.15.87:0/988953036 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2809d600 con 0x55eff7cd7560 2024-06-06T08:05:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- 172.21.15.87:0/988953036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 msgr2=0x55eff7cd7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6c2809d0d0 tx=0x7f6c2800b9d0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- 172.21.15.87:0/988953036 shutdown_connections 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd7560 0x55eff7cd7940 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd13d0 0x55eff7cd0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/988953036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd6c40 0x55eff7cd0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- 172.21.15.87:0/988953036 >> 172.21.15.87:0/988953036 conn(0x55eff7bfbdc0 msgr2=0x55eff7cda890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- 172.21.15.87:0/988953036 shutdown_connections 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- 172.21.15.87:0/988953036 wait complete. 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 -- start start 2024-06-06T08:05:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.760+0000 7f6c422a2b80 1 Processor -- start 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7ce0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 0x55eff7ce0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301aa8e0 con 0x55eff7cd13d0 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301abd50 con 0x55eff7cd7560 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c301aafb0 con 0x55eff7cd6c40 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c422a2b80 10 client.0 _ll_register_callbacks cb 0x55eff7bbca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55810/0 (socket says 172.21.15.87:55810) 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 -- 172.21.15.87:0/1389273693 learned_addr learned my addr 172.21.15.87:0/1389273693 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 0x55eff7ce0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c34fba700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7ce0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 -- 172.21.15.87:0/1389273693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 msgr2=0x55eff7ce0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7ce0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 -- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 msgr2=0x55eff7ce0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 0x55eff7ce0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c357bb700 1 -- 172.21.15.87:0/1389273693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff7d55b40 con 0x55eff7cd13d0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c34fba700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7ce0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.761+0000 7f6c35fbc700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 0x55eff7ce0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c357bb700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55eff7cedad0 tx=0x7f6c2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61d7d2dfb98256a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c01b040 con 0x55eff7cd13d0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f6c2c015a70 con 0x55eff7cd13d0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c280074e0 con 0x55eff7cd13d0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c03b050 con 0x55eff7cd13d0 2024-06-06T08:05:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 10 client.17910 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55eff7d56ee0 con 0x55eff7cd13d0 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c280074e0 con 0x55eff7cd13d0 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f6c2c02dcc0 con 0x55eff7cd13d0 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 10 client.17910 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 10 client.17910 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 20 client.17910 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7d576b0 0x55eff7d59aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 4 client.17910 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "daea42d4", "release": true, "await": true}] 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.762+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "daea42d4", "release": true, "await": true}) -- 0x55eff7c2a390 con 0x55eff7d576b0 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.763+0000 7f6c34fba700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7d576b0 0x55eff7d59aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.763+0000 7f6c18ff9700 10 client.17910 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.763+0000 7f6c34fba700 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7d576b0 0x55eff7d59aa0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a5e2b066f5444e server_cookie=39535cd63a9b579b in_seq=0 out_seq=0 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c18ff9700 1 -- 172.21.15.87:0/1389273693 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eff7c2a390 con 0x55eff7d576b0 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c18ff9700 10 client.17910 handle_command_reply: tid=0 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 client.17910 shutdown 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 msgr2=0x55eff7ce0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55eff7cedad0 tx=0x7f6c2c004750 comp rx=0 tx=0).stop 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 shutdown_connections 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55eff7d576b0 0x55eff7d59aa0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eff7cd7560 0x55eff7ce0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55eff7cd13d0 0x55eff7ce0410 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c18ff9700 0 client.17910 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 --2- 172.21.15.87:0/1389273693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eff7cd6c40 0x55eff7ce0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 >> 172.21.15.87:0/1389273693 conn(0x55eff7bfbdc0 msgr2=0x55eff7cd0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 shutdown_connections 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 1 -- 172.21.15.87:0/1389273693 wait complete. 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.852+0000 7f6c422a2b80 20 client.17910 trim_cache size 0 max 16384 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 52, 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:05:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 52, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "daea42d4": { 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 52, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 135.7, 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:05:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:05:23.855 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 msgr2=0x7f6c1c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 0x7f6c1c0a5250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c30199ad0 tx=0x7f6c2c009190 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 msgr2=0x7f6c30198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f6c28006070 tx=0x7f6c28004e40 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 shutdown_connections 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c301027a0 0x7f6c3019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6c1c0a2d90 0x7f6c1c0a5250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c30107870 0x7f6c30198990 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 --2- 172.21.15.87:0/2601412612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c30103150 0x7f6c3019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 >> 172.21.15.87:0/2601412612 conn(0x7f6c30079570 msgr2=0x7f6c30100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 shutdown_connections 2024-06-06T08:05:23.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:23.853+0000 7f6c37a1f700 1 -- 172.21.15.87:0/2601412612 wait complete. 2024-06-06T08:05:24.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 49094 14.59 MB/sec execute 86 sec latency 16388.133 ms 2024-06-06T08:05:24.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:24 smithi087 ceph-mon[32276]: pgmap v551: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-06-06T08:05:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:24 smithi195 ceph-mon[37040]: pgmap v551: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-06-06T08:05:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:24 smithi181 ceph-mon[36790]: pgmap v551: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-06-06T08:05:24.566 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'daea42d4', total seconds elapsed: 21.8 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/3308326463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f64106890 msgr2=0x7f1f64106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/3308326463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f64106890 0x7f1f64106c70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009930 tx=0x7f1f5402a420 comp rx=0 tx=0).stop 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/3308326463 shutdown_connections 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/3308326463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 0x7f1f6410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/3308326463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f641008d0 0x7f1f64100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/3308326463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f64106890 0x7f1f64106c70 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/3308326463 >> 172.21.15.87:0/3308326463 conn(0x7f1f64077a20 msgr2=0x7f1f64077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/3308326463 shutdown_connections 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.728+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/3308326463 wait complete. 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 Processor -- start 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 -- start start 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 0x7f1f64198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f64106890 0x7f1f6419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64110ca0 con 0x7f1f641008d0 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64110b20 con 0x7f1f64101290 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f6a7e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64110e20 con 0x7f1f64106890 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55830/0 (socket says 172.21.15.87:55830) 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f63fff640 1 -- 172.21.15.87:0/340536179 learned_addr learned my addr 172.21.15.87:0/340536179 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f637fe640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 0x7f1f64198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.729+0000 7f1f68d84640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f64106890 0x7f1f6419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 -- 172.21.15.87:0/340536179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f64106890 msgr2=0x7f1f6419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f64106890 0x7f1f6419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 -- 172.21.15.87:0/340536179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 msgr2=0x7f1f64198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 0x7f1f64198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 -- 172.21.15.87:0/340536179 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6419d1b0 con 0x7f1f641008d0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f63fff640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009230 tx=0x7f1f54004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f617fa640 1 -- 172.21.15.87:0/340536179 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5403f030 con 0x7f1f641008d0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6419d440 con 0x7f1f641008d0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f617fa640 1 -- 172.21.15.87:0/340536179 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f54004440 con 0x7f1f641008d0 2024-06-06T08:05:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f617fa640 1 -- 172.21.15.87:0/340536179 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54035070 con 0x7f1f641008d0 2024-06-06T08:05:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.730+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f641a4d30 con 0x7f1f641008d0 2024-06-06T08:05:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 Processor -- start 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 -- start start 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f617fa640 1 -- 172.21.15.87:0/340536179 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1f54042030 con 0x7f1f641008d0 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf8e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 0x5615ebf98e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5615ebef24f0 con 0x5615ebf99350 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615ebe90870 con 0x5615ebf8e170 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615ebeee4b0 con 0x5615ebf8ea90 2024-06-06T08:05:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 0x5615ebf98e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f68d84640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf8e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55836/0 (socket says 172.21.15.87:55836) 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 -- 172.21.15.87:0/2659824054 learned_addr learned my addr 172.21.15.87:0/2659824054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 -- 172.21.15.87:0/2659824054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 msgr2=0x5615ebf98e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 0x5615ebf98e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 -- 172.21.15.87:0/2659824054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 msgr2=0x5615ebf8e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf8e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f637fe640 1 -- 172.21.15.87:0/2659824054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ebeac9b0 con 0x5615ebf99350 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.733+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 0x5615ebf98e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f617fa640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 0x7f1f4c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009ca0 tx=0x7f1f58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526713129091219 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f68d84640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 0x7f1f4c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f617fa640 1 -- 172.21.15.87:0/340536179 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f1f540efb00 con 0x7f1f641008d0 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f4a7fc640 1 -- 172.21.15.87:0/2659824054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801e070 con 0x5615ebf99350 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f4a7fc640 1 -- 172.21.15.87:0/2659824054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f58019b10 con 0x5615ebf99350 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.734+0000 7f1f4a7fc640 1 -- 172.21.15.87:0/2659824054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800ea60 con 0x5615ebf99350 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2659824054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 msgr2=0x5615ebf9b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009ca0 tx=0x7f1f58011810 comp rx=0 tx=0).stop 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f68d84640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 0x7f1f4c0a5260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2659824054 shutdown_connections 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf99350 0x5615ebf9b740 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8ea90 0x5615ebf98e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2659824054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf8e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2659824054 >> 172.21.15.87:0/2659824054 conn(0x5615ebf8a680 msgr2=0x5615ebf8aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2659824054 shutdown_connections 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2659824054 wait complete. 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.735+0000 7f1f6d5d4740 1 -- start start 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 Processor -- start 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5615ebf9bf80 con 0x5615ebf8ea90 2024-06-06T08:05:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615ebe7bae0 con 0x5615ebf8e170 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615ebf9c100 con 0x5615ebf99350 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f6d5d4740 10 client.0 _ll_register_callbacks cb 0x5615ebeb49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55844/0 (socket says 172.21.15.87:55844) 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 -- 172.21.15.87:0/2180838803 learned_addr learned my addr 172.21.15.87:0/2180838803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f68d84640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 -- 172.21.15.87:0/2180838803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 msgr2=0x5615ebf96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 -- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 msgr2=0x5615ebf962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 -- 172.21.15.87:0/2180838803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ebc06210 con 0x5615ebf8ea90 2024-06-06T08:05:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1f5801c040 tx=0x7f1f58019aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b64502c203e68c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.736+0000 7f1f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58027020 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f58019b10 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f1f58005460 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f58032050 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 10 client.17922 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5615ebcd2620 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f58019b10 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f1f58004880 con 0x5615ebf8ea90 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 10 client.17922 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 20 client.17922 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ebfa9440 0x5615ebfab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 4 client.17922 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ebf05f10 con 0x5615ebfa9440 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.737+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ebfa9440 0x5615ebfab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.738+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ebfa9440 0x5615ebfab830 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2baf3dffcd625a76 server_cookie=957d3f55cd9fb5be in_seq=0 out_seq=0 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.738+0000 7f1f437fe640 10 client.17922 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f437fe640 1 -- 172.21.15.87:0/2180838803 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5615ebf05f10 con 0x5615ebfa9440 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f437fe640 10 client.17922 handle_command_reply: tid=0 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 client.17922 shutdown 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 msgr2=0x5615ebf96830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1f5801c040 tx=0x7f1f58019aa0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 shutdown_connections 2024-06-06T08:05:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ebfa9440 0x5615ebfab830 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f437fe640 0 client.17922 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf96830 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2180838803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebf962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 >> 172.21.15.87:0/2180838803 conn(0x5615ebf8a680 msgr2=0x5615ebf903b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 shutdown_connections 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2180838803 wait complete. 2024-06-06T08:05:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.740+0000 7f1f6d5d4740 20 client.17922 trim_cache size 0 max 16384 2024-06-06T08:05:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.756+0000 7f1f6d5d4740 1 Processor -- start 2024-06-06T08:05:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.756+0000 7f1f6d5d4740 1 -- start start 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebfb10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf93550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641b1790 con 0x5615ebf8ea90 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641af000 con 0x5615ebf8e170 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ace50 con 0x5615ebf99350 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55852/0 (socket says 172.21.15.87:55852) 2024-06-06T08:05:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 -- 172.21.15.87:0/2377282311 learned_addr learned my addr 172.21.15.87:0/2377282311 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f63fff640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebfb10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f68d84640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf93550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 -- 172.21.15.87:0/2377282311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 msgr2=0x5615ebf93550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf93550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 -- 172.21.15.87:0/2377282311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 msgr2=0x5615ebfb10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebfb10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f637fe640 1 -- 172.21.15.87:0/2377282311 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ebf85f70 con 0x5615ebf8ea90 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.757+0000 7f1f68d84640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf93550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:05:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f637fe640 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f580059f0 tx=0x7f1f58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f2017586b4e71e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f49ffb640 1 -- 172.21.15.87:0/2377282311 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58027070 con 0x5615ebf8ea90 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f49ffb640 1 -- 172.21.15.87:0/2377282311 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f5801f040 con 0x5615ebf8ea90 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f49ffb640 1 -- 172.21.15.87:0/2377282311 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801e070 con 0x5615ebf8ea90 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2377282311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 msgr2=0x5615ebfb1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f580059f0 tx=0x7f1f58011810 comp rx=0 tx=0).stop 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2377282311 shutdown_connections 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf99350 0x5615ebf93550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebfb1610 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/2377282311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf8e170 0x5615ebfb10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.758+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2377282311 >> 172.21.15.87:0/2377282311 conn(0x5615ebf8a680 msgr2=0x5615ebf91630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2377282311 shutdown_connections 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/2377282311 wait complete. 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- start start 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 Processor -- start 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8e170 0x5615ebf9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf99350 0x5615ec01a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ad4c0 con 0x5615ebf8ea90 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ae930 con 0x5615ebf99350 2024-06-06T08:05:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641adb90 con 0x5615ebf8e170 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f6d5d4740 10 client.0 _ll_register_callbacks cb 0x7f1f5801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55858/0 (socket says 172.21.15.87:55858) 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8e170 0x5615ebf9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f637fe640 1 -- 172.21.15.87:0/1679301273 learned_addr learned my addr 172.21.15.87:0/1679301273 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.759+0000 7f1f68d84640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf99350 0x5615ec01a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 -- 172.21.15.87:0/1679301273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8e170 msgr2=0x5615ebf9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8e170 0x5615ebf9c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 -- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf99350 msgr2=0x5615ec01a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf99350 0x5615ec01a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 -- 172.21.15.87:0/1679301273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ec01ade0 con 0x5615ebf8ea90 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f637fe640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009bc0 tx=0x7f1f58019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7764d1c91aaa91d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58027070 con 0x5615ebf8ea90 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f1f58005610 con 0x5615ebf8ea90 2024-06-06T08:05:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5801f040 con 0x5615ebf8ea90 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f58048050 con 0x5615ebf8ea90 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f6d5d4740 10 client.17928 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5615ec01c060 con 0x5615ebf8ea90 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.760+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f5801f040 con 0x5615ebf8ea90 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f1f58003680 con 0x5615ebf8ea90 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 10 client.17928 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 10 client.17928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 20 client.17928 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ec01c730 0x5615ec01eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 4 client.17928 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615ebeffc20 con 0x5615ec01c730 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f63fff640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ec01c730 0x5615ec01eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f437fe640 10 client.17928 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.761+0000 7f1f63fff640 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ec01c730 0x5615ec01eb50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ad5443763a447e server_cookie=7798f53057b07471 in_seq=0 out_seq=0 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.971+0000 7f1f437fe640 1 -- 172.21.15.87:0/1679301273 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5615ebeffc20 con 0x5615ec01c730 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.971+0000 7f1f437fe640 10 client.17928 handle_command_reply: tid=0 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.971+0000 7f1f6d5d4740 1 client.17928 shutdown 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 msgr2=0x5615ebf9ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009bc0 tx=0x7f1f58019c30 comp rx=0 tx=0).stop 2024-06-06T08:05:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 shutdown_connections 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5615ec01c730 0x5615ec01eb50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f437fe640 0 client.17928 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615ebf99350 0x5615ec01a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5615ebf8ea90 0x5615ebf9ce10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 --2- 172.21.15.87:0/1679301273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615ebf8e170 0x5615ebf9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 >> 172.21.15.87:0/1679301273 conn(0x5615ebf8a680 msgr2=0x5615ebf8fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 shutdown_connections 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 1 -- 172.21.15.87:0/1679301273 wait complete. 2024-06-06T08:05:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6d5d4740 20 client.17928 trim_cache size 0 max 16384 2024-06-06T08:05:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:05:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:05:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 msgr2=0x7f1f4c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 0x7f1f4c0a5260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009500 comp rx=0 tx=0).stop 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.972+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 msgr2=0x7f1f641981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009230 tx=0x7f1f54004040 comp rx=0 tx=0).stop 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 shutdown_connections 2024-06-06T08:05:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1f4c0a2da0 0x7f1f4c0a5260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f64106890 0x7f1f6419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f64101290 0x7f1f64198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 --2- 172.21.15.87:0/340536179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f641008d0 0x7f1f641981c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 >> 172.21.15.87:0/340536179 conn(0x7f1f64077a20 msgr2=0x7f1f640ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 shutdown_connections 2024-06-06T08:05:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:24.973+0000 7f1f6a7e7640 1 -- 172.21.15.87:0/340536179 wait complete. 2024-06-06T08:05:25.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 49520 14.61 MB/sec execute 87 sec latency 122.356 ms 2024-06-06T08:05:26.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 49900 14.65 MB/sec execute 88 sec latency 204.364 ms 2024-06-06T08:05:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:26 smithi087 ceph-mon[32276]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 600 KiB/s wr, 15 op/s 2024-06-06T08:05:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:26 smithi195 ceph-mon[37040]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 600 KiB/s wr, 15 op/s 2024-06-06T08:05:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:26 smithi181 ceph-mon[36790]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 600 KiB/s wr, 15 op/s 2024-06-06T08:05:27.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 50454 14.86 MB/sec execute 89 sec latency 155.753 ms 2024-06-06T08:05:28.167 INFO:tasks.workunit.client.0.smithi087.stdout: 10 50998 15.07 MB/sec execute 90 sec latency 197.117 ms 2024-06-06T08:05:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:28 smithi087 ceph-mon[32276]: pgmap v553: 129 pgs: 129 active+clean; 411 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-06-06T08:05:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:28 smithi195 ceph-mon[37040]: pgmap v553: 129 pgs: 129 active+clean; 411 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-06-06T08:05:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:28 smithi181 ceph-mon[36790]: pgmap v553: 129 pgs: 129 active+clean; 411 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-06-06T08:05:29.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 51318 15.13 MB/sec execute 91 sec latency 214.900 ms 2024-06-06T08:05:30.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 51668 15.19 MB/sec execute 92 sec latency 217.904 ms 2024-06-06T08:05:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:30 smithi087 ceph-mon[32276]: pgmap v554: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-06-06T08:05:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:30 smithi195 ceph-mon[37040]: pgmap v554: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-06-06T08:05:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:30 smithi181 ceph-mon[36790]: pgmap v554: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-06-06T08:05:31.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 52042 15.21 MB/sec execute 93 sec latency 162.561 ms 2024-06-06T08:05:32.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 52341 15.26 MB/sec execute 94 sec latency 1162.613 ms 2024-06-06T08:05:32.357 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:32 smithi181 ceph-mon[36790]: pgmap v555: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-06-06T08:05:32.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:32 smithi087 ceph-mon[32276]: pgmap v555: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-06-06T08:05:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:32 smithi195 ceph-mon[37040]: pgmap v555: 129 pgs: 129 active+clean; 412 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-06-06T08:05:33.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 52590 15.18 MB/sec execute 95 sec latency 2162.658 ms 2024-06-06T08:05:34.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 53056 15.24 MB/sec execute 96 sec latency 3162.694 ms 2024-06-06T08:05:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:34 smithi087 ceph-mon[32276]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-06-06T08:05:34.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:34 smithi195 ceph-mon[37040]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-06-06T08:05:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:34 smithi181 ceph-mon[36790]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2024-06-06T08:05:35.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 53520 15.43 MB/sec execute 97 sec latency 4162.748 ms 2024-06-06T08:05:36.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 53791 15.38 MB/sec execute 98 sec latency 4929.462 ms 2024-06-06T08:05:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:36 smithi087 ceph-mon[32276]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-06-06T08:05:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:36 smithi195 ceph-mon[37040]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-06-06T08:05:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:36 smithi181 ceph-mon[36790]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-06-06T08:05:37.168 INFO:tasks.workunit.client.0.smithi087.stdout: 10 54036 15.31 MB/sec execute 99 sec latency 4487.871 ms 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 -- 172.21.15.87:0/2572901949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04006a1b0 msgr2=0x7fe04006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 --2- 172.21.15.87:0/2572901949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04006a1b0 0x7fe04006a590 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe030009900 tx=0x7fe03002a410 comp rx=0 tx=0).stop 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 -- 172.21.15.87:0/2572901949 shutdown_connections 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 --2- 172.21.15.87:0/2572901949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04010e6c0 0x7fe040110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 --2- 172.21.15.87:0/2572901949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006aad0 0x7fe04010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 --2- 172.21.15.87:0/2572901949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04006a1b0 0x7fe04006a590 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 -- 172.21.15.87:0/2572901949 >> 172.21.15.87:0/2572901949 conn(0x7fe040077810 msgr2=0x7fe040077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 -- 172.21.15.87:0/2572901949 shutdown_connections 2024-06-06T08:05:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 -- 172.21.15.87:0/2572901949 wait complete. 2024-06-06T08:05:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.072+0000 7fe045770640 1 Processor -- start 2024-06-06T08:05:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 -- start start 2024-06-06T08:05:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006a1b0 0x7fe04019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04006aad0 0x7fe04019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe040113f80 con 0x7fe04010e6c0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe040113e00 con 0x7fe04006aad0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe045770640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe040114100 con 0x7fe04006a1b0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34398/0 (socket says 172.21.15.87:34398) 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 -- 172.21.15.87:0/3948611277 learned_addr learned my addr 172.21.15.87:0/3948611277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04006aad0 0x7fe04019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03effd640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006a1b0 0x7fe04019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 -- 172.21.15.87:0/3948611277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006a1b0 msgr2=0x7fe04019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006a1b0 0x7fe04019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 -- 172.21.15.87:0/3948611277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04006aad0 msgr2=0x7fe04019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04006aad0 0x7fe04019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 -- 172.21.15.87:0/3948611277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019f2c0 con 0x7fe04010e6c0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.073+0000 7fe03f7fe640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe03800eb40 tx=0x7fe03800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.074+0000 7fe027fff640 1 -- 172.21.15.87:0/3948611277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038004b20 con 0x7fe04010e6c0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.074+0000 7fe027fff640 1 -- 172.21.15.87:0/3948611277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe038004cc0 con 0x7fe04010e6c0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.074+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019f5b0 con 0x7fe04010e6c0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.074+0000 7fe027fff640 1 -- 172.21.15.87:0/3948611277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038010850 con 0x7fe04010e6c0 2024-06-06T08:05:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.074+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a6e40 con 0x7fe04010e6c0 2024-06-06T08:05:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.076+0000 7fe027fff640 1 -- 172.21.15.87:0/3948611277 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe038005020 con 0x7fe04010e6c0 2024-06-06T08:05:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 Processor -- start 2024-06-06T08:05:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 -- start start 2024-06-06T08:05:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 0x5561e2506550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2510e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 0x5561e2513740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5561e246a4f0 con 0x5561e2506170 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561e2408870 con 0x5561e2506a90 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe048481740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561e24664b0 con 0x5561e2511350 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 0x5561e2506550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2510e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 0x5561e2506550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34402/0 (socket says 172.21.15.87:34402) 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03e7fc640 1 -- 172.21.15.87:0/4286204285 learned_addr learned my addr 172.21.15.87:0/4286204285 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe027fff640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 0x7fe0280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 0x5561e2513740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03f7fe640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 0x7fe0280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 -- 172.21.15.87:0/4286204285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 msgr2=0x5561e2513740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 0x5561e2513740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 -- 172.21.15.87:0/4286204285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 msgr2=0x5561e2506550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 0x5561e2506550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03effd640 1 -- 172.21.15.87:0/4286204285 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e24249b0 con 0x5561e2506a90 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.077+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 0x5561e2513740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe027fff640 1 -- 172.21.15.87:0/3948611277 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fe0380cff50 con 0x7fe04010e6c0 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe03effd640 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2510e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe03002aa00 tx=0x7fe030005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53579b136baf2a39 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe03f7fe640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 0x7fe0280a52e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0380cd380 tx=0x7fe038005a20 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe0257fa640 1 -- 172.21.15.87:0/4286204285 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030035040 con 0x5561e2506a90 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe0257fa640 1 -- 172.21.15.87:0/4286204285 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe030004400 con 0x5561e2506a90 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.078+0000 7fe0257fa640 1 -- 172.21.15.87:0/4286204285 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300035f0 con 0x5561e2506a90 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- 172.21.15.87:0/4286204285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 msgr2=0x5561e2510e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2510e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe03002aa00 tx=0x7fe030005de0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- 172.21.15.87:0/4286204285 shutdown_connections 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2511350 0x5561e2513740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2510e10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- 172.21.15.87:0/4286204285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2506170 0x5561e2506550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- 172.21.15.87:0/4286204285 >> 172.21.15.87:0/4286204285 conn(0x5561e2502680 msgr2=0x5561e2502a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- 172.21.15.87:0/4286204285 shutdown_connections 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- 172.21.15.87:0/4286204285 wait complete. 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- start start 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 Processor -- start 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e250e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5561e2513f80 con 0x5561e2511350 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5561e23f3ae0 con 0x5561e2506170 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561e2514100 con 0x5561e2506a90 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe048481740 10 client.0 _ll_register_callbacks cb 0x5561e242c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34406/0 (socket says 172.21.15.87:34406) 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.079+0000 7fe03e7fc640 1 -- 172.21.15.87:0/1490124214 learned_addr learned my addr 172.21.15.87:0/1490124214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03effd640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e250e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 -- 172.21.15.87:0/1490124214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 msgr2=0x5561e250e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e250e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 -- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 msgr2=0x5561e250e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 -- 172.21.15.87:0/1490124214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e217e210 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe034010f40 tx=0x7fe034015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4b2a8c1a82ff6c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034004050 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fe0340041f0 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe030004400 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe034027050 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe048481740 10 client.17940 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.080+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561e224a620 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe030004400 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fe034022410 con 0x5561e2511350 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 10 client.17940 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 20 client.17940 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e2523a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 4 client.17940 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561e247df10 con 0x5561e2521580 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.081+0000 7fe03effd640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e2523a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.082+0000 7fe03effd640 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e2523a00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cccb351b2f041e7 server_cookie=cd6eaf2705a2f97c in_seq=0 out_seq=0 2024-06-06T08:05:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.082+0000 7fe022ff5640 10 client.17940 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe022ff5640 1 -- 172.21.15.87:0/1490124214 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561e247df10 con 0x5561e2521580 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe022ff5640 10 client.17940 handle_command_reply: tid=0 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 client.17940 shutdown 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 msgr2=0x5561e250ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe034010f40 tx=0x7fe034015040 comp rx=0 tx=0).stop 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 shutdown_connections 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e2523a00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e250ed60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e250e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe022ff5640 0 client.17940 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 --2- 172.21.15.87:0/1490124214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.085+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 >> 172.21.15.87:0/1490124214 conn(0x5561e2502680 msgr2=0x5561e2508400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.086+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 shutdown_connections 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.086+0000 7fe048481740 1 -- 172.21.15.87:0/1490124214 wait complete. 2024-06-06T08:05:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.086+0000 7fe048481740 20 client.17940 trim_cache size 0 max 16384 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 Processor -- start 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 -- start start 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e25292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b3860 con 0x5561e2511350 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b10d0 con 0x5561e2506170 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe048481740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401aef20 con 0x5561e2506a90 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e25292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34412/0 (socket says 172.21.15.87:34412) 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 -- 172.21.15.87:0/111806670 learned_addr learned my addr 172.21.15.87:0/111806670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 -- 172.21.15.87:0/111806670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 msgr2=0x5561e25292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e25292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 -- 172.21.15.87:0/111806670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 msgr2=0x5561e250d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.103+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.104+0000 7fe03e7fc640 1 -- 172.21.15.87:0/111806670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e251d1e0 con 0x5561e2511350 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.104+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe034005d30 tx=0x7fe034001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e081c3b74dd835 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.104+0000 7fe024ff9640 1 -- 172.21.15.87:0/111806670 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034010700 con 0x5561e2511350 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.104+0000 7fe024ff9640 1 -- 172.21.15.87:0/111806670 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe034010c60 con 0x5561e2511350 2024-06-06T08:05:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.104+0000 7fe024ff9640 1 -- 172.21.15.87:0/111806670 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034003de0 con 0x5561e2511350 2024-06-06T08:05:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- 172.21.15.87:0/111806670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 msgr2=0x5561e2529810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe034005d30 tx=0x7fe034001e70 comp rx=0 tx=0).stop 2024-06-06T08:05:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- 172.21.15.87:0/111806670 shutdown_connections 2024-06-06T08:05:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2529810 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506a90 0x5561e25292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- 172.21.15.87:0/111806670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506170 0x5561e250d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- 172.21.15.87:0/111806670 >> 172.21.15.87:0/111806670 conn(0x5561e2502680 msgr2=0x5561e2502a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- 172.21.15.87:0/111806670 shutdown_connections 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- 172.21.15.87:0/111806670 wait complete. 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- start start 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 Processor -- start 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506170 0x5561e25198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2514c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401af590 con 0x5561e2511350 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401b0a00 con 0x5561e2506a90 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401afc60 con 0x5561e2506170 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe048481740 10 client.0 _ll_register_callbacks cb 0x7fe034009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe03e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34426/0 (socket says 172.21.15.87:34426) 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe03e7fc640 1 -- 172.21.15.87:0/709591409 learned_addr learned my addr 172.21.15.87:0/709591409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506170 0x5561e25198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.105+0000 7fe03effd640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2514c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 -- 172.21.15.87:0/709591409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506170 msgr2=0x5561e25198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506170 0x5561e25198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 -- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 msgr2=0x5561e2514c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2514c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 -- 172.21.15.87:0/709591409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e2515760 con 0x5561e2511350 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe03e7fc640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe034022b20 tx=0x7fe034010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7b9f43e83e94cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034015070 con 0x5561e2511350 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe034010c60 con 0x5561e2511350 2024-06-06T08:05:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7fe034004030 con 0x5561e2511350 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe034010700 con 0x5561e2511350 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe048481740 10 client.17946 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561e258c0b0 con 0x5561e2511350 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.106+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe034010c60 con 0x5561e2511350 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fe03403b460 con 0x5561e2511350 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 10 client.17946 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 10 client.17946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 20 client.17946 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e258e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 4 client.17946 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561e2477c20 con 0x5561e2521580 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe03effd640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e258e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe022ff5640 10 client.17946 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:38.107+0000 7fe03effd640 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e258e820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4923983f5b3390d4 server_cookie=1431d954d89fa7f0 in_seq=0 out_seq=0 2024-06-06T08:05:38.171 INFO:tasks.workunit.client.0.smithi087.stdout: 10 54386 15.38 MB/sec execute 100 sec latency 5491.487 ms 2024-06-06T08:05:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:38 smithi087 ceph-mon[32276]: pgmap v558: 129 pgs: 129 active+clean; 433 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-06-06T08:05:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:38 smithi195 ceph-mon[37040]: pgmap v558: 129 pgs: 129 active+clean; 433 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-06-06T08:05:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:38 smithi181 ceph-mon[36790]: pgmap v558: 129 pgs: 129 active+clean; 433 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-06-06T08:05:39.171 INFO:tasks.workunit.client.0.smithi087.stdout: 10 54550 15.36 MB/sec execute 101 sec latency 6491.526 ms 2024-06-06T08:05:39.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:39 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:05:39.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:39 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:05:39.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:39 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:05:40.171 INFO:tasks.workunit.client.0.smithi087.stdout: 10 54860 15.33 MB/sec execute 102 sec latency 7491.553 ms 2024-06-06T08:05:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:40 smithi087 ceph-mon[32276]: pgmap v559: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-06-06T08:05:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:05:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:05:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:05:40.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:40 smithi195 ceph-mon[37040]: pgmap v559: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-06-06T08:05:40.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:05:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:05:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:05:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:40 smithi181 ceph-mon[36790]: pgmap v559: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-06-06T08:05:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:05:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:05:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.903+0000 7fe022ff5640 1 -- 172.21.15.87:0/709591409 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561e2477c20 con 0x5561e2521580 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.903+0000 7fe022ff5640 10 client.17946 handle_command_reply: tid=0 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.903+0000 7fe048481740 1 client.17946 shutdown 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 msgr2=0x5561e2515170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe034022b20 tx=0x7fe034010530 comp rx=0 tx=0).stop 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 shutdown_connections 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561e2521580 0x5561e258e820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5561e2511350 0x5561e2515170 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561e2506a90 0x5561e2514c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 --2- 172.21.15.87:0/709591409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5561e2506170 0x5561e25198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 >> 172.21.15.87:0/709591409 conn(0x5561e2502680 msgr2=0x5561e2508400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe022ff5640 0 client.17946 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 shutdown_connections 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 1 -- 172.21.15.87:0/709591409 wait complete. 2024-06-06T08:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.904+0000 7fe048481740 20 client.17946 trim_cache size 0 max 16384 2024-06-06T08:05:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:05:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:05:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:05:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 msgr2=0x7fe0280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 0x7fe0280a52e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0380cd380 tx=0x7fe038005a20 comp rx=0 tx=0).stop 2024-06-06T08:05:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 msgr2=0x7fe04019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe03800eb40 tx=0x7fe03800c2d0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 shutdown_connections 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe0280a2e20 0x7fe0280a52e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe04010e6c0 0x7fe04019eb90 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe04006aad0 0x7fe04019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 --2- 172.21.15.87:0/3948611277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe04006a1b0 0x7fe04019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 >> 172.21.15.87:0/3948611277 conn(0x7fe040077810 msgr2=0x7fe04010f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 shutdown_connections 2024-06-06T08:05:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:40.905+0000 7fe045770640 1 -- 172.21.15.87:0/3948611277 wait complete. 2024-06-06T08:05:41.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 55111 15.28 MB/sec execute 103 sec latency 8331.495 ms 2024-06-06T08:05:42.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 55463 15.27 MB/sec execute 104 sec latency 334.596 ms 2024-06-06T08:05:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:42 smithi087 ceph-mon[32276]: pgmap v560: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-06-06T08:05:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:42 smithi195 ceph-mon[37040]: pgmap v560: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-06-06T08:05:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:42 smithi181 ceph-mon[36790]: pgmap v560: 129 pgs: 129 active+clean; 451 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-06-06T08:05:43.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 55828 15.32 MB/sec execute 105 sec latency 311.299 ms 2024-06-06T08:05:44.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 56202 15.44 MB/sec execute 106 sec latency 202.879 ms 2024-06-06T08:05:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:44 smithi087 ceph-mon[32276]: pgmap v561: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-06-06T08:05:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:44 smithi195 ceph-mon[37040]: pgmap v561: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-06-06T08:05:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:44 smithi181 ceph-mon[36790]: pgmap v561: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-06-06T08:05:45.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 56615 15.48 MB/sec execute 107 sec latency 224.779 ms 2024-06-06T08:05:46.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 56926 15.55 MB/sec execute 108 sec latency 264.746 ms 2024-06-06T08:05:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:46 smithi087 ceph-mon[32276]: pgmap v562: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-06-06T08:05:46.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:46 smithi195 ceph-mon[37040]: pgmap v562: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-06-06T08:05:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:46 smithi181 ceph-mon[36790]: pgmap v562: 129 pgs: 129 active+clean; 433 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-06-06T08:05:47.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 57274 15.58 MB/sec execute 109 sec latency 226.288 ms 2024-06-06T08:05:48.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 57625 15.59 MB/sec execute 110 sec latency 187.796 ms 2024-06-06T08:05:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:48 smithi087 ceph-mon[32276]: pgmap v563: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-06-06T08:05:48.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:48 smithi195 ceph-mon[37040]: pgmap v563: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-06-06T08:05:48.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:48 smithi181 ceph-mon[36790]: pgmap v563: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-06-06T08:05:49.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 57893 15.56 MB/sec execute 111 sec latency 832.041 ms 2024-06-06T08:05:50.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 58365 15.64 MB/sec execute 112 sec latency 930.798 ms 2024-06-06T08:05:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:50 smithi087 ceph-mon[32276]: pgmap v564: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-06-06T08:05:50.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:50 smithi195 ceph-mon[37040]: pgmap v564: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-06-06T08:05:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:50 smithi181 ceph-mon[36790]: pgmap v564: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-06-06T08:05:51.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 58774 15.73 MB/sec execute 113 sec latency 213.508 ms 2024-06-06T08:05:52.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 59149 15.80 MB/sec execute 114 sec latency 188.754 ms 2024-06-06T08:05:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:52 smithi087 ceph-mon[32276]: pgmap v565: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2024-06-06T08:05:52.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:52 smithi195 ceph-mon[37040]: pgmap v565: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2024-06-06T08:05:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:52 smithi181 ceph-mon[36790]: pgmap v565: 129 pgs: 129 active+clean; 444 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 426 op/s 2024-06-06T08:05:53.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 59414 15.75 MB/sec execute 115 sec latency 181.067 ms 2024-06-06T08:05:53.724 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-06-06T08:05:53.725 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:05:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 -- 172.21.15.87:0/3435807071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c4100900 msgr2=0x7f41c4100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 --2- 172.21.15.87:0/3435807071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c4100900 0x7f41c4100d80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009930 tx=0x7f41b802a420 comp rx=0 tx=0).stop 2024-06-06T08:05:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 -- 172.21.15.87:0/3435807071 shutdown_connections 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 --2- 172.21.15.87:0/3435807071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c41012c0 0x7f41c410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 --2- 172.21.15.87:0/3435807071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c4100900 0x7f41c4100d80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 --2- 172.21.15.87:0/3435807071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41068c0 0x7f41c4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 -- 172.21.15.87:0/3435807071 >> 172.21.15.87:0/3435807071 conn(0x7f41c40779f0 msgr2=0x7f41c4077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 -- 172.21.15.87:0/3435807071 shutdown_connections 2024-06-06T08:05:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.038+0000 7f41ca29e640 1 -- 172.21.15.87:0/3435807071 wait complete. 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.039+0000 7f41ca29e640 1 Processor -- start 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.039+0000 7f41ca29e640 1 -- start start 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c4100900 0x7f41c4195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41012c0 0x7f41c4196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410fbc0 con 0x7f41c41068c0 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410fa40 con 0x7f41c41012c0 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41ca29e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410fd40 con 0x7f41c4100900 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41012c0 0x7f41c4196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43602/0 (socket says 172.21.15.87:43602) 2024-06-06T08:05:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 -- 172.21.15.87:0/224755436 learned_addr learned my addr 172.21.15.87:0/224755436 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c3fff640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c4100900 0x7f41c4195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 -- 172.21.15.87:0/224755436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c4100900 msgr2=0x7f41c4195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c4100900 0x7f41c4195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 -- 172.21.15.87:0/224755436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41012c0 msgr2=0x7f41c4196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41012c0 0x7f41c4196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 -- 172.21.15.87:0/224755436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c41aab10 con 0x7f41c41068c0 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c883b640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00eb10 tx=0x7f41bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.040+0000 7f41c17fa640 1 -- 172.21.15.87:0/224755436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc004b20 con 0x7f41c41068c0 2024-06-06T08:05:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.041+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c41aae00 con 0x7f41c41068c0 2024-06-06T08:05:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.041+0000 7f41c17fa640 1 -- 172.21.15.87:0/224755436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc004cc0 con 0x7f41c41068c0 2024-06-06T08:05:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.041+0000 7f41c17fa640 1 -- 172.21.15.87:0/224755436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc010720 con 0x7f41c41068c0 2024-06-06T08:05:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.041+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41ab2d0 con 0x7f41c41068c0 2024-06-06T08:05:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.043+0000 7f41c17fa640 1 -- 172.21.15.87:0/224755436 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f41bc005020 con 0x7f41c41068c0 2024-06-06T08:05:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.043+0000 7f41c17fa640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 0x7f41ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.043+0000 7f41c3fff640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 0x7f41ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.043+0000 7f41c17fa640 1 -- 172.21.15.87:0/224755436 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f41bc0cfd30 con 0x7f41c41068c0 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41c3fff640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 0x7f41ac0a5260 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f41b400dac0 tx=0x7f41b400f040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 Processor -- start 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 -- start start 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fc4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fd1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bc66f284f0 con 0x55bc66fc4a90 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bc66ec6870 con 0x55bc66fcf350 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41ccfae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bc66f244b0 con 0x55bc66fc4170 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.044+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43618/0 (socket says 172.21.15.87:43618) 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fc4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 -- 172.21.15.87:0/3555796108 learned_addr learned my addr 172.21.15.87:0/3555796108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c883b640 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fd1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 -- 172.21.15.87:0/3555796108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 msgr2=0x55bc66fc4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fc4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 -- 172.21.15.87:0/3555796108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 msgr2=0x55bc66fd1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fd1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 -- 172.21.15.87:0/3555796108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc66ee29b0 con 0x55bc66fc4a90 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f41b802a940 tx=0x7f41b8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4697ac8605581d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41aa7fc640 1 -- 172.21.15.87:0/3555796108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8040070 con 0x55bc66fc4a90 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.045+0000 7f41aa7fc640 1 -- 172.21.15.87:0/3555796108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b80317b0 con 0x55bc66fc4a90 2024-06-06T08:05:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41aa7fc640 1 -- 172.21.15.87:0/3555796108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8035040 con 0x55bc66fc4a90 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 -- 172.21.15.87:0/3555796108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 msgr2=0x55bc66fcee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f41b802a940 tx=0x7f41b8005bd0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 -- 172.21.15.87:0/3555796108 shutdown_connections 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fd1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcee10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3555796108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fc4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 -- 172.21.15.87:0/3555796108 >> 172.21.15.87:0/3555796108 conn(0x55bc66fc0680 msgr2=0x55bc66fc0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 -- 172.21.15.87:0/3555796108 shutdown_connections 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.046+0000 7f41ccfae740 1 -- 172.21.15.87:0/3555796108 wait complete. 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 -- start start 2024-06-06T08:05:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 Processor -- start 2024-06-06T08:05:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fcc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bc66fd1f20 con 0x55bc66fc4a90 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bc66eb1ae0 con 0x55bc66fc4170 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bc66fd20a0 con 0x55bc66fcf350 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41ccfae740 10 client.0 _ll_register_callbacks cb 0x55bc66eea9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43622/0 (socket says 172.21.15.87:43622) 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41c883b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fcc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.047+0000 7f41c37fe640 1 -- 172.21.15.87:0/3293223350 learned_addr learned my addr 172.21.15.87:0/3293223350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 -- 172.21.15.87:0/3293223350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 msgr2=0x55bc66fccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 -- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 msgr2=0x55bc66fcc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fcc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 -- 172.21.15.87:0/3293223350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc66c3c210 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c883b640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c3fff640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fcc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41c37fe640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f41b80057c0 tx=0x7f41b8009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825564405efe0bbb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8040030 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b80317b0 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f41b8004250 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f41b8056050 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41ccfae740 10 client.17958 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.048+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55bc66d08620 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b80317b0 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f41b8003710 con 0x55bc66fc4a90 2024-06-06T08:05:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 10 client.17958 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 10 client.17958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 20 client.17958 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc66fdf2f0 0x55bc66fe16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 4 client.17958 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc66f3bf10 con 0x55bc66fdf2f0 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41c3fff640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc66fdf2f0 0x55bc66fe16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41a37fe640 10 client.17958 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.049+0000 7f41c3fff640 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc66fdf2f0 0x55bc66fe16e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b708d0186b0b7985 server_cookie=57a78901118a7dcc in_seq=0 out_seq=0 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41a37fe640 1 -- 172.21.15.87:0/3293223350 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc66f3bf10 con 0x55bc66fdf2f0 2024-06-06T08:05:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41a37fe640 10 client.17958 handle_command_reply: tid=0 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 client.17958 shutdown 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 msgr2=0x55bc66fcc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f41b80057c0 tx=0x7f41b8009610 comp rx=0 tx=0).stop 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 shutdown_connections 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc66fdf2f0 0x55bc66fe16e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fcc7d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 --2- 172.21.15.87:0/3293223350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fcc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41a37fe640 0 client.17958 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 >> 172.21.15.87:0/3293223350 conn(0x55bc66fc0680 msgr2=0x55bc66fc63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 shutdown_connections 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 1 -- 172.21.15.87:0/3293223350 wait complete. 2024-06-06T08:05:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.051+0000 7f41ccfae740 20 client.17958 trim_cache size 0 max 16384 2024-06-06T08:05:54.062 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.071+0000 7f41ccfae740 1 Processor -- start 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.071+0000 7f41ccfae740 1 -- start start 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fe72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fcc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b7e40 con 0x55bc66fc4a90 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b56b0 con 0x55bc66fcf350 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41ccfae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b3500 con 0x55bc66fc4170 2024-06-06T08:05:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fe72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c883b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fcc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:38062/0 (socket says 172.21.15.87:38062) 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 -- 172.21.15.87:0/1355875207 learned_addr learned my addr 172.21.15.87:0/1355875207 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fe72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43636/0 (socket says 172.21.15.87:43636) 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 -- 172.21.15.87:0/1355875207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 msgr2=0x55bc66fcc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fcc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 -- 172.21.15.87:0/1355875207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 msgr2=0x55bc66fe72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fe72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 -- 172.21.15.87:0/1355875207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc66fc6aa0 con 0x55bc66fc4170 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41c3fff640 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55bc66fe1ce0 tx=0x7f41b4015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d9960c12ae2e9e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.072+0000 7f41a9ffb640 1 -- 172.21.15.87:0/1355875207 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4037070 con 0x55bc66fc4170 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.073+0000 7f41a9ffb640 1 -- 172.21.15.87:0/1355875207 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4009da0 con 0x55bc66fc4170 2024-06-06T08:05:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.073+0000 7f41a9ffb640 1 -- 172.21.15.87:0/1355875207 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4032a20 con 0x55bc66fc4170 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- 172.21.15.87:0/1355875207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 msgr2=0x55bc66fe6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55bc66fe1ce0 tx=0x7f41b4015000 comp rx=0 tx=0).stop 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- 172.21.15.87:0/1355875207 shutdown_connections 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fcf350 0x55bc66fcc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fe72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 --2- 172.21.15.87:0/1355875207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fc4170 0x55bc66fe6d70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- 172.21.15.87:0/1355875207 >> 172.21.15.87:0/1355875207 conn(0x55bc66fc0680 msgr2=0x55bc66fcab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- 172.21.15.87:0/1355875207 shutdown_connections 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- 172.21.15.87:0/1355875207 wait complete. 2024-06-06T08:05:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 -- start start 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.074+0000 7f41ccfae740 1 Processor -- start 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fc8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fc9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b3b70 con 0x55bc66fc4a90 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b4fe0 con 0x55bc66fc4170 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41b4240 con 0x55bc66fcf350 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41ccfae740 10 client.0 _ll_register_callbacks cb 0x7f41bc005800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fc8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c883b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fc9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fc8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43646/0 (socket says 172.21.15.87:43646) 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c37fe640 1 -- 172.21.15.87:0/708312752 learned_addr learned my addr 172.21.15.87:0/708312752 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 -- 172.21.15.87:0/708312752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 msgr2=0x55bc66fc9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fc9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 -- 172.21.15.87:0/708312752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 msgr2=0x55bc66fc8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fc8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 -- 172.21.15.87:0/708312752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc66fc9c40 con 0x55bc66fc4170 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c883b640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fc9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41c3fff640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fd79d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004410 tx=0x7f41b40324f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c93f9947b7d8164 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.075+0000 7f41a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:05:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4037070 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b4009da0 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f41b4032b70 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f41b4043050 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 10 client.26942 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55bc67049c40 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4009da0 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f41b4028b30 con 0x55bc66fc4170 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 10 client.26942 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 10 client.26942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 20 client.26942 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc6704a370 0x55bc6704c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 4 client.26942 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:05:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc66f35c20 con 0x55bc6704a370 2024-06-06T08:05:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.076+0000 7f41c37fe640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc6704a370 0x55bc6704c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.077+0000 7f41c37fe640 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc6704a370 0x55bc6704c790 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5d1fc2d02cb78f server_cookie=fa4ac5151c7acb97 in_seq=0 out_seq=0 2024-06-06T08:05:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.077+0000 7f41a37fe640 10 client.26942 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:54.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 59732 15.71 MB/sec execute 116 sec latency 231.251 ms 2024-06-06T08:05:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:54 smithi087 ceph-mon[32276]: pgmap v566: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 545 op/s 2024-06-06T08:05:54.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:54 smithi195 ceph-mon[37040]: pgmap v566: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 545 op/s 2024-06-06T08:05:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:54 smithi181 ceph-mon[36790]: pgmap v566: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 545 op/s 2024-06-06T08:05:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.920+0000 7f41a37fe640 1 -- 172.21.15.87:0/708312752 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc66f35c20 con 0x55bc6704a370 2024-06-06T08:05:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.920+0000 7f41a37fe640 10 client.26942 handle_command_reply: tid=0 2024-06-06T08:05:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.920+0000 7f41ccfae740 1 client.26942 shutdown 2024-06-06T08:05:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 msgr2=0x55bc66fd79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fd79d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004410 tx=0x7f41b40324f0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 shutdown_connections 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bc6704a370 0x55bc6704c790 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc66fcf350 0x55bc66fc9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc66fc4a90 0x55bc66fc8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 --2- 172.21.15.87:0/708312752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc66fc4170 0x55bc66fd79d0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004410 tx=0x7f41b40324f0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 >> 172.21.15.87:0/708312752 conn(0x55bc66fc0680 msgr2=0x55bc66fc5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41a37fe640 0 client.26942 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 shutdown_connections 2024-06-06T08:05:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 1 -- 172.21.15.87:0/708312752 wait complete. 2024-06-06T08:05:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.921+0000 7f41ccfae740 20 client.26942 trim_cache size 0 max 16384 2024-06-06T08:05:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:05:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:05:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 msgr2=0x7f41ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 0x7f41ac0a5260 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f41b400dac0 tx=0x7f41b400f040 comp rx=0 tx=0).stop 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 msgr2=0x7f41c4196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00eb10 tx=0x7f41bc00c2d0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 shutdown_connections 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f41ac0a2da0 0x7f41ac0a5260 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41c41068c0 0x7f41c4196940 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41c41012c0 0x7f41c4196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 --2- 172.21.15.87:0/224755436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c4100900 0x7f41c4195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 >> 172.21.15.87:0/224755436 conn(0x7f41c40779f0 msgr2=0x7f41c41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 shutdown_connections 2024-06-06T08:05:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:05:54.922+0000 7f41ca29e640 1 -- 172.21.15.87:0/224755436 wait complete. 2024-06-06T08:05:55.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 59885 15.63 MB/sec execute 117 sec latency 592.197 ms 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 -- 172.21.15.87:0/330331994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8101390 msgr2=0x7f3ca8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 --2- 172.21.15.87:0/330331994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8101390 0x7f3ca8101810 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009990 tx=0x7f3c98009ca0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 -- 172.21.15.87:0/330331994 shutdown_connections 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 --2- 172.21.15.87:0/330331994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 --2- 172.21.15.87:0/330331994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8107400 0x7f3ca81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 --2- 172.21.15.87:0/330331994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8101390 0x7f3ca8101810 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 -- 172.21.15.87:0/330331994 >> 172.21.15.87:0/330331994 conn(0x7f3ca8078340 msgr2=0x7f3ca8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 -- 172.21.15.87:0/330331994 shutdown_connections 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.826+0000 7f3caf602700 1 -- 172.21.15.87:0/330331994 wait complete. 2024-06-06T08:05:55.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 Processor -- start 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 -- start start 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 0x7f3ca819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111690 con 0x7f3ca8107400 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111510 con 0x7f3ca8101d50 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3caf602700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111810 con 0x7f3ca8101390 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cadb9f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 0x7f3ca819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cacb9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:38088/0 (socket says 172.21.15.87:38088) 2024-06-06T08:05:55.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 -- 172.21.15.87:0/384879052 learned_addr learned my addr 172.21.15.87:0/384879052 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:55.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 -- 172.21.15.87:0/384879052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 msgr2=0x7f3ca819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:55.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 -- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 msgr2=0x7f3ca819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 0x7f3ca819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cad39e700 1 -- 172.21.15.87:0/384879052 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca819fde0 con 0x7f3ca8101390 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cadb9f700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 0x7f3ca819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.827+0000 7f3cacb9d700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3cad39e700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400e9c0 tx=0x7f3ca400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400cbd0 con 0x7f3ca8101390 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca81a00d0 con 0x7f3ca8101390 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca400ee60 con 0x7f3ca8101390 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4005630 con 0x7f3ca8101390 2024-06-06T08:05:55.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.828+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a7950 con 0x7f3ca8101390 2024-06-06T08:05:55.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.829+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca8102ab0 con 0x7f3ca8101390 2024-06-06T08:05:55.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.831+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3ca4004750 con 0x7f3ca8101390 2024-06-06T08:05:55.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.831+0000 7f3c9e7fc700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 0x7f3c940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:55.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.831+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f3ca40cfc90 con 0x7f3ca8101390 2024-06-06T08:05:55.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.831+0000 7f3cacb9d700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 0x7f3c940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:55.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.832+0000 7f3cacb9d700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 0x7f3c940a5250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b2e0 tx=0x7f3c98005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:55.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:55.837+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ca4053850 con 0x7f3ca8101390 2024-06-06T08:05:56.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.135+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ca8069180 con 0x7f3ca8101390 2024-06-06T08:05:56.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.136+0000 7f3c9e7fc700 1 -- 172.21.15.87:0/384879052 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11881 (secure 0 0 0) 0x7f3ca409c480 con 0x7f3ca8101390 2024-06-06T08:05:56.136 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:05:56.137 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:04:59.022587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 msgr2=0x7f3c940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 0x7f3c940a5250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b2e0 tx=0x7f3c98005960 comp rx=0 tx=0).stop 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 msgr2=0x7f3ca819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400e9c0 tx=0x7f3ca400ecd0 comp rx=0 tx=0).stop 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 shutdown_connections 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c940a2d90 0x7f3c940a5250 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca8107400 0x7f3ca819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca8101390 0x7f3ca819add0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:56.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 --2- 172.21.15.87:0/384879052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca8101d50 0x7f3ca819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 >> 172.21.15.87:0/384879052 conn(0x7f3ca8078340 msgr2=0x7f3ca806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 shutdown_connections 2024-06-06T08:05:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:56.139+0000 7f3caf602700 1 -- 172.21.15.87:0/384879052 wait complete. 2024-06-06T08:05:56.140 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 52 2024-06-06T08:05:56.172 INFO:tasks.workunit.client.0.smithi087.stdout: 10 60278 15.69 MB/sec execute 118 sec latency 210.498 ms 2024-06-06T08:05:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:56 smithi087 ceph-mon[32276]: pgmap v567: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-06-06T08:05:56.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:56 smithi195 ceph-mon[37040]: pgmap v567: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-06-06T08:05:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:56 smithi181 ceph-mon[36790]: pgmap v567: 129 pgs: 129 active+clean; 459 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-06-06T08:05:56.792 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-06-06T08:05:56.792 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-06-06T08:05:57.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 60605 15.73 MB/sec execute 119 sec latency 263.901 ms 2024-06-06T08:05:57.202 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:05:57.403 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/384879052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:57.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/384879052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:57.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/384879052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:05:58.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 60843 15.72 MB/sec execute 120 sec latency 422.258 ms 2024-06-06T08:05:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:05:58 smithi087 ceph-mon[32276]: pgmap v568: 129 pgs: 129 active+clean; 454 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-06-06T08:05:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:05:58 smithi195 ceph-mon[37040]: pgmap v568: 129 pgs: 129 active+clean; 454 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-06-06T08:05:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:05:58 smithi181 ceph-mon[36790]: pgmap v568: 129 pgs: 129 active+clean; 454 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-06-06T08:05:59.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61264 15.76 MB/sec execute 121 sec latency 258.124 ms 2024-06-06T08:05:59.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.748+0000 7f18039a6700 1 -- 172.21.15.87:0/2434478097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc1016b0 msgr2=0x7f17fc101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.748+0000 7f18039a6700 1 --2- 172.21.15.87:0/2434478097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc1016b0 0x7f17fc101b30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009990 tx=0x7f17ec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 -- 172.21.15.87:0/2434478097 shutdown_connections 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 --2- 172.21.15.87:0/2434478097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc102070 0x7f17fc10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 --2- 172.21.15.87:0/2434478097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc107500 0x7f17fc1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 --2- 172.21.15.87:0/2434478097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc1016b0 0x7f17fc101b30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 -- 172.21.15.87:0/2434478097 >> 172.21.15.87:0/2434478097 conn(0x7f17fc0780f0 msgr2=0x7f17fc078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 -- 172.21.15.87:0/2434478097 shutdown_connections 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 -- 172.21.15.87:0/2434478097 wait complete. 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 Processor -- start 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 -- start start 2024-06-06T08:05:59.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.749+0000 7f18039a6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 0x7f17fc1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 0x7f17fc1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc107500 0x7f17fc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1116e0 con 0x7f17fc102070 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111560 con 0x7f17fc107500 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111860 con 0x7f17fc1016b0 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1800f41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 0x7f17fc1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 0x7f17fc1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1800f41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 0x7f17fc1a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39570/0 (socket says 172.21.15.87:39570) 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1800f41700 1 -- 172.21.15.87:0/617651459 learned_addr learned my addr 172.21.15.87:0/617651459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801f43700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc107500 0x7f17fc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 -- 172.21.15.87:0/617651459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc107500 msgr2=0x7f17fc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc107500 0x7f17fc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 -- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 msgr2=0x7f17fc1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 0x7f17fc1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 -- 172.21.15.87:0/617651459 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19b260 con 0x7f17fc1016b0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f1801742700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 0x7f17fc1a0940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d950 tx=0x7f17f800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f17f27fc700 1 -- 172.21.15.87:0/617651459 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8016070 con 0x7f17fc1016b0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f17f27fc700 1 -- 172.21.15.87:0/617651459 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f800fd70 con 0x7f17fc1016b0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19b550 con 0x7f17fc1016b0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.750+0000 7f17f27fc700 1 -- 172.21.15.87:0/617651459 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800e3b0 con 0x7f17fc1016b0 2024-06-06T08:05:59.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.751+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a7810 con 0x7f17fc1016b0 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.753+0000 7f17f27fc700 1 -- 172.21.15.87:0/617651459 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f17f8019070 con 0x7f17fc1016b0 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.753+0000 7f17f27fc700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 0x7f17e80a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.753+0000 7f17f27fc700 1 -- 172.21.15.87:0/617651459 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f17f80cf1f0 con 0x7f17fc1016b0 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.753+0000 7f1800f41700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 0x7f17e80a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.754+0000 7f1800f41700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 0x7f17e80a52f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17fc078a80 tx=0x7f17ec009590 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 Processor -- start 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 -- start start 2024-06-06T08:05:59.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e2e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e29100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561364d1b0f0 con 0x561364e2eeb0 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1800f41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561364d799e0 con 0x561364e28ce0 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f180e229b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561364ce15f0 con 0x561364e2e590 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1800f41700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39578/0 (socket says 172.21.15.87:39578) 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1800f41700 1 -- 172.21.15.87:0/2867418703 learned_addr learned my addr 172.21.15.87:0/2867418703 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e2e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801f43700 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e29100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 -- 172.21.15.87:0/2867418703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 msgr2=0x561364e29100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e29100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 -- 172.21.15.87:0/2867418703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 msgr2=0x561364e287a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e287a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.755+0000 7f1801742700 1 -- 172.21.15.87:0/2867418703 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561364cc8a60 con 0x561364e2e590 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.756+0000 7f1801742700 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e2e970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17f80cc570 tx=0x7f17f809ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50f60699cc801029 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.756+0000 7f17e77fe700 1 -- 172.21.15.87:0/2867418703 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8020050 con 0x561364e2e590 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.756+0000 7f17e77fe700 1 -- 172.21.15.87:0/2867418703 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f809b340 con 0x561364e2e590 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.756+0000 7f17e77fe700 1 -- 172.21.15.87:0/2867418703 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80a3b20 con 0x561364e2e590 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- 172.21.15.87:0/2867418703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 msgr2=0x561364e2e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e2e970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17f80cc570 tx=0x7f17f809ad70 comp rx=0 tx=0).stop 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- 172.21.15.87:0/2867418703 shutdown_connections 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e29100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e2e970 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 --2- 172.21.15.87:0/2867418703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e287a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- 172.21.15.87:0/2867418703 >> 172.21.15.87:0/2867418703 conn(0x561364d55210 msgr2=0x561364e23a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- 172.21.15.87:0/2867418703 shutdown_connections 2024-06-06T08:05:59.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- 172.21.15.87:0/2867418703 wait complete. 2024-06-06T08:05:59.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 -- start start 2024-06-06T08:05:59.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.757+0000 7f180e229b80 1 Processor -- start 2024-06-06T08:05:59.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e34c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561364d1a150 con 0x561364e28ce0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561364d759a0 con 0x561364e2e590 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561364e35890 con 0x561364e2eeb0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f180e229b80 10 client.0 _ll_register_callbacks cb 0x561364d0f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1801f43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1801742700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e34c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53900/0 (socket says 172.21.15.87:53900) 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1801f43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:41360/0 (socket says 172.21.15.87:41360) 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 -- 172.21.15.87:0/809643320 learned_addr learned my addr 172.21.15.87:0/809643320 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:59.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 -- 172.21.15.87:0/809643320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 msgr2=0x561364e324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 -- 172.21.15.87:0/809643320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 msgr2=0x561364e34c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e34c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1800f41700 1 -- 172.21.15.87:0/809643320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561364d3f8e0 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.758+0000 7f1801f43700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f1800f41700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0057a0 tx=0x7f17ec01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec12ad804c326b49 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec034060 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f809b340 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f17ec00f660 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f17ec03b050 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f180e229b80 10 client.26960 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561364e400b0 con 0x561364e2e590 2024-06-06T08:05:59.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f809b340 con 0x561364e2e590 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.759+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f17ec02f860 con 0x561364e2e590 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 10 client.26960 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 10 client.26960 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 20 client.26960 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364e42690 0x561364e44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 4 client.26960 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561364d86ee0 con 0x561364e42690 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f1801742700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364e42690 0x561364e44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f17e4ff9700 10 client.26960 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.760+0000 7f1801742700 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364e42690 0x561364e44a80 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3829377b55aae4 server_cookie=e93b749457db3027 in_seq=0 out_seq=0 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f17e4ff9700 1 -- 172.21.15.87:0/809643320 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561364d86ee0 con 0x561364e42690 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f17e4ff9700 10 client.26960 handle_command_reply: tid=0 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 client.26960 shutdown 2024-06-06T08:05:59.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 msgr2=0x561364e31f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0057a0 tx=0x7f17ec01f000 comp rx=0 tx=0).stop 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 shutdown_connections 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364e42690 0x561364e44a80 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e34c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 --2- 172.21.15.87:0/809643320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e31f90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f17e4ff9700 0 client.26960 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 >> 172.21.15.87:0/809643320 conn(0x561364d55210 msgr2=0x561364e223b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 shutdown_connections 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 1 -- 172.21.15.87:0/809643320 wait complete. 2024-06-06T08:05:59.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.762+0000 7f180e229b80 20 client.26960 trim_cache size 0 max 16384 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.784+0000 7f180e229b80 1 Processor -- start 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.784+0000 7f180e229b80 1 -- start start 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.784+0000 7f180e229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.784+0000 7f180e229b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f180e229b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f180e229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b3080 con 0x561364e28ce0 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f180e229b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b08b0 con 0x561364e2e590 2024-06-06T08:05:59.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f180e229b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ae700 con 0x561364e2eeb0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39594/0 (socket says 172.21.15.87:39594) 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 -- 172.21.15.87:0/200106571 learned_addr learned my addr 172.21.15.87:0/200106571 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1800f41700 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801f43700 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 -- 172.21.15.87:0/200106571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 msgr2=0x561364e347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 -- 172.21.15.87:0/200106571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 msgr2=0x561364e22c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e22c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 -- 172.21.15.87:0/200106571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561364e23310 con 0x561364e28ce0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f1801742700 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f17f80ccb70 tx=0x7f17f80975e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0da31ad26819bc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f17e6ffd700 1 -- 172.21.15.87:0/200106571 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8020070 con 0x561364e28ce0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f17e6ffd700 1 -- 172.21.15.87:0/200106571 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f809b340 con 0x561364e28ce0 2024-06-06T08:05:59.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.785+0000 7f17e6ffd700 1 -- 172.21.15.87:0/200106571 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80a3b20 con 0x561364e28ce0 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 -- 172.21.15.87:0/200106571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 msgr2=0x561364e287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f17f80ccb70 tx=0x7f17f80975e0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 -- 172.21.15.87:0/200106571 shutdown_connections 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2eeb0 0x561364e347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e28ce0 0x561364e287a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 --2- 172.21.15.87:0/200106571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e2e590 0x561364e22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 -- 172.21.15.87:0/200106571 >> 172.21.15.87:0/200106571 conn(0x561364d55210 msgr2=0x561364e2d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 -- 172.21.15.87:0/200106571 shutdown_connections 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.786+0000 7f180e229b80 1 -- 172.21.15.87:0/200106571 wait complete. 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 -- start start 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 Processor -- start 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e3b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e37af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1aed70 con 0x561364e2eeb0 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b01e0 con 0x561364e28ce0 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1af440 con 0x561364e2e590 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f180e229b80 10 client.0 _ll_register_callbacks cb 0x561364d0f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:05:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.787+0000 7f1801742700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:53920/0 (socket says 172.21.15.87:53920) 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 -- 172.21.15.87:0/782211598 learned_addr learned my addr 172.21.15.87:0/782211598 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801f43700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e37af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1800f41700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e3b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 -- 172.21.15.87:0/782211598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 msgr2=0x561364e3b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e3b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 -- 172.21.15.87:0/782211598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 msgr2=0x561364e37af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e37af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 -- 172.21.15.87:0/782211598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561364e38130 con 0x561364e28ce0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801f43700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e37af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f1801742700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17f80a3a80 tx=0x7f17f80a40b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3399ddfaa8774b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f17e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8020070 con 0x561364e28ce0 2024-06-06T08:05:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f809b340 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.788+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9495+0+0 (secure 0 0 0) 0x7f17f8010660 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f17f809b5a0 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 10 client.26966 fetch_fsmap learned FSMap version 52 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561364eadbc0 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f809b340 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f17f809a790 con 0x561364e28ce0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 10 client.26966 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 10 client.26966 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 20 client.26966 populate_metadata read hostname 'smithi087' 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364eaddf0 0x561364eb01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 4 client.26966 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x561364d81460 con 0x561364eaddf0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.789+0000 7f1800f41700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364eaddf0 0x561364eb01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:05:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.790+0000 7f17e4ff9700 10 client.26966 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:05:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:05:59.790+0000 7f1800f41700 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364eaddf0 0x561364eb01e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7527aa27a98b2655 server_cookie=c42e9d81901625a8 in_seq=0 out_seq=0 2024-06-06T08:06:00.174 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61528 15.78 MB/sec execute 122 sec latency 244.416 ms 2024-06-06T08:06:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:00 smithi195 ceph-mon[37040]: pgmap v569: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.7 MiB/s wr, 498 op/s 2024-06-06T08:06:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:00 smithi181 ceph-mon[36790]: pgmap v569: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.7 MiB/s wr, 498 op/s 2024-06-06T08:06:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:00 smithi087 ceph-mon[32276]: pgmap v569: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.7 MiB/s wr, 498 op/s 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f17e4ff9700 1 -- 172.21.15.87:0/782211598 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561364d81460 con 0x561364eaddf0 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f17e4ff9700 10 client.26966 handle_command_reply: tid=0 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 client.26966 shutdown 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 msgr2=0x561364e3ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17f80a3a80 tx=0x7f17f80a40b0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 shutdown_connections 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x561364eaddf0 0x561364eb01e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561364e28ce0 0x561364e3ab20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561364e2eeb0 0x561364e37af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 --2- 172.21.15.87:0/782211598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561364e2e590 0x561364e3b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f17e4ff9700 0 client.26966 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 >> 172.21.15.87:0/782211598 conn(0x561364d55210 msgr2=0x561364e22ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 shutdown_connections 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 1 -- 172.21.15.87:0/782211598 wait complete. 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.705+0000 7f180e229b80 20 client.26966 trim_cache size 0 max 16384 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 52, 2024-06-06T08:06:00.707 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 54, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "4a954962": { 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 54, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 130.0, 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:06:00.708 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 msgr2=0x7f17e80a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 0x7f17e80a52f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17fc078a80 tx=0x7f17ec009590 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 msgr2=0x7f17fc1a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 0x7f17fc1a0940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d950 tx=0x7f17f800dc60 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 shutdown_connections 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17fc107500 0x7f17fc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f17e80a2e30 0x7f17e80a52f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17fc1016b0 0x7f17fc1a0940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 --2- 172.21.15.87:0/617651459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17fc102070 0x7f17fc1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 >> 172.21.15.87:0/617651459 conn(0x7f17fc0780f0 msgr2=0x7f17fc06ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 shutdown_connections 2024-06-06T08:06:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:00.706+0000 7f18039a6700 1 -- 172.21.15.87:0/617651459 wait complete. 2024-06-06T08:06:01.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.65 MB/sec execute 123 sec latency 1244.452 ms 2024-06-06T08:06:01.559 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4a954962', quiesce took 7.8 seconds. Will release after: 2.2 2024-06-06T08:06:02.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.53 MB/sec execute 124 sec latency 2244.497 ms 2024-06-06T08:06:02.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:02 smithi195 ceph-mon[37040]: pgmap v570: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-06-06T08:06:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:02 smithi195 ceph-mon[37040]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T08:06:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:02 smithi181 ceph-mon[36790]: pgmap v570: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-06-06T08:06:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:02 smithi181 ceph-mon[36790]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T08:06:02.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:02 smithi087 ceph-mon[32276]: pgmap v570: 129 pgs: 129 active+clean; 448 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-06-06T08:06:02.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:02 smithi087 ceph-mon[32276]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T08:06:03.173 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.40 MB/sec execute 125 sec latency 3244.541 ms 2024-06-06T08:06:03.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:06:04.175 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.28 MB/sec execute 126 sec latency 4245.208 ms 2024-06-06T08:06:04.217 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:04.403 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:04 smithi087 ceph-mon[32276]: pgmap v572: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:04 smithi195 ceph-mon[37040]: pgmap v572: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:04 smithi181 ceph-mon[36790]: pgmap v572: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:05.174 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.16 MB/sec execute 127 sec latency 5245.258 ms 2024-06-06T08:06:06.174 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 15.04 MB/sec execute 128 sec latency 6245.296 ms 2024-06-06T08:06:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:06 smithi195 ceph-mon[37040]: pgmap v573: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:06 smithi181 ceph-mon[36790]: pgmap v573: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:06 smithi087 ceph-mon[32276]: pgmap v573: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-06-06T08:06:07.174 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 14.92 MB/sec execute 129 sec latency 7245.374 ms 2024-06-06T08:06:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- 172.21.15.87:0/2773977265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3270102030 msgr2=0x7f3270105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 --2- 172.21.15.87:0/2773977265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3270102030 0x7f3270105ff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f32600099c0 tx=0x7f3260009cd0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- 172.21.15.87:0/2773977265 shutdown_connections 2024-06-06T08:06:07.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 --2- 172.21.15.87:0/2773977265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32701066d0 0x7f327010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 --2- 172.21.15.87:0/2773977265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 --2- 172.21.15.87:0/2773977265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3270102030 0x7f3270105ff0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- 172.21.15.87:0/2773977265 >> 172.21.15.87:0/2773977265 conn(0x7f32700773e0 msgr2=0x7f32700777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- 172.21.15.87:0/2773977265 shutdown_connections 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- 172.21.15.87:0/2773977265 wait complete. 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 Processor -- start 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 -- start start 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.596+0000 7f32781a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 0x7f3270199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32701116b0 con 0x7f32701066d0 2024-06-06T08:06:07.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3270111530 con 0x7f3270102030 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3270111830 con 0x7f3270101680 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275f44700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275f44700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:46104/0 (socket says 172.21.15.87:46104) 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32832/0 (socket says 172.21.15.87:32832) 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275f44700 1 -- 172.21.15.87:0/127991708 learned_addr learned my addr 172.21.15.87:0/127991708 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275743700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 0x7f3270199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 -- 172.21.15.87:0/127991708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 msgr2=0x7f3270199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 -- 172.21.15.87:0/127991708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 msgr2=0x7f3270199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 0x7f3270199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 -- 172.21.15.87:0/127991708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701a4fd0 con 0x7f32701066d0 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275f44700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:07.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3275743700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 0x7f3270199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3276745700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f326800ab40 tx=0x7f326800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268017020 con 0x7f32701066d0 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701a5260 con 0x7f32701066d0 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3268004bb0 con 0x7f32701066d0 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.597+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268005260 con 0x7f32701066d0 2024-06-06T08:06:07.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.598+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701a5720 con 0x7f32701066d0 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.598+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3270102ad0 con 0x7f32701066d0 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.600+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f326800c580 con 0x7f32701066d0 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.600+0000 7f3266ffd700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 0x7f325c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.600+0000 7f3275f44700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 0x7f325c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.600+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9671+0+0 (secure 0 0 0) 0x7f32680cf3e0 con 0x7f32701066d0 2024-06-06T08:06:07.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.601+0000 7f3275f44700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 0x7f325c0a5250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f326c005950 tx=0x7f326c00a3b0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:07.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.604+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3268021020 con 0x7f32701066d0 2024-06-06T08:06:07.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.848+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f327002dca0 con 0x7f32701066d0 2024-06-06T08:06:07.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.849+0000 7f3266ffd700 1 -- 172.21.15.87:0/127991708 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11881 (secure 0 0 0) 0x7f326809bbb0 con 0x7f32701066d0 2024-06-06T08:06:07.848 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:06:07.849 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:04:59.022587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 msgr2=0x7f325c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 0x7f325c0a5250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f326c005950 tx=0x7f326c00a3b0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 msgr2=0x7f32701a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f326800ab40 tx=0x7f326800ae50 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 shutdown_connections 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f325c0a2d90 0x7f325c0a5250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32701066d0 0x7f32701a4a90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3270101680 0x7f3270199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 --2- 172.21.15.87:0/127991708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270102030 0x7f3270199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 >> 172.21.15.87:0/127991708 conn(0x7f32700773e0 msgr2=0x7f32700ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 shutdown_connections 2024-06-06T08:06:07.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:07.851+0000 7f32781a8700 1 -- 172.21.15.87:0/127991708 wait complete. 2024-06-06T08:06:07.852 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 52 2024-06-06T08:06:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.065+0000 7fee73276640 1 -- 172.21.15.87:0/2296742551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c1008d0 msgr2=0x7fee6c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.065+0000 7fee73276640 1 --2- 172.21.15.87:0/2296742551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c1008d0 0x7fee6c100d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee68009890 tx=0x7fee6802a370 comp rx=0 tx=0).stop 2024-06-06T08:06:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- 172.21.15.87:0/2296742551 shutdown_connections 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- 172.21.15.87:0/2296742551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c101290 0x7fee6c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- 172.21.15.87:0/2296742551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c1008d0 0x7fee6c100d50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- 172.21.15.87:0/2296742551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c106890 0x7fee6c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- 172.21.15.87:0/2296742551 >> 172.21.15.87:0/2296742551 conn(0x7fee6c077a20 msgr2=0x7fee6c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- 172.21.15.87:0/2296742551 shutdown_connections 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- 172.21.15.87:0/2296742551 wait complete. 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 Processor -- start 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- start start 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c1008d0 0x7fee6c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 0x7fee6c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c10fb90 con 0x7fee6c106890 2024-06-06T08:06:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c10fa10 con 0x7fee6c1008d0 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.066+0000 7fee73276640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c10fd10 con 0x7fee6c101290 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71012640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c1008d0 0x7fee6c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32848/0 (socket says 172.21.15.87:32848) 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 -- 172.21.15.87:0/2009484180 learned_addr learned my addr 172.21.15.87:0/2009484180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee70811640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 0x7fee6c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 -- 172.21.15.87:0/2009484180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 msgr2=0x7fee6c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 0x7fee6c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 -- 172.21.15.87:0/2009484180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c1008d0 msgr2=0x7fee6c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c1008d0 0x7fee6c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 -- 172.21.15.87:0/2009484180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c19af60 con 0x7fee6c106890 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee70811640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 0x7fee6c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee71813640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fee6400beb0 tx=0x7fee64008600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee627fc640 1 -- 172.21.15.87:0/2009484180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64019020 con 0x7fee6c106890 2024-06-06T08:06:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee627fc640 1 -- 172.21.15.87:0/2009484180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee64005020 con 0x7fee6c106890 2024-06-06T08:06:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c19b250 con 0x7fee6c106890 2024-06-06T08:06:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee627fc640 1 -- 172.21.15.87:0/2009484180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64015770 con 0x7fee6c106890 2024-06-06T08:06:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.067+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1a2ae0 con 0x7fee6c106890 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.070+0000 7fee627fc640 1 -- 172.21.15.87:0/2009484180 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fee640051d0 con 0x7fee6c106890 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.070+0000 7fee75f87740 1 Processor -- start 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.070+0000 7fee75f87740 1 -- start start 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 0x55ea4d703550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d710740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d6674f0 con 0x55ea4d703a90 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d605870 con 0x55ea4d703170 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee75f87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d6634b0 con 0x55ea4d70e350 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32860/0 (socket says 172.21.15.87:32860) 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 -- 172.21.15.87:0/4219856138 learned_addr learned my addr 172.21.15.87:0/4219856138 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee70811640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d710740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee627fc640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 0x7fee580a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee70811640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 0x55ea4d703550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 -- 172.21.15.87:0/4219856138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 msgr2=0x55ea4d710740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d710740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71813640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 0x7fee580a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 -- 172.21.15.87:0/4219856138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 msgr2=0x55ea4d703550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 0x55ea4d703550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 -- 172.21.15.87:0/4219856138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4d6219b0 con 0x55ea4d703a90 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee70811640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 0x55ea4d703550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee627fc640 1 -- 172.21.15.87:0/2009484180 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9671+0+0 (secure 0 0 0) 0x7fee640cfc60 con 0x7fee6c106890 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71012640 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fee5c017630 tx=0x7fee5c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66327b55dfb61849 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee71813640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 0x7fee580a5260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fee640cd530 tx=0x7fee64005940 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.071+0000 7fee577fe640 1 -- 172.21.15.87:0/4219856138 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c021070 con 0x55ea4d703a90 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee577fe640 1 -- 172.21.15.87:0/4219856138 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c005500 con 0x55ea4d703a90 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee577fe640 1 -- 172.21.15.87:0/4219856138 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011330 con 0x55ea4d703a90 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 -- 172.21.15.87:0/4219856138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 msgr2=0x55ea4d70de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fee5c017630 tx=0x7fee5c005f90 comp rx=0 tx=0).stop 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 -- 172.21.15.87:0/4219856138 shutdown_connections 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d710740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703a90 0x55ea4d70de10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 --2- 172.21.15.87:0/4219856138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703170 0x55ea4d703550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 -- 172.21.15.87:0/4219856138 >> 172.21.15.87:0/4219856138 conn(0x55ea4d6ff680 msgr2=0x55ea4d6ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 -- 172.21.15.87:0/4219856138 shutdown_connections 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.072+0000 7fee75f87740 1 -- 172.21.15.87:0/4219856138 wait complete. 2024-06-06T08:06:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 -- start start 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 Processor -- start 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d70b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d70bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d710f80 con 0x55ea4d703170 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d5f0ae0 con 0x55ea4d703a90 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea4d711100 con 0x55ea4d70e350 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee75f87740 10 client.0 _ll_register_callbacks cb 0x55ea4d6299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32870/0 (socket says 172.21.15.87:32870) 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 -- 172.21.15.87:0/3608832196 learned_addr learned my addr 172.21.15.87:0/3608832196 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 -- 172.21.15.87:0/3608832196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 msgr2=0x55ea4d70bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d70bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 -- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 msgr2=0x55ea4d70b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee71012640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d70b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d70b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.073+0000 7fee70811640 1 -- 172.21.15.87:0/3608832196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea4d37b210 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee71012640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d70b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee70811640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fee68009250 tx=0x7fee68005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeb1d97b61e25f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68045020 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9671+0+0 (secure 0 0 0) 0x7fee68004040 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c005500 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fee68050050 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 10 client.18006 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ea4d447620 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c005500 con 0x55ea4d703170 2024-06-06T08:06:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fee68032440 con 0x55ea4d703170 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 10 client.18006 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 10 client.18006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 20 client.18006 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d71e480 0x55ea4d720890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 4 client.18006 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.074+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea4d67af10 con 0x55ea4d71e480 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.075+0000 7fee71012640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d71e480 0x55ea4d720890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.075+0000 7fee71012640 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d71e480 0x55ea4d720890 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ca3045d926d1bf server_cookie=fcbbdeb131470930 in_seq=0 out_seq=0 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.075+0000 7fee54ff9640 10 client.18006 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee54ff9640 1 -- 172.21.15.87:0/3608832196 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea4d67af10 con 0x55ea4d71e480 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee54ff9640 10 client.18006 handle_command_reply: tid=0 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 client.18006 shutdown 2024-06-06T08:06:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 msgr2=0x55ea4d70b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fee68009250 tx=0x7fee68005b00 comp rx=0 tx=0).stop 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 shutdown_connections 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d71e480 0x55ea4d720890 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee54ff9640 0 client.18006 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d70bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d70b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 --2- 172.21.15.87:0/3608832196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d70b250 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 >> 172.21.15.87:0/3608832196 conn(0x55ea4d6ff680 msgr2=0x55ea4d705400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 shutdown_connections 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 1 -- 172.21.15.87:0/3608832196 wait complete. 2024-06-06T08:06:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.076+0000 7fee75f87740 20 client.18006 trim_cache size 0 max 16384 2024-06-06T08:06:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 Processor -- start 2024-06-06T08:06:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 -- start start 2024-06-06T08:06:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d703170 0x55ea4d726150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d726690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1af890 con 0x55ea4d70e350 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1ad100 con 0x55ea4d703a90 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee75f87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1aaf70 con 0x55ea4d703170 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32884/0 (socket says 172.21.15.87:32884) 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 -- 172.21.15.87:0/4068903328 learned_addr learned my addr 172.21.15.87:0/4068903328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d703170 0x55ea4d726150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee71012640 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d726690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 -- 172.21.15.87:0/4068903328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d703170 msgr2=0x55ea4d726150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.091+0000 7fee70811640 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d703170 0x55ea4d726150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee70811640 1 -- 172.21.15.87:0/4068903328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 msgr2=0x55ea4d726690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee70811640 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d726690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee70811640 1 -- 172.21.15.87:0/4068903328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4d7038c0 con 0x55ea4d70e350 2024-06-06T08:06:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee70811640 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fee68030ca0 tx=0x7fee68005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22393fcd3034055 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee56ffd640 1 -- 172.21.15.87:0/4068903328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68045040 con 0x55ea4d70e350 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee56ffd640 1 -- 172.21.15.87:0/4068903328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee68032500 con 0x55ea4d70e350 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee56ffd640 1 -- 172.21.15.87:0/4068903328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68037030 con 0x55ea4d70e350 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 -- 172.21.15.87:0/4068903328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 msgr2=0x55ea4d704b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fee68030ca0 tx=0x7fee68005a90 comp rx=0 tx=0).stop 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 -- 172.21.15.87:0/4068903328 shutdown_connections 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d70e350 0x55ea4d704b20 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d726690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 --2- 172.21.15.87:0/4068903328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d703170 0x55ea4d726150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.092+0000 7fee75f87740 1 -- 172.21.15.87:0/4068903328 >> 172.21.15.87:0/4068903328 conn(0x55ea4d6ff680 msgr2=0x55ea4d706590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- 172.21.15.87:0/4068903328 shutdown_connections 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- 172.21.15.87:0/4068903328 wait complete. 2024-06-06T08:06:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- start start 2024-06-06T08:06:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 Processor -- start 2024-06-06T08:06:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d716c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d711c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1ab5c0 con 0x55ea4d703170 2024-06-06T08:06:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1aca30 con 0x55ea4d703a90 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1abc90 con 0x55ea4d70e350 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee75f87740 10 client.0 _ll_register_callbacks cb 0x7fee6400d870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee70811640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32892/0 (socket says 172.21.15.87:32892) 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee70811640 1 -- 172.21.15.87:0/1765713504 learned_addr learned my addr 172.21.15.87:0/1765713504 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.093+0000 7fee71012640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d716c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d711c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 -- 172.21.15.87:0/1765713504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 msgr2=0x55ea4d711c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d711c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 -- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 msgr2=0x55ea4d716c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d716c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 -- 172.21.15.87:0/1765713504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea4d712470 con 0x55ea4d703170 2024-06-06T08:06:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee70811640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fee68005b60 tx=0x7fee68005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c36f557477016fb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68037060 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9671+0+0 (secure 0 0 0) 0x7fee68004230 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee68032500 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fee68045070 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 10 client.18012 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ea4d788cf0 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee68032500 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7fee680047e0 con 0x55ea4d703170 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 10 client.18012 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.094+0000 7fee75f87740 20 client.18012 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d789360 0x55ea4d78b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee75f87740 4 client.18012 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:06:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea4d674c20 con 0x55ea4d789360 2024-06-06T08:06:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee71012640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d789360 0x55ea4d78b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee71012640 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d789360 0x55ea4d78b780 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee9d0bfb56c1d0e server_cookie=e748e279a6f962ff in_seq=0 out_seq=0 2024-06-06T08:06:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.095+0000 7fee54ff9640 10 client.18012 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:08.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 14.81 MB/sec execute 130 sec latency 8246.742 ms 2024-06-06T08:06:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee54ff9640 1 -- 172.21.15.87:0/1765713504 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea4d674c20 con 0x55ea4d789360 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee54ff9640 10 client.18012 handle_command_reply: tid=0 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 client.18012 shutdown 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 msgr2=0x55ea4d716700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fee68005b60 tx=0x7fee68005bd0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 shutdown_connections 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ea4d789360 0x55ea4d78b780 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee54ff9640 0 client.18012 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea4d70e350 0x55ea4d711c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea4d703a90 0x55ea4d716c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 --2- 172.21.15.87:0/1765713504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea4d703170 0x55ea4d716700 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 >> 172.21.15.87:0/1765713504 conn(0x55ea4d6ff680 msgr2=0x55ea4d704b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.507+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 shutdown_connections 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee75f87740 1 -- 172.21.15.87:0/1765713504 wait complete. 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee75f87740 20 client.18012 trim_cache size 0 max 16384 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:06:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:06:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 msgr2=0x7fee580a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 0x7fee580a5260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fee640cd530 tx=0x7fee64005940 comp rx=0 tx=0).stop 2024-06-06T08:06:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 msgr2=0x7fee6c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fee6400beb0 tx=0x7fee64008600 comp rx=0 tx=0).stop 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 shutdown_connections 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fee580a2da0 0x7fee580a5260 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee6c106890 0x7fee6c19a830 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee6c101290 0x7fee6c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 --2- 172.21.15.87:0/2009484180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c1008d0 0x7fee6c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.508+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 >> 172.21.15.87:0/2009484180 conn(0x7fee6c077a20 msgr2=0x7fee6c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.509+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 shutdown_connections 2024-06-06T08:06:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:08.509+0000 7fee73276640 1 -- 172.21.15.87:0/2009484180 wait complete. 2024-06-06T08:06:08.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:08 smithi195 ceph-mon[37040]: pgmap v574: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 479 op/s 2024-06-06T08:06:08.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:08 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/127991708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:08 smithi181 ceph-mon[36790]: pgmap v574: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 479 op/s 2024-06-06T08:06:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:08 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/127991708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:08 smithi087 ceph-mon[32276]: pgmap v574: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 455 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 479 op/s 2024-06-06T08:06:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:08 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/127991708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:08.814 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 4a954962 --release --await' 2024-06-06T08:06:08.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 4a954962 --release --await 2024-06-06T08:06:09.086 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:09.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 14.70 MB/sec execute 131 sec latency 9246.822 ms 2024-06-06T08:06:10.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61547 14.58 MB/sec execute 132 sec latency 10246.879 ms 2024-06-06T08:06:10.421 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:10 smithi087 ceph-mon[32276]: pgmap v575: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-06-06T08:06:10.421 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:10 smithi087 ceph-mon[32276]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T08:06:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:10 smithi195 ceph-mon[37040]: pgmap v575: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-06-06T08:06:10.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:10 smithi195 ceph-mon[37040]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T08:06:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:10 smithi181 ceph-mon[36790]: pgmap v575: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-06-06T08:06:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:10 smithi181 ceph-mon[36790]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T08:06:10.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 -- 172.21.15.87:0/2502170931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102530 msgr2=0x7f1d401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2502170931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102530 0x7f1d401029b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1d300099c0 tx=0x7f1d30009cd0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 -- 172.21.15.87:0/2502170931 shutdown_connections 2024-06-06T08:06:10.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2502170931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d401085d0 0x7f1d401089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2502170931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102ef0 0x7f1d4010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2502170931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102530 0x7f1d401029b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 -- 172.21.15.87:0/2502170931 >> 172.21.15.87:0/2502170931 conn(0x7f1d40079640 msgr2=0x7f1d40079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 -- 172.21.15.87:0/2502170931 shutdown_connections 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.825+0000 7f1d4752c700 1 -- 172.21.15.87:0/2502170931 wait complete. 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 Processor -- start 2024-06-06T08:06:10.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 -- start start 2024-06-06T08:06:10.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 0x7f1d40074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d40102ef0 0x7f1d40074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401128f0 con 0x7f1d401085d0 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40112770 con 0x7f1d40102ef0 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d4752c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40112a70 con 0x7f1d40102530 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d452c8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 0x7f1d40074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32908/0 (socket says 172.21.15.87:32908) 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 -- 172.21.15.87:0/2659626904 learned_addr learned my addr 172.21.15.87:0/2659626904 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d40102ef0 0x7f1d40074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 -- 172.21.15.87:0/2659626904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 msgr2=0x7f1d40074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 0x7f1d40074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 -- 172.21.15.87:0/2659626904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d40102ef0 msgr2=0x7f1d40074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d40102ef0 0x7f1d40074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d45ac9700 1 -- 172.21.15.87:0/2659626904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401a71d0 con 0x7f1d401085d0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.826+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 0x7f1d40074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007ee0 tx=0x7f1d3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d367fc700 1 -- 172.21.15.87:0/2659626904 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004bb0 con 0x7f1d401085d0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d367fc700 1 -- 172.21.15.87:0/2659626904 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d38004d50 con 0x7f1d401085d0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d367fc700 1 -- 172.21.15.87:0/2659626904 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38005960 con 0x7f1d401085d0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a74c0 con 0x7f1d401085d0 2024-06-06T08:06:10.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.827+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a79d0 con 0x7f1d401085d0 2024-06-06T08:06:10.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.829+0000 7f1d367fc700 1 -- 172.21.15.87:0/2659626904 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1d3801a020 con 0x7f1d401085d0 2024-06-06T08:06:10.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.830+0000 7f1d367fc700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 0x7f1d2c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.830+0000 7f1d367fc700 1 -- 172.21.15.87:0/2659626904 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f1d380cefb0 con 0x7f1d401085d0 2024-06-06T08:06:10.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.830+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 0x7f1d2c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.830+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 0x7f1d2c0a5250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009de0 tx=0x7f1d3c009190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.831+0000 7f1d51dafb80 1 Processor -- start 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.831+0000 7f1d51dafb80 1 -- start start 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb2898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb283710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55abeb176060 con 0x55abeb283c50 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55abeb1d4950 con 0x55abeb289e20 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55abeb13c560 con 0x55abeb289500 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32916/0 (socket says 172.21.15.87:32916) 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 -- 172.21.15.87:0/1257685601 learned_addr learned my addr 172.21.15.87:0/1257685601 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d452c8700 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb2898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb283710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 -- 172.21.15.87:0/1257685601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 msgr2=0x55abeb2898e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb2898e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 -- 172.21.15.87:0/1257685601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 msgr2=0x55abeb283710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb283710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 -- 172.21.15.87:0/1257685601 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abeb1239d0 con 0x55abeb283c50 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004b80 tx=0x7f1d3809ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2ddc442721be3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d2b7fe700 1 -- 172.21.15.87:0/1257685601 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3809b050 con 0x55abeb283c50 2024-06-06T08:06:10.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d2b7fe700 1 -- 172.21.15.87:0/1257685601 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3809f400 con 0x55abeb283c50 2024-06-06T08:06:10.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.832+0000 7f1d2b7fe700 1 -- 172.21.15.87:0/1257685601 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380976c0 con 0x55abeb283c50 2024-06-06T08:06:10.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 -- 172.21.15.87:0/1257685601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 msgr2=0x55abeb284070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004b80 tx=0x7f1d3809ac10 comp rx=0 tx=0).stop 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 -- 172.21.15.87:0/1257685601 shutdown_connections 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb284070 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb2898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/1257685601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb283710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 -- 172.21.15.87:0/1257685601 >> 172.21.15.87:0/1257685601 conn(0x55abeb1b0180 msgr2=0x55abeb27ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 -- 172.21.15.87:0/1257685601 shutdown_connections 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.833+0000 7f1d51dafb80 1 -- 172.21.15.87:0/1257685601 wait complete. 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 -- start start 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 Processor -- start 2024-06-06T08:06:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb28cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289e20 0x55abeb28d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55abeb1750c0 con 0x55abeb283c50 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55abeb1d0910 con 0x55abeb289500 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55abeb290800 con 0x55abeb289e20 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d51dafb80 10 client.0 _ll_register_callbacks cb 0x55abeb16a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d452c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d452c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32926/0 (socket says 172.21.15.87:32926) 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d452c8700 1 -- 172.21.15.87:0/2641463034 learned_addr learned my addr 172.21.15.87:0/2641463034 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289e20 0x55abeb28d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.834+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb28cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 -- 172.21.15.87:0/2641463034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289e20 msgr2=0x55abeb28d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289e20 0x55abeb28d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 -- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 msgr2=0x55abeb28cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb28cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 -- 172.21.15.87:0/2641463034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abeb19a850 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d452c8700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0158d0 tx=0x7f1d3c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34366d32719d20f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c025070 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f1d3c013450 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3809f400 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c031050 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d51dafb80 10 client.18024 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55abeb29b090 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3809f400 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.835+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f1d3c021840 con 0x55abeb283c50 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 10 client.18024 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 10 client.18024 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 20 client.18024 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb29d6c0 0x55abeb29fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 4 client.18024 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abeb1e1e50 con 0x55abeb29d6c0 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb29d6c0 0x55abeb29fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d28ff9700 10 client.18024 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.836+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb29d6c0 0x55abeb29fb40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46fccbe3e191ec8 server_cookie=bc80de07eee114dc in_seq=0 out_seq=0 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d28ff9700 1 -- 172.21.15.87:0/2641463034 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abeb1e1e50 con 0x55abeb29d6c0 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d28ff9700 10 client.18024 handle_command_reply: tid=0 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 client.18024 shutdown 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 msgr2=0x55abeb28fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0158d0 tx=0x7f1d3c004860 comp rx=0 tx=0).stop 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 shutdown_connections 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb29d6c0 0x55abeb29fb40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289e20 0x55abeb28d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb28fc20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d28ff9700 0 client.18024 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/2641463034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb28cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.837+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 >> 172.21.15.87:0/2641463034 conn(0x55abeb1b0180 msgr2=0x55abeb27dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.838+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 shutdown_connections 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.838+0000 7f1d51dafb80 1 -- 172.21.15.87:0/2641463034 wait complete. 2024-06-06T08:06:10.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.838+0000 7f1d51dafb80 20 client.18024 trim_cache size 0 max 16384 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 Processor -- start 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 -- start start 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb288a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb28f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b33a0 con 0x55abeb283c50 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b0bd0 con 0x55abeb289e20 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.857+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401aea40 con 0x55abeb289500 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32930/0 (socket says 172.21.15.87:32930) 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 -- 172.21.15.87:0/771028787 learned_addr learned my addr 172.21.15.87:0/771028787 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb288a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 -- 172.21.15.87:0/771028787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 msgr2=0x55abeb288a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb288a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 -- 172.21.15.87:0/771028787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 msgr2=0x55abeb28f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb28f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb28f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 -- 172.21.15.87:0/771028787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abeb2764a0 con 0x55abeb283c50 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb288a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d452c8700 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0097f0 tx=0x7f1d3c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a026dee5af4e904e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d2affd700 1 -- 172.21.15.87:0/771028787 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c025040 con 0x55abeb283c50 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d2affd700 1 -- 172.21.15.87:0/771028787 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c021e50 con 0x55abeb283c50 2024-06-06T08:06:10.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.858+0000 7f1d2affd700 1 -- 172.21.15.87:0/771028787 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c01b040 con 0x55abeb283c50 2024-06-06T08:06:10.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- 172.21.15.87:0/771028787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 msgr2=0x55abeb283710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0097f0 tx=0x7f1d3c00ea50 comp rx=0 tx=0).stop 2024-06-06T08:06:10.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- 172.21.15.87:0/771028787 shutdown_connections 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289e20 0x55abeb28f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb283c50 0x55abeb283710 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/771028787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb289500 0x55abeb288a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- 172.21.15.87:0/771028787 >> 172.21.15.87:0/771028787 conn(0x55abeb1b0180 msgr2=0x55abeb28d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- 172.21.15.87:0/771028787 shutdown_connections 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- 172.21.15.87:0/771028787 wait complete. 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 -- start start 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.859+0000 7f1d51dafb80 1 Processor -- start 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb295c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb292a60 0x55abeb292ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401af090 con 0x55abeb289e20 2024-06-06T08:06:10.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b0500 con 0x55abeb289500 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401af760 con 0x55abeb292a60 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 10 client.0 _ll_register_callbacks cb 0x55abeb16a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:32940/0 (socket says 172.21.15.87:32940) 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 -- 172.21.15.87:0/3246071876 learned_addr learned my addr 172.21.15.87:0/3246071876 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d45ac9700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb292a60 0x55abeb292ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d452c8700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb295c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 -- 172.21.15.87:0/3246071876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb292a60 msgr2=0x55abeb292ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb292a60 0x55abeb292ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 -- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 msgr2=0x55abeb295c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb295c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 -- 172.21.15.87:0/3246071876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abeb293580 con 0x55abeb289e20 2024-06-06T08:06:10.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d452c8700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb295c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d44ac7700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1d300099c0 tx=0x7f1d30003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4b54e6a63a22fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30033060 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f1d300050b0 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.860+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c021e50 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1d3003a050 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 10 client.18030 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55abeb309270 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c021e50 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f1d3000f460 con 0x55abeb289e20 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 10 client.18030 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 10 client.18030 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 20 client.18030 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:10.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb309a40 0x55abeb30be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:10.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 4 client.18030 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a954962", "release": true, "await": true}] 2024-06-06T08:06:10.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a954962", "release": true, "await": true}) -- 0x55abeb1dc3d0 con 0x55abeb309a40 2024-06-06T08:06:10.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.861+0000 7f1d452c8700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb309a40 0x55abeb30be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:10.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.862+0000 7f1d28ff9700 10 client.18030 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:10.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.862+0000 7f1d452c8700 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb309a40 0x55abeb30be60 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a54cf1c6291b37 server_cookie=a76f6d7aeb3e4f7e in_seq=0 out_seq=0 2024-06-06T08:06:10.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d28ff9700 1 -- 172.21.15.87:0/3246071876 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55abeb1dc3d0 con 0x55abeb309a40 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d28ff9700 10 client.18030 handle_command_reply: tid=0 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 client.18030 shutdown 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 msgr2=0x55abeb2961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1d300099c0 tx=0x7f1d30003a10 comp rx=0 tx=0).stop 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 shutdown_connections 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55abeb309a40 0x55abeb30be60 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55abeb289500 0x55abeb295c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abeb292a60 0x55abeb292ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 --2- 172.21.15.87:0/3246071876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55abeb289e20 0x55abeb2961b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 >> 172.21.15.87:0/3246071876 conn(0x55abeb1b0180 msgr2=0x55abeb282700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d28ff9700 0 client.18030 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 shutdown_connections 2024-06-06T08:06:10.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 1 -- 172.21.15.87:0/3246071876 wait complete. 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.941+0000 7f1d51dafb80 20 client.18030 trim_cache size 0 max 16384 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 52, 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 56, 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "4a954962": { 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 56, 2024-06-06T08:06:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 130.0, 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:10.945 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 msgr2=0x7f1d2c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 0x7f1d2c0a5250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009de0 tx=0x7f1d3c009190 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 msgr2=0x7f1d40075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007ee0 tx=0x7f1d3800d3b0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 shutdown_connections 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1d2c0a2d90 0x7f1d2c0a5250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d401085d0 0x7f1d40075210 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40102530 0x7f1d40074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 --2- 172.21.15.87:0/2659626904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d40102ef0 0x7f1d40074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 >> 172.21.15.87:0/2659626904 conn(0x7f1d40079640 msgr2=0x7f1d40100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:10.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 shutdown_connections 2024-06-06T08:06:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:10.942+0000 7f1d4752c700 1 -- 172.21.15.87:0/2659626904 wait complete. 2024-06-06T08:06:11.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61593 14.48 MB/sec execute 133 sec latency 11013.351 ms 2024-06-06T08:06:12.127 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4a954962', total seconds elapsed: 18.4 2024-06-06T08:06:12.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 61862 14.47 MB/sec execute 134 sec latency 253.640 ms 2024-06-06T08:06:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:12 smithi087 ceph-mon[32276]: pgmap v577: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-06T08:06:12.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:12 smithi195 ceph-mon[37040]: pgmap v577: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-06T08:06:12.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:12 smithi181 ceph-mon[36790]: pgmap v577: 129 pgs: 129 active+clean; 454 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-06T08:06:13.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 62079 14.46 MB/sec execute 135 sec latency 411.774 ms 2024-06-06T08:06:14.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 62298 14.45 MB/sec execute 136 sec latency 384.229 ms 2024-06-06T08:06:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:14 smithi195 ceph-mon[37040]: pgmap v578: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:14 smithi181 ceph-mon[36790]: pgmap v578: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:14 smithi087 ceph-mon[32276]: pgmap v578: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:15.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 62621 14.44 MB/sec execute 137 sec latency 169.208 ms 2024-06-06T08:06:16.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 63040 14.51 MB/sec execute 138 sec latency 747.176 ms 2024-06-06T08:06:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:16 smithi195 ceph-mon[37040]: pgmap v579: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:16.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:16 smithi181 ceph-mon[36790]: pgmap v579: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:16 smithi087 ceph-mon[32276]: pgmap v579: 129 pgs: 129 active+clean; 441 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-06-06T08:06:17.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 63487 14.63 MB/sec execute 139 sec latency 175.768 ms 2024-06-06T08:06:18.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 63715 14.66 MB/sec execute 140 sec latency 437.025 ms 2024-06-06T08:06:18.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:18 smithi195 ceph-mon[37040]: pgmap v580: 129 pgs: 129 active+clean; 447 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-06-06T08:06:18.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:18 smithi181 ceph-mon[36790]: pgmap v580: 129 pgs: 129 active+clean; 447 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-06-06T08:06:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:18 smithi087 ceph-mon[32276]: pgmap v580: 129 pgs: 129 active+clean; 447 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-06-06T08:06:19.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 63907 14.61 MB/sec execute 141 sec latency 312.352 ms 2024-06-06T08:06:20.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 64145 14.57 MB/sec execute 142 sec latency 388.001 ms 2024-06-06T08:06:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:20 smithi195 ceph-mon[37040]: pgmap v581: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-06-06T08:06:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:20 smithi181 ceph-mon[36790]: pgmap v581: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-06-06T08:06:20.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:20 smithi087 ceph-mon[32276]: pgmap v581: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-06-06T08:06:21.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 64445 14.52 MB/sec execute 143 sec latency 220.239 ms 2024-06-06T08:06:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- 172.21.15.87:0/4203883775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0101290 msgr2=0x7f27a010d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4203883775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0101290 0x7f27a010d750 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2798009cd0 tx=0x7f279802a8e0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- 172.21.15.87:0/4203883775 shutdown_connections 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4203883775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0101290 0x7f27a010d750 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4203883775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a01008d0 0x7f27a0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4203883775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a0106890 0x7f27a0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- 172.21.15.87:0/4203883775 >> 172.21.15.87:0/4203883775 conn(0x7f27a0077a20 msgr2=0x7f27a0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- 172.21.15.87:0/4203883775 shutdown_connections 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- 172.21.15.87:0/4203883775 wait complete. 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 Processor -- start 2024-06-06T08:06:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.604+0000 7f27a6c12640 1 -- start start 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a01008d0 0x7f27a0197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a0101290 0x7f27a01975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0110c60 con 0x7f27a0106890 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0110ae0 con 0x7f27a01008d0 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a6c12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0110de0 con 0x7f27a0101290 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a01008d0 0x7f27a0197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33068/0 (socket says 172.21.15.87:33068) 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 -- 172.21.15.87:0/4273729359 learned_addr learned my addr 172.21.15.87:0/4273729359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f279ffff640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a0101290 0x7f27a01975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 -- 172.21.15.87:0/4273729359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a0101290 msgr2=0x7f27a01975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a0101290 0x7f27a01975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 -- 172.21.15.87:0/4273729359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a01008d0 msgr2=0x7f27a0197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a01008d0 0x7f27a0197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 -- 172.21.15.87:0/4273729359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a019c050 con 0x7f27a0106890 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f27a51af640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2798031e70 tx=0x7f2798032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.605+0000 7f279dffb640 1 -- 172.21.15.87:0/4273729359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004040 con 0x7f27a0106890 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.606+0000 7f279dffb640 1 -- 172.21.15.87:0/4273729359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27980046c0 con 0x7f27a0106890 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.606+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a019c2e0 con 0x7f27a0106890 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.606+0000 7f279dffb640 1 -- 172.21.15.87:0/4273729359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798046480 con 0x7f27a0106890 2024-06-06T08:06:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.606+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a3bd0 con 0x7f27a0106890 2024-06-06T08:06:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.608+0000 7f279dffb640 1 -- 172.21.15.87:0/4273729359 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2798030630 con 0x7f27a0106890 2024-06-06T08:06:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 Processor -- start 2024-06-06T08:06:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 -- start start 2024-06-06T08:06:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f279dffb640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 0x7f27900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a192470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f26a0f64f0 con 0x55f26a19f560 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f26a094870 con 0x55f26a192090 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a99f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f26a0f24b0 con 0x55f26a1929b0 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f279ffff640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 0x7f27900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f279dffb640 1 -- 172.21.15.87:0/4273729359 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f27980f0c50 con 0x7f27a0106890 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a51af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a192470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a51af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33082/0 (socket says 172.21.15.87:33082) 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a51af640 1 -- 172.21.15.87:0/1505037724 learned_addr learned my addr 172.21.15.87:0/1505037724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a192470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:35604/0 (socket says 172.21.15.87:35604) 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f279ffff640 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a51af640 1 -- 172.21.15.87:0/1505037724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 msgr2=0x55f26a19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.609+0000 7f27a51af640 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a51af640 1 -- 172.21.15.87:0/1505037724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 msgr2=0x55f26a192470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a51af640 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a192470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a51af640 1 -- 172.21.15.87:0/1505037724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26a0b09b0 con 0x55f26a19f560 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f279ffff640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 0x7f27900a52e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27a0105c40 tx=0x7f278c0128d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a51af640 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f27980b97d0 tx=0x7f27980309d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee47b2919129a4f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f278affd640 1 -- 172.21.15.87:0/1505037724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980bd500 con 0x55f26a19f560 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f278affd640 1 -- 172.21.15.87:0/1505037724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798036040 con 0x55f26a19f560 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f278affd640 1 -- 172.21.15.87:0/1505037724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279804dda0 con 0x55f26a19f560 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a99f6740 1 -- 172.21.15.87:0/1505037724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 msgr2=0x55f26a1a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.610+0000 7f27a99f6740 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f27980b97d0 tx=0x7f27980309d0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- 172.21.15.87:0/1505037724 shutdown_connections 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a1a1950 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- 172.21.15.87:0/1505037724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a192470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- 172.21.15.87:0/1505037724 >> 172.21.15.87:0/1505037724 conn(0x55f26a18e680 msgr2=0x55f26a18ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- 172.21.15.87:0/1505037724 shutdown_connections 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- 172.21.15.87:0/1505037724 wait complete. 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- start start 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 Processor -- start 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a19f560 0x55f26a19d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f26a0f64f0 con 0x55f26a192090 2024-06-06T08:06:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f26a07fae0 con 0x55f26a19f560 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.611+0000 7f27a99f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f26a1a1f50 con 0x55f26a1929b0 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a99f6740 10 client.0 _ll_register_callbacks cb 0x55f26a0b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33092/0 (socket says 172.21.15.87:33092) 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 -- 172.21.15.87:0/2248503328 learned_addr learned my addr 172.21.15.87:0/2248503328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a51af640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a19f560 0x55f26a19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f279ffff640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 -- 172.21.15.87:0/2248503328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 msgr2=0x55f26a19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 -- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a19f560 msgr2=0x55f26a19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a19f560 0x55f26a19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 -- 172.21.15.87:0/2248503328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f269e0a210 con 0x55f26a192090 2024-06-06T08:06:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a49ae640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2794012760 tx=0x7f2794012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cb7358a0d7f1c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f277ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400f920 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f279400fac0 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2798036040 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f279400fcc0 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a99f6740 10 client.18042 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.612+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f269ed6620 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798036040 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f279401c420 con 0x55f26a192090 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 10 client.18042 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 20 client.18042 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a1af5a0 0x55f26a1b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 4 client.18042 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26a109f10 con 0x55f26a1af5a0 2024-06-06T08:06:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f279ffff640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a1af5a0 0x55f26a1b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f279ffff640 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a1af5a0 0x55f26a1b1a10 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1b00dc7b42eaff server_cookie=ef51fa2820b88f2e in_seq=0 out_seq=0 2024-06-06T08:06:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.613+0000 7f277ffff640 10 client.18042 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f277ffff640 1 -- 172.21.15.87:0/2248503328 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f26a109f10 con 0x55f26a1af5a0 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f277ffff640 10 client.18042 handle_command_reply: tid=0 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 client.18042 shutdown 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 msgr2=0x55f26a19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2794012760 tx=0x7f2794012b60 comp rx=0 tx=0).stop 2024-06-06T08:06:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 shutdown_connections 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a1af5a0 0x55f26a1b1a10 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a19f560 0x55f26a19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f277ffff640 0 client.18042 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 --2- 172.21.15.87:0/2248503328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a192090 0x55f26a19c5b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.615+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 >> 172.21.15.87:0/2248503328 conn(0x55f26a18e680 msgr2=0x55f26a1944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.616+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 shutdown_connections 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.616+0000 7f27a99f6740 1 -- 172.21.15.87:0/2248503328 wait complete. 2024-06-06T08:06:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.616+0000 7f27a99f6740 20 client.18042 trim_cache size 0 max 16384 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 Processor -- start 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 -- start start 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a19f560 0x55f26a193c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01af620 con 0x55f26a1929b0 2024-06-06T08:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ace90 con 0x55f26a192090 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a99f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aad00 con 0x55f26a19f560 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33094/0 (socket says 172.21.15.87:33094) 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 -- 172.21.15.87:0/67795987 learned_addr learned my addr 172.21.15.87:0/67795987 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a49ae640 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f27a51af640 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a19f560 0x55f26a193c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 -- 172.21.15.87:0/67795987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a19f560 msgr2=0x55f26a193c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a19f560 0x55f26a193c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 -- 172.21.15.87:0/67795987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 msgr2=0x55f26a1b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.633+0000 7f279ffff640 1 -- 172.21.15.87:0/67795987 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26a188ae0 con 0x55f26a1929b0 2024-06-06T08:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.634+0000 7f279ffff640 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f278c005d00 tx=0x7f278c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70909aa008ee4df5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.634+0000 7f278a7fc640 1 -- 172.21.15.87:0/67795987 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0220d0 con 0x55f26a1929b0 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.634+0000 7f278a7fc640 1 -- 172.21.15.87:0/67795987 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278c022270 con 0x55f26a1929b0 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.634+0000 7f278a7fc640 1 -- 172.21.15.87:0/67795987 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c034aa0 con 0x55f26a1929b0 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- 172.21.15.87:0/67795987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 msgr2=0x55f26a1b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f278c005d00 tx=0x7f278c01d000 comp rx=0 tx=0).stop 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- 172.21.15.87:0/67795987 shutdown_connections 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a19f560 0x55f26a193c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a1929b0 0x55f26a1b7820 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 --2- 172.21.15.87:0/67795987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- 172.21.15.87:0/67795987 >> 172.21.15.87:0/67795987 conn(0x55f26a18e680 msgr2=0x55f26a1978e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- 172.21.15.87:0/67795987 shutdown_connections 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- 172.21.15.87:0/67795987 wait complete. 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 -- start start 2024-06-06T08:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.635+0000 7f27a99f6740 1 Processor -- start 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a199be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ab350 con 0x55f26a19f560 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ac7c0 con 0x55f26a192090 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aba20 con 0x55f26a1929b0 2024-06-06T08:06:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a99f6740 10 client.0 _ll_register_callbacks cb 0x7f2794009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f279ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a49ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1a7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:35634/0 (socket says 172.21.15.87:35634) 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a49ae640 1 -- 172.21.15.87:0/192102377 learned_addr learned my addr 172.21.15.87:0/192102377 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 -- 172.21.15.87:0/192102377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 msgr2=0x55f26a1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 -- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 msgr2=0x55f26a1a7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1a7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 -- 172.21.15.87:0/192102377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26a1a55a0 con 0x55f26a19f560 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f27a51af640 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a199be0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f27980c5c30 tx=0x7f27980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43798958b26b48e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f277ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.636+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980bd500 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f27980ee320 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278c022270 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f27980bce00 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 10 client.18048 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f26a218480 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278c022270 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f279804e070 con 0x55f26a19f560 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 10 client.18048 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 20 client.18048 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a218880 0x55f26a21ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 4 client.18048 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f26a103c20 con 0x55f26a218880 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.637+0000 7f27a49ae640 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a218880 0x55f26a21ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.638+0000 7f27a49ae640 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a218880 0x55f26a21ac70 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ec5ebb6ccfb6f3 server_cookie=65160f618241069e in_seq=0 out_seq=0 2024-06-06T08:06:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:21.638+0000 7f277ffff640 10 client.18048 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:22.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 64626 14.47 MB/sec execute 144 sec latency 526.354 ms 2024-06-06T08:06:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.385+0000 7f277ffff640 1 -- 172.21.15.87:0/192102377 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f26a103c20 con 0x55f26a218880 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.385+0000 7f277ffff640 10 client.18048 handle_command_reply: tid=0 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.385+0000 7f27a99f6740 1 client.18048 shutdown 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 msgr2=0x55f26a199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a199be0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f27980c5c30 tx=0x7f27980095c0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 shutdown_connections 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f26a218880 0x55f26a21ac70 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f277ffff640 0 client.18048 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f26a19f560 0x55f26a199be0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f26a1929b0 0x55f26a1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 --2- 172.21.15.87:0/192102377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f26a192090 0x55f26a1a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 >> 172.21.15.87:0/192102377 conn(0x55f26a18e680 msgr2=0x55f26a18cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 shutdown_connections 2024-06-06T08:06:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 1 -- 172.21.15.87:0/192102377 wait complete. 2024-06-06T08:06:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.386+0000 7f27a99f6740 20 client.18048 trim_cache size 0 max 16384 2024-06-06T08:06:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:06:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:06:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:06:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 msgr2=0x7f27900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 0x7f27900a52e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27a0105c40 tx=0x7f278c0128d0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 msgr2=0x7f27a019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2798031e70 tx=0x7f2798032fb0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 shutdown_connections 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f27900a2e20 0x7f27900a52e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a0106890 0x7f27a019b920 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27a0101290 0x7f27a01975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 --2- 172.21.15.87:0/4273729359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27a01008d0 0x7f27a0197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 >> 172.21.15.87:0/4273729359 conn(0x7f27a0077a20 msgr2=0x7f27a0103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 shutdown_connections 2024-06-06T08:06:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:22.387+0000 7f27a6c12640 1 -- 172.21.15.87:0/4273729359 wait complete. 2024-06-06T08:06:22.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:22 smithi087 ceph-mon[32276]: pgmap v582: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 457 op/s 2024-06-06T08:06:22.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:22 smithi195 ceph-mon[37040]: pgmap v582: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 457 op/s 2024-06-06T08:06:22.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:22 smithi181 ceph-mon[36790]: pgmap v582: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 457 op/s 2024-06-06T08:06:23.176 INFO:tasks.workunit.client.0.smithi087.stdout: 10 64824 14.44 MB/sec execute 145 sec latency 553.113 ms 2024-06-06T08:06:24.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 65135 14.44 MB/sec execute 146 sec latency 624.711 ms 2024-06-06T08:06:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:24 smithi195 ceph-mon[37040]: pgmap v583: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 641 op/s 2024-06-06T08:06:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:24 smithi181 ceph-mon[36790]: pgmap v583: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 641 op/s 2024-06-06T08:06:24.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:24 smithi087 ceph-mon[32276]: pgmap v583: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.1 MiB/s wr, 641 op/s 2024-06-06T08:06:25.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 65554 14.55 MB/sec execute 147 sec latency 166.136 ms 2024-06-06T08:06:26.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 65857 14.57 MB/sec execute 148 sec latency 344.125 ms 2024-06-06T08:06:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:26 smithi195 ceph-mon[37040]: pgmap v584: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 445 op/s 2024-06-06T08:06:26.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:26 smithi181 ceph-mon[36790]: pgmap v584: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 445 op/s 2024-06-06T08:06:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:26 smithi087 ceph-mon[32276]: pgmap v584: 129 pgs: 129 active+clean; 468 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 445 op/s 2024-06-06T08:06:27.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 66213 14.60 MB/sec execute 149 sec latency 259.854 ms 2024-06-06T08:06:28.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 66469 14.62 MB/sec execute 150 sec latency 456.726 ms 2024-06-06T08:06:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:28 smithi195 ceph-mon[37040]: pgmap v585: 129 pgs: 129 active+clean; 463 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-06-06T08:06:28.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:28 smithi181 ceph-mon[36790]: pgmap v585: 129 pgs: 129 active+clean; 463 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-06-06T08:06:28.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:28 smithi087 ceph-mon[32276]: pgmap v585: 129 pgs: 129 active+clean; 463 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-06-06T08:06:29.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 66744 14.61 MB/sec execute 151 sec latency 428.077 ms 2024-06-06T08:06:30.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 67071 14.60 MB/sec execute 152 sec latency 1224.197 ms 2024-06-06T08:06:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:30 smithi195 ceph-mon[37040]: pgmap v586: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.8 MiB/s wr, 668 op/s 2024-06-06T08:06:30.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:30 smithi181 ceph-mon[36790]: pgmap v586: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.8 MiB/s wr, 668 op/s 2024-06-06T08:06:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:30 smithi087 ceph-mon[32276]: pgmap v586: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.8 MiB/s wr, 668 op/s 2024-06-06T08:06:30.700 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.3 2024-06-06T08:06:30.700 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:06:30.977 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:31.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 67417 14.63 MB/sec execute 153 sec latency 2224.236 ms 2024-06-06T08:06:32.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 67822 14.71 MB/sec execute 154 sec latency 3224.273 ms 2024-06-06T08:06:32.442 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:32 smithi087 ceph-mon[32276]: pgmap v587: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-06-06T08:06:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:32 smithi195 ceph-mon[37040]: pgmap v587: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-06-06T08:06:32.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:32 smithi181 ceph-mon[36790]: pgmap v587: 129 pgs: 129 active+clean; 471 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-06-06T08:06:33.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 68105 14.72 MB/sec execute 155 sec latency 4224.312 ms 2024-06-06T08:06:34.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 68361 14.71 MB/sec execute 156 sec latency 5224.342 ms 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 -- 172.21.15.87:0/746145116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006ab30 msgr2=0x7ff6c010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 --2- 172.21.15.87:0/746145116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006ab30 0x7ff6c010f320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800b0a0 tx=0x7ff6b800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 -- 172.21.15.87:0/746145116 shutdown_connections 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 --2- 172.21.15.87:0/746145116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 0x7ff6c01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 --2- 172.21.15.87:0/746145116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006ab30 0x7ff6c010f320 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 --2- 172.21.15.87:0/746145116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006a170 0x7ff6c006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 -- 172.21.15.87:0/746145116 >> 172.21.15.87:0/746145116 conn(0x7ff6c0078340 msgr2=0x7ff6c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:34.252 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:34 smithi087 ceph-mon[32276]: pgmap v588: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.6 MiB/s wr, 665 op/s 2024-06-06T08:06:34.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 -- 172.21.15.87:0/746145116 shutdown_connections 2024-06-06T08:06:34.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.251+0000 7ff6c5182700 1 -- 172.21.15.87:0/746145116 wait complete. 2024-06-06T08:06:34.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.252+0000 7ff6c5182700 1 Processor -- start 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.252+0000 7ff6c5182700 1 -- start start 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 0x7ff6c0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 0x7ff6c01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 0x7ff6c0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0112780 con 0x7ff6c006a170 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0112600 con 0x7ff6c0108500 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6c5182700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0112900 con 0x7ff6c006ab30 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6bed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 0x7ff6c0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 0x7ff6c01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6bed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 0x7ff6c0101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59098/0 (socket says 172.21.15.87:59098) 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6bed9d700 1 -- 172.21.15.87:0/2627182588 learned_addr learned my addr 172.21.15.87:0/2627182588 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6bf59e700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 0x7ff6c0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 -- 172.21.15.87:0/2627182588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 msgr2=0x7ff6c0101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:34.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 0x7ff6c0101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 -- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 msgr2=0x7ff6c0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 0x7ff6c0101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 -- 172.21.15.87:0/2627182588 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c01a70e0 con 0x7ff6c006ab30 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.253+0000 7ff6be59c700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 0x7ff6c01019a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e9f0 tx=0x7ff6b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.254+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004220 con 0x7ff6c006ab30 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.254+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c01a73d0 con 0x7ff6c006ab30 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.254+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b4004d10 con 0x7ff6c006ab30 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.254+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4005020 con 0x7ff6c006ab30 2024-06-06T08:06:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.254+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c01a78c0 con 0x7ff6c006ab30 2024-06-06T08:06:34.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.255+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c0069180 con 0x7ff6c006ab30 2024-06-06T08:06:34.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.256+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff6b4004830 con 0x7ff6c006ab30 2024-06-06T08:06:34.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.256+0000 7ff6a7fff700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 0x7ff6a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:34.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.257+0000 7ff6bed9d700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 0x7ff6a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:34.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.257+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7ff6b40cf770 con 0x7ff6c006ab30 2024-06-06T08:06:34.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.257+0000 7ff6bed9d700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 0x7ff6a80a5250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009ca0 tx=0x7ff6b0009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:34.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.262+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6b409bf70 con 0x7ff6c006ab30 2024-06-06T08:06:34.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:34 smithi195 ceph-mon[37040]: pgmap v588: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.6 MiB/s wr, 665 op/s 2024-06-06T08:06:34.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:34 smithi181 ceph-mon[36790]: pgmap v588: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.6 MiB/s wr, 665 op/s 2024-06-06T08:06:34.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.567+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6c0102990 con 0x7ff6c006ab30 2024-06-06T08:06:34.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.568+0000 7ff6a7fff700 1 -- 172.21.15.87:0/2627182588 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11881 (secure 0 0 0) 0x7ff6b409b870 con 0x7ff6c006ab30 2024-06-06T08:06:34.568 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:06:34.569 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:04:59.022587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24446,24467]},"id":1}]} 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 msgr2=0x7ff6a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 0x7ff6a80a5250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009ca0 tx=0x7ff6b0009260 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 msgr2=0x7ff6c01019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 0x7ff6c01019a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e9f0 tx=0x7ff6b400ed00 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 shutdown_connections 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0108500 0x7ff6c0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff6a80a2d90 0x7ff6a80a5250 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c006a170 0x7ff6c0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 --2- 172.21.15.87:0/2627182588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c006ab30 0x7ff6c01019a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 >> 172.21.15.87:0/2627182588 conn(0x7ff6c0078340 msgr2=0x7ff6c00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 shutdown_connections 2024-06-06T08:06:34.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:34.571+0000 7ff6c5182700 1 -- 172.21.15.87:0/2627182588 wait complete. 2024-06-06T08:06:34.572 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 52 2024-06-06T08:06:35.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 68687 14.69 MB/sec execute 157 sec latency 6224.376 ms 2024-06-06T08:06:35.255 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 143.3 --await-for 45.0' 2024-06-06T08:06:35.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 143.3 --await-for 45.0 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 -- 172.21.15.87:0/920877620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41008b0 msgr2=0x7f70c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 --2- 172.21.15.87:0/920877620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41008b0 0x7f70c4100d30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009930 tx=0x7f70b802a420 comp rx=0 tx=0).stop 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 -- 172.21.15.87:0/920877620 shutdown_connections 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 --2- 172.21.15.87:0/920877620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c4101270 0x7f70c410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 --2- 172.21.15.87:0/920877620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41008b0 0x7f70c4100d30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 --2- 172.21.15.87:0/920877620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c41068c0 0x7f70c4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 -- 172.21.15.87:0/920877620 >> 172.21.15.87:0/920877620 conn(0x7f70c4077ab0 msgr2=0x7f70c4077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 -- 172.21.15.87:0/920877620 shutdown_connections 2024-06-06T08:06:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.505+0000 7f70c8aba640 1 -- 172.21.15.87:0/920877620 wait complete. 2024-06-06T08:06:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 Processor -- start 2024-06-06T08:06:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 -- start start 2024-06-06T08:06:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c41008b0 0x7f70c4196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41068c0 0x7f70c419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410fc10 con 0x7f70c41068c0 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410fa90 con 0x7f70c4101270 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:50690/0 (socket says 172.21.15.87:50690) 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c1d9b640 1 -- 172.21.15.87:0/1592689288 learned_addr learned my addr 172.21.15.87:0/1592689288 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c8aba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410fd90 con 0x7f70c41008b0 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41068c0 0x7f70c419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.506+0000 7f70c259c640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c41008b0 0x7f70c4196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 -- 172.21.15.87:0/1592689288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c41008b0 msgr2=0x7f70c4196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c41008b0 0x7f70c4196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 -- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41068c0 msgr2=0x7f70c419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41068c0 0x7f70c419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 -- 172.21.15.87:0/1592689288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c419b000 con 0x7f70c4101270 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70b802a990 tx=0x7f70b8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70b37fe640 1 -- 172.21.15.87:0/1592689288 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b803f070 con 0x7f70c4101270 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c419b290 con 0x7f70c4101270 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70b37fe640 1 -- 172.21.15.87:0/1592689288 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8031c90 con 0x7f70c4101270 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41a2b80 con 0x7f70c4101270 2024-06-06T08:06:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.507+0000 7f70b37fe640 1 -- 172.21.15.87:0/1592689288 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8035030 con 0x7f70c4101270 2024-06-06T08:06:35.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:35 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2627182588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70b37fe640 1 -- 172.21.15.87:0/1592689288 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70b8042070 con 0x7f70c4101270 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 Processor -- start 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 -- start start 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0170 0x55e63b7b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e63b7144f0 con 0x55e63b7bb350 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e63b6b2870 con 0x55e63b7b0170 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.510+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e63b7104b0 con 0x55e63b7b0a90 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:39634/0 (socket says 172.21.15.87:39634) 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 -- 172.21.15.87:0/3058462488 learned_addr learned my addr 172.21.15.87:0/3058462488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0170 0x55e63b7b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 -- 172.21.15.87:0/3058462488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0170 msgr2=0x55e63b7b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0170 0x55e63b7b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 -- 172.21.15.87:0/3058462488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 msgr2=0x55e63b7bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7bd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 -- 172.21.15.87:0/3058462488 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63b6ce9b0 con 0x55e63b7b0a90 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70b37fe640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 0x7f70ac0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c259c640 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004530 tx=0x7f70b400dc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a198f155f9c1b9c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 0x7f70ac0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70b37fe640 1 -- 172.21.15.87:0/1592689288 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f70b80efe20 con 0x7f70c4101270 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70b0ff9640 1 -- 172.21.15.87:0/3058462488 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4003d70 con 0x55e63b7b0a90 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70b0ff9640 1 -- 172.21.15.87:0/3058462488 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401e070 con 0x55e63b7b0a90 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.511+0000 7f70b0ff9640 1 -- 172.21.15.87:0/3058462488 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40187c0 con 0x55e63b7b0a90 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3058462488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 msgr2=0x55e63b7bae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 0x7f70ac0a5330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70bc005fd0 tx=0x7f70bc0096a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004530 tx=0x7f70b400dc70 comp rx=0 tx=0).stop 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3058462488 shutdown_connections 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7bd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0a90 0x55e63b7bae10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3058462488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0170 0x55e63b7b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3058462488 >> 172.21.15.87:0/3058462488 conn(0x55e63b7ac680 msgr2=0x55e63b7aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3058462488 shutdown_connections 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.512+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3058462488 wait complete. 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 -- start start 2024-06-06T08:06:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 Processor -- start 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e63b7bdf80 con 0x55e63b7bb350 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e63b69dae0 con 0x55e63b7b0a90 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70c2d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e63b7be100 con 0x55e63b7b0170 2024-06-06T08:06:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70c259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70c2d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59142/0 (socket says 172.21.15.87:59142) 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70c2d9d640 1 -- 172.21.15.87:0/2957999158 learned_addr learned my addr 172.21.15.87:0/2957999158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.513+0000 7f70cb7ca740 10 client.0 _ll_register_callbacks cb 0x55e63b6d69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 -- 172.21.15.87:0/2957999158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 msgr2=0x55e63b7b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 -- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 msgr2=0x55e63b7b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 -- 172.21.15.87:0/2957999158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63b428210 con 0x55e63b7bb350 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c259c640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f70bc01f480 tx=0x7f70bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5398a053442a6efc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc02c060 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f70bc017030 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b401e070 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f70bc033050 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70cb7ca740 10 client.18066 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e63b4f4620 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.514+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401e070 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f70bc0278a0 con 0x55e63b7bb350 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 10 client.18066 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 20 client.18066 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b7cb540 0x55e63b7cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 4 client.18066 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e63b727f10 con 0x55e63b7cb540 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.515+0000 7f70c259c640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b7cb540 0x55e63b7cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.516+0000 7f70a9ffb640 10 client.18066 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.516+0000 7f70c259c640 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b7cb540 0x55e63b7cd9b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510a6617bf44388 server_cookie=4ee7d557f1310af9 in_seq=0 out_seq=0 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.516+0000 7f70a9ffb640 1 -- 172.21.15.87:0/2957999158 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e63b727f10 con 0x55e63b7cb540 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.516+0000 7f70a9ffb640 10 client.18066 handle_command_reply: tid=0 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.516+0000 7f70cb7ca740 1 client.18066 shutdown 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 msgr2=0x55e63b7b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f70bc01f480 tx=0x7f70bc004ab0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 shutdown_connections 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b7cb540 0x55e63b7cd9b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7bb350 0x55e63b7b8c60 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70a9ffb640 0 client.18066 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2957999158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 >> 172.21.15.87:0/2957999158 conn(0x55e63b7ac680 msgr2=0x55e63b7b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 shutdown_connections 2024-06-06T08:06:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2957999158 wait complete. 2024-06-06T08:06:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.517+0000 7f70cb7ca740 20 client.18066 trim_cache size 0 max 16384 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.532+0000 7f70cb7ca740 1 Processor -- start 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 -- start start 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7bb350 0x55e63b7d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4012a20 con 0x55e63b7b0170 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4010290 con 0x55e63b7b0a90 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400e100 con 0x55e63b7bb350 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59148/0 (socket says 172.21.15.87:59148) 2024-06-06T08:06:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 -- 172.21.15.87:0/2694560454 learned_addr learned my addr 172.21.15.87:0/2694560454 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7bb350 0x55e63b7d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 -- 172.21.15.87:0/2694560454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7bb350 msgr2=0x55e63b7d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7bb350 0x55e63b7d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 -- 172.21.15.87:0/2694560454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 msgr2=0x55e63b7d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 -- 172.21.15.87:0/2694560454 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63b7a7fe0 con 0x55e63b7b0170 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.533+0000 7f70c259c640 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019f60 tx=0x7f70b4019f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82195a6114961408 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70abfff640 1 -- 172.21.15.87:0/2694560454 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4005340 con 0x55e63b7b0170 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70abfff640 1 -- 172.21.15.87:0/2694560454 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401e050 con 0x55e63b7b0170 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70abfff640 1 -- 172.21.15.87:0/2694560454 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40196e0 con 0x55e63b7b0170 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2694560454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 msgr2=0x55e63b7b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019f60 tx=0x7f70b4019f90 comp rx=0 tx=0).stop 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.534+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2694560454 shutdown_connections 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7bb350 0x55e63b7d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7b0a90 0x55e63b7d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/2694560454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0170 0x55e63b7b71c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2694560454 >> 172.21.15.87:0/2694560454 conn(0x55e63b7ac680 msgr2=0x55e63b7b6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2694560454 shutdown_connections 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- 172.21.15.87:0/2694560454 wait complete. 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- start start 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 Processor -- start 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7bb350 0x55e63b838690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400e750 con 0x55e63b7b0a90 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400fbc0 con 0x55e63b7bb350 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400ee20 con 0x55e63b7b0170 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70cb7ca740 10 client.0 _ll_register_callbacks cb 0x7f70bc003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70c1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59154/0 (socket says 172.21.15.87:59154) 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70c1d9b640 1 -- 172.21.15.87:0/3713917835 learned_addr learned my addr 172.21.15.87:0/3713917835 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70c259c640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.535+0000 7f70c2d9d640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7bb350 0x55e63b838690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 -- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7bb350 msgr2=0x55e63b838690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7bb350 0x55e63b838690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 -- 172.21.15.87:0/3713917835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 msgr2=0x55e63b7bee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 -- 172.21.15.87:0/3713917835 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63b838bd0 con 0x55e63b7b0170 2024-06-06T08:06:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70c259c640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7be940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004ee0 tx=0x7f70b400d3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a086091afd69e301 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4005550 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b401e050 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f70b4004680 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f70b40039c0 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70cb7ca740 10 client.27184 fetch_fsmap learned FSMap version 52 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e63b839e80 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.536+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401e050 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f70b402b400 con 0x55e63b7b0170 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 10 client.27184 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 10 client.27184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 20 client.27184 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b83a3a0 0x55e63b83c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 4 client.27184 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e63b721c20 con 0x55e63b83a3a0 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b83a3a0 0x55e63b83c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70a9ffb640 10 client.27184 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:35.537+0000 7f70c1d9b640 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b83a3a0 0x55e63b83c790 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c063163a51185d7 server_cookie=64d16b4b4e721456 in_seq=0 out_seq=0 2024-06-06T08:06:35.552 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:35.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:35 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2627182588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:35.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:35 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2627182588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:36.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 68853 14.65 MB/sec execute 158 sec latency 7224.419 ms 2024-06-06T08:06:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70a9ffb640 1 -- 172.21.15.87:0/3713917835 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e63b721c20 con 0x55e63b83a3a0 2024-06-06T08:06:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70a9ffb640 10 client.27184 handle_command_reply: tid=0 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 client.27184 shutdown 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 msgr2=0x55e63b7be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7be940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004ee0 tx=0x7f70b400d3a0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 shutdown_connections 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e63b83a3a0 0x55e63b83c790 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e63b7bb350 0x55e63b838690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e63b7b0a90 0x55e63b7bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 --2- 172.21.15.87:0/3713917835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e63b7b0170 0x55e63b7be940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 >> 172.21.15.87:0/3713917835 conn(0x55e63b7ac680 msgr2=0x55e63b7b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70a9ffb640 0 client.27184 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 shutdown_connections 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.324+0000 7f70cb7ca740 1 -- 172.21.15.87:0/3713917835 wait complete. 2024-06-06T08:06:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.325+0000 7f70cb7ca740 20 client.27184 trim_cache size 0 max 16384 2024-06-06T08:06:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:06:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:06:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.325+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 msgr2=0x7f70ac0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.325+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 0x7f70ac0a5330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70bc005fd0 tx=0x7f70bc0096a0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.325+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 msgr2=0x7f70c4196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.325+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70b802a990 tx=0x7f70b8003b90 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 shutdown_connections 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f70ac0a2e70 0x7f70ac0a5330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70c41068c0 0x7f70c419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70c4101270 0x7f70c4196550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 --2- 172.21.15.87:0/1592689288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c41008b0 0x7f70c4196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 >> 172.21.15.87:0/1592689288 conn(0x7f70c4077ab0 msgr2=0x7f70c4103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 shutdown_connections 2024-06-06T08:06:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:36.326+0000 7f70c8aba640 1 -- 172.21.15.87:0/1592689288 wait complete. 2024-06-06T08:06:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:36 smithi087 ceph-mon[32276]: pgmap v589: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 437 op/s 2024-06-06T08:06:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:36 smithi195 ceph-mon[37040]: pgmap v589: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 437 op/s 2024-06-06T08:06:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:36 smithi181 ceph-mon[36790]: pgmap v589: 129 pgs: 129 active+clean; 489 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 437 op/s 2024-06-06T08:06:37.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69217 14.67 MB/sec execute 159 sec latency 7528.701 ms 2024-06-06T08:06:37.464 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:37 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:37.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.483+0000 7fedddf33700 1 -- 172.21.15.87:0/944428689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 msgr2=0x7fedd8107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.483+0000 7fedddf33700 1 --2- 172.21.15.87:0/944428689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd8107120 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0099c0 tx=0x7fedcc009cd0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 -- 172.21.15.87:0/944428689 shutdown_connections 2024-06-06T08:06:37.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 --2- 172.21.15.87:0/944428689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd8107850 0x7fedd810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 --2- 172.21.15.87:0/944428689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd806a3d0 0x7fedd806a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 --2- 172.21.15.87:0/944428689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd8107120 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 -- 172.21.15.87:0/944428689 >> 172.21.15.87:0/944428689 conn(0x7fedd8078330 msgr2=0x7fedd8078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 -- 172.21.15.87:0/944428689 shutdown_connections 2024-06-06T08:06:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 -- 172.21.15.87:0/944428689 wait complete. 2024-06-06T08:06:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 Processor -- start 2024-06-06T08:06:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.484+0000 7fedddf33700 1 -- start start 2024-06-06T08:06:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd806a3d0 0x7fedd819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd8107850 0x7fedd8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112780 con 0x7fedd806ad80 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112600 con 0x7fedd806a3d0 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112900 con 0x7fedd8107850 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd806a3d0 0x7fedd819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39612/0 (socket says 172.21.15.87:39612) 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 -- 172.21.15.87:0/662382540 learned_addr learned my addr 172.21.15.87:0/662382540 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd7fff700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd8107850 0x7fedd8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 -- 172.21.15.87:0/662382540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd8107850 msgr2=0x7fedd8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd8107850 0x7fedd8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 -- 172.21.15.87:0/662382540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd806a3d0 msgr2=0x7fedd819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd806a3d0 0x7fedd819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 -- 172.21.15.87:0/662382540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd8199160 con 0x7fedd806ad80 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006010 tx=0x7fedcc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd4ff9700 1 -- 172.21.15.87:0/662382540 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc003680 con 0x7fedd806ad80 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd4ff9700 1 -- 172.21.15.87:0/662382540 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0050d0 con 0x7fedd806ad80 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd81993f0 con 0x7fedd806ad80 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.485+0000 7fedd4ff9700 1 -- 172.21.15.87:0/662382540 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc003db0 con 0x7fedd806ad80 2024-06-06T08:06:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.486+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a56c0 con 0x7fedd806ad80 2024-06-06T08:06:37.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.488+0000 7fedd4ff9700 1 -- 172.21.15.87:0/662382540 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fedcc005280 con 0x7fedd806ad80 2024-06-06T08:06:37.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.489+0000 7fedd4ff9700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 0x7fedc00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.489+0000 7fedd4ff9700 1 -- 172.21.15.87:0/662382540 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7fedcc0d0480 con 0x7fedd806ad80 2024-06-06T08:06:37.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.489+0000 7fedd77fe700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 0x7fedc00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 Processor -- start 2024-06-06T08:06:37.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd77fe700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 0x7fedc00a5510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fedd81073b0 tx=0x7fedc8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 -- start start 2024-06-06T08:06:37.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777daa37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daa4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55777d9960f0 con 0x55777daa3ce0 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55777d9f49e0 con 0x55777daa9eb0 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fede87b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55777d95c5f0 con 0x55777daa9590 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:57796/0 (socket says 172.21.15.87:57796) 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd77fe700 1 -- 172.21.15.87:0/4251381740 learned_addr learned my addr 172.21.15.87:0/4251381740 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777daa37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.490+0000 7fedd7fff700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daa4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 -- 172.21.15.87:0/4251381740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 msgr2=0x55777daa37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777daa37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 -- 172.21.15.87:0/4251381740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 msgr2=0x55777daa4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daa4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 -- 172.21.15.87:0/4251381740 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777d943a60 con 0x55777daa9590 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd7fff700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daa4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedd77fe700 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fedc8011dd0 tx=0x7fedc8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62ed90aa29de499 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedbdffb700 1 -- 172.21.15.87:0/4251381740 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8003880 con 0x55777daa9590 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedbdffb700 1 -- 172.21.15.87:0/4251381740 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc801e420 con 0x55777daa9590 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.491+0000 7fedbdffb700 1 -- 172.21.15.87:0/4251381740 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8026620 con 0x55777daa9590 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- 172.21.15.87:0/4251381740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 msgr2=0x55777daa9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fedc8011dd0 tx=0x7fedc8014000 comp rx=0 tx=0).stop 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- 172.21.15.87:0/4251381740 shutdown_connections 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daa4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9590 0x55777daa9970 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 --2- 172.21.15.87:0/4251381740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777daa37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- 172.21.15.87:0/4251381740 >> 172.21.15.87:0/4251381740 conn(0x55777d9d0210 msgr2=0x55777da9ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:37.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- 172.21.15.87:0/4251381740 shutdown_connections 2024-06-06T08:06:37.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- 172.21.15.87:0/4251381740 wait complete. 2024-06-06T08:06:37.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 -- start start 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.492+0000 7fede87b6b80 1 Processor -- start 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 0x55777daafc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 0x55777daad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55777d995150 con 0x55777daa9590 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55777d9f09a0 con 0x55777daa3ce0 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55777dab0890 con 0x55777daa9eb0 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fede87b6b80 10 client.0 _ll_register_callbacks cb 0x55777d98a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 0x55777daafc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39636/0 (socket says 172.21.15.87:39636) 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 0x55777daad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 -- 172.21.15.87:0/2442856559 learned_addr learned my addr 172.21.15.87:0/2442856559 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 -- 172.21.15.87:0/2442856559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 msgr2=0x55777daad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 0x55777daad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 -- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 msgr2=0x55777daafc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 0x55777daafc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 -- 172.21.15.87:0/2442856559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777d9ba8e0 con 0x55777daa9590 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd77fe700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 0x55777daafc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd7fff700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 0x55777daad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fedcc09c5a0 tx=0x7fedcc09c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32a4973ade2e6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.493+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc09cd10 con 0x55777daa9590 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7fedcc0163a0 con 0x55777daa9590 2024-06-06T08:06:37.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc801e420 con 0x55777daa9590 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedcc0165a0 con 0x55777daa9590 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 10 client.18084 fetch_fsmap learned FSMap version 53 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55777dabb080 con 0x55777daa9590 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc801e420 con 0x55777daa9590 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fedcc0a4d60 con 0x55777daa9590 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 10 client.18084 fetch_fsmap finished waiting for FSMap version 53 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 10 client.18084 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 20 client.18084 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777dabd690 0x55777dabfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 4 client.18084 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777da01ee0 con 0x55777dabd690 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.494+0000 7fedd77fe700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777dabd690 0x55777dabfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.495+0000 7fedd77fe700 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777dabd690 0x55777dabfa80 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daaed656ae3df83 server_cookie=e87938153ac0b5aa in_seq=0 out_seq=0 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.495+0000 7fedb6ffd700 10 client.18084 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fedb6ffd700 1 -- 172.21.15.87:0/2442856559 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55777da01ee0 con 0x55777dabd690 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fedb6ffd700 10 client.18084 handle_command_reply: tid=0 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 client.18084 shutdown 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 msgr2=0x55777daacf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fedcc09c5a0 tx=0x7fedcc09c150 comp rx=0 tx=0).stop 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 shutdown_connections 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777dabd690 0x55777dabfa80 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa3ce0 0x55777daafc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa9eb0 0x55777daad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fedb6ffd700 0 client.18084 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 --2- 172.21.15.87:0/2442856559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa9590 0x55777daacf60 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 >> 172.21.15.87:0/2442856559 conn(0x55777d9d0210 msgr2=0x55777da9dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:37.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 shutdown_connections 2024-06-06T08:06:37.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.496+0000 7fede87b6b80 1 -- 172.21.15.87:0/2442856559 wait complete. 2024-06-06T08:06:37.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.497+0000 7fede87b6b80 20 client.18084 trim_cache size 0 max 16384 2024-06-06T08:06:37.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:37 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 Processor -- start 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 -- start start 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777da9e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa8de0 0x55777da9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81b0eb0 con 0x55777daa3ce0 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ae710 con 0x55777daa9eb0 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fede87b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ac580 con 0x55777daa8de0 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39646/0 (socket says 172.21.15.87:39646) 2024-06-06T08:06:37.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fedd77fe700 1 -- 172.21.15.87:0/59671395 learned_addr learned my addr 172.21.15.87:0/59671395 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777da9e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.518+0000 7fedd7fff700 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa8de0 0x55777da9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 -- 172.21.15.87:0/59671395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa8de0 msgr2=0x55777da9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa8de0 0x55777da9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 -- 172.21.15.87:0/59671395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 msgr2=0x55777da9e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777da9e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 -- 172.21.15.87:0/59671395 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777da96530 con 0x55777daa3ce0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedd77fe700 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fedc801ee90 tx=0x7fedc8029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71472f662c12e55 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedbd7fa700 1 -- 172.21.15.87:0/59671395 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc801ec00 con 0x55777daa3ce0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedbd7fa700 1 -- 172.21.15.87:0/59671395 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8005520 con 0x55777daa3ce0 2024-06-06T08:06:37.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.519+0000 7fedbd7fa700 1 -- 172.21.15.87:0/59671395 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8004630 con 0x55777daa3ce0 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- 172.21.15.87:0/59671395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 msgr2=0x55777daaf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fedc801ee90 tx=0x7fedc8029040 comp rx=0 tx=0).stop 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- 172.21.15.87:0/59671395 shutdown_connections 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa8de0 0x55777da9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa3ce0 0x55777daaf7a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 --2- 172.21.15.87:0/59671395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777da9e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- 172.21.15.87:0/59671395 >> 172.21.15.87:0/59671395 conn(0x55777d9d0210 msgr2=0x55777daad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- 172.21.15.87:0/59671395 shutdown_connections 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- 172.21.15.87:0/59671395 wait complete. 2024-06-06T08:06:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.520+0000 7fede87b6b80 1 -- start start 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 Processor -- start 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 0x55777dab5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777dab2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81acbd0 con 0x55777daa8de0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ae040 con 0x55777daa9eb0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81ad2a0 con 0x55777daa3ce0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fede87b6b80 10 client.0 _ll_register_callbacks cb 0x55777d98a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 0x55777dab5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39656/0 (socket says 172.21.15.87:39656) 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd6ffd700 1 -- 172.21.15.87:0/142437404 learned_addr learned my addr 172.21.15.87:0/142437404 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd7fff700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777dab2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 -- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 msgr2=0x55777dab2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777dab2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 -- 172.21.15.87:0/142437404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 msgr2=0x55777dab62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd77fe700 1 -- 172.21.15.87:0/142437404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777dab3390 con 0x55777daa3ce0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.521+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedd77fe700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 0x55777dab5d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fedc80117d0 tx=0x7fedc8011b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ccd299b0fd5e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc801e420 con 0x55777daa3ce0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8005520 con 0x55777daa3ce0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7fedc8003bb0 con 0x55777daa3ce0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedc8050050 con 0x55777daa3ce0 2024-06-06T08:06:37.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 10 client.27202 fetch_fsmap learned FSMap version 53 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55777db2d160 con 0x55777daa3ce0 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8005520 con 0x55777daa3ce0 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fedc8032aa0 con 0x55777daa3ce0 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 10 client.27202 fetch_fsmap finished waiting for FSMap version 53 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 10 client.27202 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 20 client.27202 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777db2d5a0 0x55777db2f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 4 client.27202 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.3, "await_for": 45.0}] 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.522+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.3, "await_for": 45.0}) -- 0x55777d9fc460 con 0x55777db2d5a0 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.523+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777db2d5a0 0x55777db2f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.523+0000 7fedb6ffd700 10 client.27202 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:37.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:37.523+0000 7fedd6ffd700 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777db2d5a0 0x55777db2f9c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38d96915afcbfed server_cookie=f796a06d25a63841 in_seq=0 out_seq=0 2024-06-06T08:06:37.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:37 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:38.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69350 14.61 MB/sec execute 160 sec latency 633.126 ms 2024-06-06T08:06:38.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fedb6ffd700 1 -- 172.21.15.87:0/142437404 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55777d9fc460 con 0x55777db2d5a0 2024-06-06T08:06:38.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fedb6ffd700 10 client.27202 handle_command_reply: tid=0 2024-06-06T08:06:38.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 client.27202 shutdown 2024-06-06T08:06:38.474 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:38 smithi087 ceph-mon[32276]: pgmap v590: 129 pgs: 129 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 msgr2=0x55777dab5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 0x55777dab5d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fedc80117d0 tx=0x7fedc8011b00 comp rx=0 tx=0).stop 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 shutdown_connections 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55777db2d5a0 0x55777db2f9c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55777daa9eb0 0x55777dab2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55777daa3ce0 0x55777dab5d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fedb6ffd700 0 client.27202 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 --2- 172.21.15.87:0/142437404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55777daa8de0 0x55777dab62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 >> 172.21.15.87:0/142437404 conn(0x55777d9d0210 msgr2=0x55777daa2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 shutdown_connections 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 1 -- 172.21.15.87:0/142437404 wait complete. 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.474+0000 7fede87b6b80 20 client.27202 trim_cache size 0 max 16384 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 53, 2024-06-06T08:06:38.476 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 58, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "d85bb652": { 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 58, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:06:38.477 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 143.3, 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:38.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 msgr2=0x7fedc00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 0x7fedc00a5510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fedd81073b0 tx=0x7fedc8009260 comp rx=0 tx=0).stop 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 msgr2=0x7fedd819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006010 tx=0x7fedcc00f690 comp rx=0 tx=0).stop 2024-06-06T08:06:38.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 shutdown_connections 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedd806a3d0 0x7fedd819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fedc00a3050 0x7fedc00a5510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedd8107850 0x7fedd8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 --2- 172.21.15.87:0/662382540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedd806ad80 0x7fedd819ed30 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 >> 172.21.15.87:0/662382540 conn(0x7fedd8078330 msgr2=0x7fedd80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 shutdown_connections 2024-06-06T08:06:38.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:38.475+0000 7fedddf33700 1 -- 172.21.15.87:0/662382540 wait complete. 2024-06-06T08:06:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:38 smithi195 ceph-mon[37040]: pgmap v590: 129 pgs: 129 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-06-06T08:06:38.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:38 smithi181 ceph-mon[36790]: pgmap v590: 129 pgs: 129 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-06-06T08:06:39.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.52 MB/sec execute 161 sec latency 1633.189 ms 2024-06-06T08:06:39.327 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd85bb652', quiesce took 8.6 seconds. Will release after: 14.700000000000001 2024-06-06T08:06:40.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.43 MB/sec execute 162 sec latency 2633.223 ms 2024-06-06T08:06:40.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:40 smithi195 ceph-mon[37040]: pgmap v591: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 613 op/s 2024-06-06T08:06:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:06:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:06:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:06:40.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:06:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:40 smithi181 ceph-mon[36790]: pgmap v591: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 613 op/s 2024-06-06T08:06:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:06:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:06:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:06:40.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:06:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:40 smithi087 ceph-mon[32276]: pgmap v591: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 613 op/s 2024-06-06T08:06:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:06:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:06:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:06:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:06:41.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.34 MB/sec execute 163 sec latency 3633.257 ms 2024-06-06T08:06:42.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.25 MB/sec execute 164 sec latency 4633.350 ms 2024-06-06T08:06:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:42 smithi195 ceph-mon[37040]: pgmap v592: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 440 op/s 2024-06-06T08:06:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:42 smithi181 ceph-mon[36790]: pgmap v592: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 440 op/s 2024-06-06T08:06:42.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:42 smithi087 ceph-mon[32276]: pgmap v592: 129 pgs: 129 active+clean; 478 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 440 op/s 2024-06-06T08:06:43.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.17 MB/sec execute 165 sec latency 5633.417 ms 2024-06-06T08:06:44.177 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.08 MB/sec execute 166 sec latency 6633.491 ms 2024-06-06T08:06:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:44 smithi195 ceph-mon[37040]: pgmap v593: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-06T08:06:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:44 smithi181 ceph-mon[36790]: pgmap v593: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-06T08:06:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:44 smithi087 ceph-mon[32276]: pgmap v593: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-06-06T08:06:45.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 14.00 MB/sec execute 167 sec latency 7633.611 ms 2024-06-06T08:06:46.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.91 MB/sec execute 168 sec latency 8633.651 ms 2024-06-06T08:06:46.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:46 smithi195 ceph-mon[37040]: pgmap v594: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-06-06T08:06:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:46 smithi181 ceph-mon[36790]: pgmap v594: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-06-06T08:06:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:46 smithi087 ceph-mon[32276]: pgmap v594: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-06-06T08:06:47.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.83 MB/sec execute 169 sec latency 9633.744 ms 2024-06-06T08:06:48.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.75 MB/sec execute 170 sec latency 10633.863 ms 2024-06-06T08:06:48.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:48 smithi195 ceph-mon[37040]: pgmap v595: 129 pgs: 129 active+clean; 486 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2024-06-06T08:06:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:48 smithi181 ceph-mon[36790]: pgmap v595: 129 pgs: 129 active+clean; 486 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2024-06-06T08:06:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:48 smithi087 ceph-mon[32276]: pgmap v595: 129 pgs: 129 active+clean; 486 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2024-06-06T08:06:49.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.67 MB/sec execute 171 sec latency 11633.977 ms 2024-06-06T08:06:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 -- 172.21.15.87:0/2674374153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974105850 msgr2=0x7f0974105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 --2- 172.21.15.87:0/2674374153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974105850 0x7f0974105c30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0970009900 tx=0x7f097002a410 comp rx=0 tx=0).stop 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 -- 172.21.15.87:0/2674374153 shutdown_connections 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 --2- 172.21.15.87:0/2674374153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 0x7f097410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 --2- 172.21.15.87:0/2674374153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974100bf0 0x7f0974101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 --2- 172.21.15.87:0/2674374153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974105850 0x7f0974105c30 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 -- 172.21.15.87:0/2674374153 >> 172.21.15.87:0/2674374153 conn(0x7f09740766a0 msgr2=0x7f0974076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 -- 172.21.15.87:0/2674374153 shutdown_connections 2024-06-06T08:06:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.422+0000 7f097bac4640 1 -- 172.21.15.87:0/2674374153 wait complete. 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 Processor -- start 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 -- start start 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 0x7f0974197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 0x7f097419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0974110c90 con 0x7f0974100bf0 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0974110b10 con 0x7f0974105850 2024-06-06T08:06:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f097bac4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0974110e10 con 0x7f09741015b0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f0979860640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f0979860640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48604/0 (socket says 172.21.15.87:48604) 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.423+0000 7f0979860640 1 -- 172.21.15.87:0/603052325 learned_addr learned my addr 172.21.15.87:0/603052325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f097905f640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 0x7f0974197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f097a061640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 0x7f097419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 -- 172.21.15.87:0/603052325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 msgr2=0x7f0974197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 0x7f0974197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 -- 172.21.15.87:0/603052325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 msgr2=0x7f097419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 0x7f097419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 -- 172.21.15.87:0/603052325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097419bff0 con 0x7f0974100bf0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f097a061640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 0x7f097419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f0979860640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f096affd640 1 -- 172.21.15.87:0/603052325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970035070 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f096affd640 1 -- 172.21.15.87:0/603052325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970031a20 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097419c280 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f096affd640 1 -- 172.21.15.87:0/603052325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970038ae0 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.424+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097406a1f0 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.426+0000 7f096affd640 1 -- 172.21.15.87:0/603052325 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0970031bd0 con 0x7f0974100bf0 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 Processor -- start 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 -- start start 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb52350 0x5571afb54740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f096affd640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 0x7f09600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5571afaab4f0 con 0x5571afb47170 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5571afa49870 con 0x5571afb52350 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.427+0000 7f097e7d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5571afaa74b0 con 0x5571afb47a90 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48620/0 (socket says 172.21.15.87:48620) 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 -- 172.21.15.87:0/1753501949 learned_addr learned my addr 172.21.15.87:0/1753501949 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 0x7f09600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f096affd640 1 -- 172.21.15.87:0/603052325 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f09700f1850 con 0x7f0974100bf0 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097905f640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb52350 0x5571afb54740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f0979860640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 -- 172.21.15.87:0/1753501949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 msgr2=0x5571afb51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 -- 172.21.15.87:0/1753501949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb52350 msgr2=0x5571afb54740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb52350 0x5571afb54740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 -- 172.21.15.87:0/1753501949 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571afa659b0 con 0x5571afb47170 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f0979860640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f096c014f30 tx=0x7f096c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c6cec007456215 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f097a061640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 0x7f09600a5260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f096c011d70 tx=0x7f096c01c000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f095ffff640 1 -- 172.21.15.87:0/1753501949 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c022070 con 0x5571afb47170 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f095ffff640 1 -- 172.21.15.87:0/1753501949 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c003e50 con 0x5571afb47170 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.428+0000 7f095ffff640 1 -- 172.21.15.87:0/1753501949 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c004680 con 0x5571afb47170 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 -- 172.21.15.87:0/1753501949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 msgr2=0x5571afb47550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f096c014f30 tx=0x7f096c011870 comp rx=0 tx=0).stop 2024-06-06T08:06:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 -- 172.21.15.87:0/1753501949 shutdown_connections 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb52350 0x5571afb54740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1753501949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47170 0x5571afb47550 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 -- 172.21.15.87:0/1753501949 >> 172.21.15.87:0/1753501949 conn(0x5571afb43680 msgr2=0x5571afb43a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 -- 172.21.15.87:0/1753501949 shutdown_connections 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.429+0000 7f097e7d8740 1 -- 172.21.15.87:0/1753501949 wait complete. 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 -- start start 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 Processor -- start 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb4f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb52350 0x5571afb4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5571afb54f20 con 0x5571afb47a90 2024-06-06T08:06:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5571afa34ae0 con 0x5571afb47170 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5571afb550a0 con 0x5571afb52350 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097e7d8740 10 client.0 _ll_register_callbacks cb 0x5571afa6d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48636/0 (socket says 172.21.15.87:48636) 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 -- 172.21.15.87:0/1358397405 learned_addr learned my addr 172.21.15.87:0/1358397405 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f0979860640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb4f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097a061640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb52350 0x5571afb4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 -- 172.21.15.87:0/1358397405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb52350 msgr2=0x5571afb4fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb52350 0x5571afb4fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 -- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 msgr2=0x5571afb4f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb4f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.430+0000 7f097905f640 1 -- 172.21.15.87:0/1358397405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571af7bf210 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097905f640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0964010820 tx=0x7f0964010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1a5a868ff46247 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096400fb10 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f096400fcb0 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c003e50 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f0964024050 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 10 client.18102 fetch_fsmap learned FSMap version 53 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5571af88b620 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c003e50 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f096400e5e0 con 0x5571afb47a90 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 10 client.18102 fetch_fsmap finished waiting for FSMap version 53 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 10 client.18102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 20 client.18102 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afb625a0 0x5571afb64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 4 client.18102 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.431+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571afabef10 con 0x5571afb625a0 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.432+0000 7f0979860640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afb625a0 0x5571afb64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.432+0000 7f0979860640 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afb625a0 0x5571afb64a10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee98f102bb8db919 server_cookie=9fc4857d3f203e34 in_seq=0 out_seq=0 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.432+0000 7f095d7fa640 10 client.18102 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f095d7fa640 1 -- 172.21.15.87:0/1358397405 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5571afabef10 con 0x5571afb625a0 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f095d7fa640 10 client.18102 handle_command_reply: tid=0 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 client.18102 shutdown 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 msgr2=0x5571afb4f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0964010820 tx=0x7f0964010ce0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 shutdown_connections 2024-06-06T08:06:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afb625a0 0x5571afb64a10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb52350 0x5571afb4fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb47a90 0x5571afb4f920 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 --2- 172.21.15.87:0/1358397405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb4f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f095d7fa640 0 client.18102 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 >> 172.21.15.87:0/1358397405 conn(0x5571afb43680 msgr2=0x5571afb493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 shutdown_connections 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 1 -- 172.21.15.87:0/1358397405 wait complete. 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.433+0000 7f097e7d8740 20 client.18102 trim_cache size 0 max 16384 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.448+0000 7f097e7d8740 1 Processor -- start 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 -- start start 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47170 0x5571afb42f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 0x5571afb6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09741af690 con 0x5571afb52350 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09741acf30 con 0x5571afb47a90 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097e7d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09741aada0 con 0x5571afb47170 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48642/0 (socket says 172.21.15.87:48642) 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 -- 172.21.15.87:0/3306785960 learned_addr learned my addr 172.21.15.87:0/3306785960 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f0979860640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47170 0x5571afb42f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097905f640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 0x5571afb6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 -- 172.21.15.87:0/3306785960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47170 msgr2=0x5571afb42f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47170 0x5571afb42f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 -- 172.21.15.87:0/3306785960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 msgr2=0x5571afb6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 0x5571afb6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 -- 172.21.15.87:0/3306785960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571afb5e180 con 0x5571afb52350 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097905f640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 0x5571afb6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f097a061640 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f096c009b00 tx=0x7f096c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324b05c56da4727a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f095f7fe640 1 -- 172.21.15.87:0/3306785960 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c022040 con 0x5571afb52350 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f095f7fe640 1 -- 172.21.15.87:0/3306785960 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c02b070 con 0x5571afb52350 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.449+0000 7f095f7fe640 1 -- 172.21.15.87:0/3306785960 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c005570 con 0x5571afb52350 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- 172.21.15.87:0/3306785960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 msgr2=0x5571afb6a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f096c009b00 tx=0x7f096c009fd0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- 172.21.15.87:0/3306785960 shutdown_connections 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afb6a820 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47a90 0x5571afb6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 --2- 172.21.15.87:0/3306785960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47170 0x5571afb42f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- 172.21.15.87:0/3306785960 >> 172.21.15.87:0/3306785960 conn(0x5571afb43680 msgr2=0x5571afb514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- 172.21.15.87:0/3306785960 shutdown_connections 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- 172.21.15.87:0/3306785960 wait complete. 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.450+0000 7f097e7d8740 1 -- start start 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 Processor -- start 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09741ab3f0 con 0x5571afb52350 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09741ac860 con 0x5571afb47170 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09741abac0 con 0x5571afb47a90 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097e7d8740 10 client.0 _ll_register_callbacks cb 0x7f096400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48658/0 (socket says 172.21.15.87:48658) 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 -- 172.21.15.87:0/482328293 learned_addr learned my addr 172.21.15.87:0/482328293 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097905f640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f0979860640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 -- 172.21.15.87:0/482328293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 msgr2=0x5571afb56d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb56d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 -- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 msgr2=0x5571afb567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.451+0000 7f097a061640 1 -- 172.21.15.87:0/482328293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571afbcc7f0 con 0x5571afb52350 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097a061640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f096c0118b0 tx=0x7f096c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9131a6cd60b7c061 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c022050 con 0x5571afb52350 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f096c0053f0 con 0x5571afb52350 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c02b070 con 0x5571afb52350 2024-06-06T08:06:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f096c0055f0 con 0x5571afb52350 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 10 client.18108 fetch_fsmap learned FSMap version 53 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5571afbcda70 con 0x5571afb52350 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c02b070 con 0x5571afb52350 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f096c01d070 con 0x5571afb52350 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 10 client.18108 fetch_fsmap finished waiting for FSMap version 53 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 10 client.18108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 20 client.18108 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afbcdf70 0x5571afbd0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 4 client.18108 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.452+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571afab8c20 con 0x5571afbcdf70 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.453+0000 7f0979860640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afbcdf70 0x5571afbd0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.453+0000 7f095d7fa640 10 client.18108 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.453+0000 7f0979860640 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afbcdf70 0x5571afbd0390 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5a201f9c7eba78 server_cookie=ff377d28ab1a00c7 in_seq=0 out_seq=0 2024-06-06T08:06:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f095d7fa640 1 -- 172.21.15.87:0/482328293 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5571afab8c20 con 0x5571afbcdf70 2024-06-06T08:06:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f095d7fa640 10 client.18108 handle_command_reply: tid=0 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 client.18108 shutdown 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 msgr2=0x5571afbcc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f096c0118b0 tx=0x7f096c004c30 comp rx=0 tx=0).stop 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 shutdown_connections 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5571afbcdf70 0x5571afbd0390 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5571afb52350 0x5571afbcc2b0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571afb47a90 0x5571afb56d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f095d7fa640 0 client.18108 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 --2- 172.21.15.87:0/482328293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5571afb47170 0x5571afb567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 >> 172.21.15.87:0/482328293 conn(0x5571afb43680 msgr2=0x5571afb48b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 shutdown_connections 2024-06-06T08:06:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.721+0000 7f097e7d8740 1 -- 172.21.15.87:0/482328293 wait complete. 2024-06-06T08:06:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097e7d8740 20 client.18108 trim_cache size 0 max 16384 2024-06-06T08:06:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:06:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:06:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 msgr2=0x7f09600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 0x7f09600a5260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f096c011d70 tx=0x7f096c01c000 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 msgr2=0x7f0974197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970003b90 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 shutdown_connections 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f09600a2da0 0x7f09600a5260 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0974105850 0x7f097419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741015b0 0x7f0974197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 --2- 172.21.15.87:0/603052325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0974100bf0 0x7f0974197050 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 >> 172.21.15.87:0/603052325 conn(0x7f09740766a0 msgr2=0x7f09740fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 shutdown_connections 2024-06-06T08:06:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:06:49.722+0000 7f097bac4640 1 -- 172.21.15.87:0/603052325 wait complete. 2024-06-06T08:06:50.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.59 MB/sec execute 172 sec latency 12634.017 ms 2024-06-06T08:06:50.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:50 smithi195 ceph-mon[37040]: pgmap v596: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-06-06T08:06:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:50 smithi181 ceph-mon[36790]: pgmap v596: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-06-06T08:06:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:50 smithi087 ceph-mon[32276]: pgmap v596: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-06-06T08:06:51.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.51 MB/sec execute 173 sec latency 13634.116 ms 2024-06-06T08:06:52.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.43 MB/sec execute 174 sec latency 14634.193 ms 2024-06-06T08:06:52.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:52 smithi195 ceph-mon[37040]: pgmap v597: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-06-06T08:06:52.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:52 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:52 smithi181 ceph-mon[36790]: pgmap v597: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-06-06T08:06:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:52 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:52 smithi087 ceph-mon[32276]: pgmap v597: 129 pgs: 129 active+clean; 484 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-06-06T08:06:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:52 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:53.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.36 MB/sec execute 175 sec latency 15634.327 ms 2024-06-06T08:06:54.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:06:54.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.28 MB/sec execute 176 sec latency 16634.387 ms 2024-06-06T08:06:54.293 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:54 smithi087 ceph-mon[32276]: pgmap v598: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-06-06T08:06:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:54 smithi087 ceph-mon[32276]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:54 smithi195 ceph-mon[37040]: pgmap v598: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-06-06T08:06:54.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:54 smithi195 ceph-mon[37040]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:54.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:54 smithi181 ceph-mon[36790]: pgmap v598: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-06-06T08:06:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:54 smithi181 ceph-mon[36790]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:06:55.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.21 MB/sec execute 177 sec latency 17634.414 ms 2024-06-06T08:06:56.178 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.13 MB/sec execute 178 sec latency 18634.451 ms 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.304+0000 7f04c1f7d700 1 -- 172.21.15.87:0/1602419941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc102fd0 msgr2=0x7f04bc107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.304+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/1602419941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc102fd0 0x7f04bc107020 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f04a80099c0 tx=0x7f04a8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.304+0000 7f04c1f7d700 1 -- 172.21.15.87:0/1602419941 shutdown_connections 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.304+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/1602419941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc107700 0x7f04bc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.304+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/1602419941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102620 0x7f04bc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/1602419941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc102fd0 0x7f04bc107020 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- 172.21.15.87:0/1602419941 >> 172.21.15.87:0/1602419941 conn(0x7f04bc0fdf10 msgr2=0x7f04bc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- 172.21.15.87:0/1602419941 shutdown_connections 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- 172.21.15.87:0/1602419941 wait complete. 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 Processor -- start 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- start start 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 0x7f04bc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 0x7f04bc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc107700 0x7f04bc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1126c0 con 0x7f04bc107700 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc112540 con 0x7f04bc102620 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.305+0000 7f04c1f7d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc112840 con 0x7f04bc102fd0 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04bbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 0x7f04bc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 0x7f04bc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:56.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04bbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 0x7f04bc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:33224/0 (socket says 172.21.15.87:33224) 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04bbfff700 1 -- 172.21.15.87:0/2743236859 learned_addr learned my addr 172.21.15.87:0/2743236859 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c177c700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc107700 0x7f04bc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 -- 172.21.15.87:0/2743236859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 msgr2=0x7f04bc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 0x7f04bc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 -- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc107700 msgr2=0x7f04bc198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc107700 0x7f04bc198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 -- 172.21.15.87:0/2743236859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc199160 con 0x7f04bc102620 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c0f7b700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 0x7f04bc19e7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c970 tx=0x7f04b000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0004a30 con 0x7f04bc102620 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b0005500 con 0x7f04bc102620 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc199450 con 0x7f04bc102620 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.306+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000ed20 con 0x7f04bc102620 2024-06-06T08:06:56.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.307+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1a56c0 con 0x7f04bc102620 2024-06-06T08:06:56.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.307+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc069180 con 0x7f04bc102620 2024-06-06T08:06:56.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.308+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f04b0004d60 con 0x7f04bc102620 2024-06-06T08:06:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.309+0000 7f04b9ffb700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 0x7f04ac0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.309+0000 7f04bbfff700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 0x7f04ac0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.309+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f04b00cf640 con 0x7f04bc102620 2024-06-06T08:06:56.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.309+0000 7f04bbfff700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 0x7f04ac0a52d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04bc101600 tx=0x7f04a8005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:56.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.313+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04b0014030 con 0x7f04bc102620 2024-06-06T08:06:56.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:56 smithi195 ceph-mon[37040]: pgmap v599: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-06-06T08:06:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:56 smithi181 ceph-mon[36790]: pgmap v599: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-06-06T08:06:56.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.560+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04bc199ee0 con 0x7f04bc102620 2024-06-06T08:06:56.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:56 smithi087 ceph-mon[32276]: pgmap v599: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-06-06T08:06:56.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.561+0000 7f04b9ffb700 1 -- 172.21.15.87:0/2743236859 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7f04b009bec0 con 0x7f04bc102620 2024-06-06T08:06:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:06:56.561 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:06:56.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 msgr2=0x7f04ac0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 0x7f04ac0a52d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04bc101600 tx=0x7f04a8005fb0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 msgr2=0x7f04bc19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 0x7f04bc19e7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c970 tx=0x7f04b000cc80 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 shutdown_connections 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04bc102620 0x7f04bc19e7f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f04ac0a2e10 0x7f04ac0a52d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04bc107700 0x7f04bc198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 --2- 172.21.15.87:0/2743236859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102fd0 0x7f04bc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 >> 172.21.15.87:0/2743236859 conn(0x7f04bc0fdf10 msgr2=0x7f04bc100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 shutdown_connections 2024-06-06T08:06:56.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:56.564+0000 7f04c1f7d700 1 -- 172.21.15.87:0/2743236859 wait complete. 2024-06-06T08:06:56.565 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:06:57.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 13.06 MB/sec execute 179 sec latency 19634.541 ms 2024-06-06T08:06:57.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id d85bb652 --release --await' 2024-06-06T08:06:57.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id d85bb652 --release --await 2024-06-06T08:06:57.450 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:06:57.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:57 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2743236859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:57.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:57 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2743236859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:57.572 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:57 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2743236859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:06:58.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 12.99 MB/sec execute 180 sec latency 20634.591 ms 2024-06-06T08:06:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:06:58 smithi195 ceph-mon[37040]: pgmap v600: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T08:06:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:06:58 smithi181 ceph-mon[36790]: pgmap v600: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T08:06:58.599 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:06:58 smithi087 ceph-mon[32276]: pgmap v600: 129 pgs: 129 active+clean; 482 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-06-06T08:06:59.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69358 12.91 MB/sec execute 181 sec latency 21634.697 ms 2024-06-06T08:06:59.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.339+0000 7f822ac78700 1 -- 172.21.15.87:0/979422850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224106750 msgr2=0x7f822410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.339+0000 7f822ac78700 1 --2- 172.21.15.87:0/979422850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224106750 0x7f822410e2d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f821c00b0d0 tx=0x7f821c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- 172.21.15.87:0/979422850 shutdown_connections 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- 172.21.15.87:0/979422850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224106750 0x7f822410e2d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- 172.21.15.87:0/979422850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8224101df0 0x7f82241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- 172.21.15.87:0/979422850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82241027a0 0x7f8224106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- 172.21.15.87:0/979422850 >> 172.21.15.87:0/979422850 conn(0x7f8224077a00 msgr2=0x7f82240fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- 172.21.15.87:0/979422850 shutdown_connections 2024-06-06T08:06:59.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- 172.21.15.87:0/979422850 wait complete. 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 Processor -- start 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- start start 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82241027a0 0x7f8224199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8224106750 0x7f82241a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8224111690 con 0x7f8224101df0 2024-06-06T08:06:59.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8224111510 con 0x7f8224106750 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.340+0000 7f822ac78700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8224111810 con 0x7f82241027a0 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54896/0 (socket says 172.21.15.87:54896) 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 -- 172.21.15.87:0/2419954911 learned_addr learned my addr 172.21.15.87:0/2419954911 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8223fff700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82241027a0 0x7f8224199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8229215700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8224106750 0x7f82241a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 -- 172.21.15.87:0/2419954911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82241027a0 msgr2=0x7f8224199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82241027a0 0x7f8224199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 -- 172.21.15.87:0/2419954911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8224106750 msgr2=0x7f82241a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8224106750 0x7f82241a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 -- 172.21.15.87:0/2419954911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241a4f70 con 0x7f8224101df0 2024-06-06T08:06:59.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8228a14700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9c0 tx=0x7f821800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8221ffb700 1 -- 172.21.15.87:0/2419954911 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800cbd0 con 0x7f8224101df0 2024-06-06T08:06:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8224197800 con 0x7f8224101df0 2024-06-06T08:06:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8221ffb700 1 -- 172.21.15.87:0/2419954911 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821800ee60 con 0x7f8224101df0 2024-06-06T08:06:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.341+0000 7f8221ffb700 1 -- 172.21.15.87:0/2419954911 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218005630 con 0x7f8224101df0 2024-06-06T08:06:59.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.342+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8224197d40 con 0x7f8224101df0 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.343+0000 7f8221ffb700 1 -- 172.21.15.87:0/2419954911 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8218004750 con 0x7f8224101df0 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.344+0000 7f8221ffb700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 0x7f82140a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.344+0000 7f8223fff700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 0x7f82140a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.344+0000 7f8221ffb700 1 -- 172.21.15.87:0/2419954911 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f82180cfb10 con 0x7f8224101df0 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f8223fff700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 0x7f82140a5250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82240ffe10 tx=0x7f821000a300 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 Processor -- start 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 -- start start 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 0x555f7e2f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555f7e1ea060 con 0x555f7e2fd500 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7e248950 con 0x555f7e2fde20 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f82354fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f7e1b0560 con 0x555f7e2f7c50 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54904/0 (socket says 172.21.15.87:54904) 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.345+0000 7f8228a14700 1 -- 172.21.15.87:0/3527195549 learned_addr learned my addr 172.21.15.87:0/3527195549 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8229215700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8223fff700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 0x555f7e2f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 -- 172.21.15.87:0/3527195549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 msgr2=0x555f7e2f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 -- 172.21.15.87:0/3527195549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 msgr2=0x555f7e2f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 0x555f7e2f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 -- 172.21.15.87:0/3527195549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7e1979d0 con 0x555f7e2fd500 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8229215700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8223fff700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 0x555f7e2f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f8228a14700 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f821809bba0 tx=0x7f821809b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8097295a85de99c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f820effd700 1 -- 172.21.15.87:0/3527195549 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180a4880 con 0x555f7e2fd500 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f820effd700 1 -- 172.21.15.87:0/3527195549 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218099f10 con 0x555f7e2fd500 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.346+0000 7f820effd700 1 -- 172.21.15.87:0/3527195549 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180a4650 con 0x555f7e2fd500 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- 172.21.15.87:0/3527195549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 msgr2=0x555f7e2fd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f821809bba0 tx=0x7f821809b660 comp rx=0 tx=0).stop 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- 172.21.15.87:0/3527195549 shutdown_connections 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fd500 0x555f7e2fd8e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 --2- 172.21.15.87:0/3527195549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fde20 0x555f7e2f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- 172.21.15.87:0/3527195549 >> 172.21.15.87:0/3527195549 conn(0x555f7e224180 msgr2=0x555f7e2f2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- 172.21.15.87:0/3527195549 shutdown_connections 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- 172.21.15.87:0/3527195549 wait complete. 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 -- start start 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.347+0000 7f82354fbb80 1 Processor -- start 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e300e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e301380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x555f7e1e90c0 con 0x555f7e2f7c50 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7e244910 con 0x555f7e2fd500 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f7e304800 con 0x555f7e2fde20 2024-06-06T08:06:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f82354fbb80 10 client.0 _ll_register_callbacks cb 0x555f7e1de6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54914/0 (socket says 172.21.15.87:54914) 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 -- 172.21.15.87:0/1960550097 learned_addr learned my addr 172.21.15.87:0/1960550097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8223fff700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e300e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8229215700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e301380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 -- 172.21.15.87:0/1960550097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 msgr2=0x555f7e301380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e301380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 -- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 msgr2=0x555f7e300e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e300e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 -- 172.21.15.87:0/1960550097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7e20e850 con 0x555f7e2f7c50 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8228a14700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f82180cd440 tx=0x7f821809c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3bc0695cbe58db server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8203fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.348+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180ba050 con 0x555f7e2f7c50 2024-06-06T08:06:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f82180a4600 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218099f10 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f82180a4880 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 10 client.18126 fetch_fsmap learned FSMap version 55 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555f7e30eea0 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218099f10 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f821809aad0 con 0x555f7e2f7c50 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 10 client.18126 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 10 client.18126 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 20 client.18126 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e311370 0x555f7e313760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 4 client.18126 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7e255e50 con 0x555f7e311370 2024-06-06T08:06:59.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.349+0000 7f8223fff700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e311370 0x555f7e313760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.350+0000 7f8203fff700 10 client.18126 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.350+0000 7f8223fff700 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e311370 0x555f7e313760 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b14a6dd255d5b06 server_cookie=b39b246c4e7e90b in_seq=0 out_seq=0 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.350+0000 7f8203fff700 1 -- 172.21.15.87:0/1960550097 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f7e255e50 con 0x555f7e311370 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.350+0000 7f8203fff700 10 client.18126 handle_command_reply: tid=0 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.350+0000 7f82354fbb80 1 client.18126 shutdown 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 msgr2=0x555f7e303b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f82180cd440 tx=0x7f821809c120 comp rx=0 tx=0).stop 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 shutdown_connections 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e311370 0x555f7e313760 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e301380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e303b40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 --2- 172.21.15.87:0/1960550097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e300e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 >> 172.21.15.87:0/1960550097 conn(0x555f7e224180 msgr2=0x555f7e2f1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f8203fff700 0 client.18126 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:59.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 shutdown_connections 2024-06-06T08:06:59.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 1 -- 172.21.15.87:0/1960550097 wait complete. 2024-06-06T08:06:59.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.351+0000 7f82354fbb80 20 client.18126 trim_cache size 0 max 16384 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.370+0000 7f82354fbb80 1 Processor -- start 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.370+0000 7f82354fbb80 1 -- start start 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e2f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82241b3200 con 0x555f7e2fde20 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82241b0a90 con 0x555f7e2fd500 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f82354fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82241ae900 con 0x555f7e2f7c50 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54918/0 (socket says 172.21.15.87:54918) 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 -- 172.21.15.87:0/2458254898 learned_addr learned my addr 172.21.15.87:0/2458254898 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 -- 172.21.15.87:0/2458254898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 msgr2=0x555f7e2f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8223fff700 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e2f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 -- 172.21.15.87:0/2458254898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 msgr2=0x555f7e2f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e2f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 -- 172.21.15.87:0/2458254898 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7e304320 con 0x555f7e2fde20 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8223fff700 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e2f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:06:59.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f8229215700 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f821c009520 tx=0x7f821c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754014abaf1bbe3c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f820e7fc700 1 -- 172.21.15.87:0/2458254898 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00bd20 con 0x555f7e2fde20 2024-06-06T08:06:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f820e7fc700 1 -- 172.21.15.87:0/2458254898 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c021070 con 0x555f7e2fde20 2024-06-06T08:06:59.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.371+0000 7f820e7fc700 1 -- 172.21.15.87:0/2458254898 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0050c0 con 0x555f7e2fde20 2024-06-06T08:06:59.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- 172.21.15.87:0/2458254898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 msgr2=0x555f7e3163a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f821c009520 tx=0x7f821c009550 comp rx=0 tx=0).stop 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- 172.21.15.87:0/2458254898 shutdown_connections 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2fde20 0x555f7e3163a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2f7c50 0x555f7e2f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 --2- 172.21.15.87:0/2458254898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e2f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- 172.21.15.87:0/2458254898 >> 172.21.15.87:0/2458254898 conn(0x555f7e224180 msgr2=0x555f7e302f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- 172.21.15.87:0/2458254898 shutdown_connections 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- 172.21.15.87:0/2458254898 wait complete. 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 -- start start 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.372+0000 7f82354fbb80 1 Processor -- start 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e314670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e314bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82241aef50 con 0x555f7e2f7c50 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82241b03c0 con 0x555f7e2fd500 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82241af620 con 0x555f7e2fde20 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54922/0 (socket says 172.21.15.87:54922) 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f82354fbb80 10 client.0 _ll_register_callbacks cb 0x555f7e1de6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 -- 172.21.15.87:0/4142818085 learned_addr learned my addr 172.21.15.87:0/4142818085 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8223fff700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e314670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 -- 172.21.15.87:0/4142818085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 msgr2=0x555f7e314bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e314bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 -- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 msgr2=0x555f7e314670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e314670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 -- 172.21.15.87:0/4142818085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7e37acb0 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8228a14700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f821809b910 tx=0x7f821809bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9cd62852efbfe1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.373+0000 7f8203fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218097eb0 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9583+0+0 (secure 0 0 0) 0x7f821809aad0 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c021070 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f82180ba850 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 10 client.18132 fetch_fsmap learned FSMap version 55 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555f7e37bf30 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c021070 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f821809b060 con 0x555f7e2f7c50 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 10 client.18132 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 10 client.18132 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:06:59.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 20 client.18132 populate_metadata read hostname 'smithi087' 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e37c440 0x555f7e37e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 4 client.18132 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d85bb652", "release": true, "await": true}] 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.374+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d85bb652", "release": true, "await": true}) -- 0x555f7e2503d0 con 0x555f7e37c440 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.375+0000 7f8223fff700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e37c440 0x555f7e37e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.375+0000 7f8223fff700 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e37c440 0x555f7e37e860 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc00ebecd126b84 server_cookie=a0ec3384e2a6640e in_seq=0 out_seq=0 2024-06-06T08:06:59.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.375+0000 7f8203fff700 10 client.18132 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:59.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.471+0000 7f8203fff700 1 -- 172.21.15.87:0/4142818085 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f7e2503d0 con 0x555f7e37c440 2024-06-06T08:06:59.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.471+0000 7f8203fff700 10 client.18132 handle_command_reply: tid=0 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.471+0000 7f82354fbb80 1 client.18132 shutdown 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.471+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 msgr2=0x555f7e314130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.471+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f821809b910 tx=0x7f821809bb10 comp rx=0 tx=0).stop 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 shutdown_connections 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x555f7e37c440 0x555f7e37e860 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f7e2fde20 0x555f7e314bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f7e2f7c50 0x555f7e314130 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 --2- 172.21.15.87:0/4142818085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7e2fd500 0x555f7e314670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 >> 172.21.15.87:0/4142818085 conn(0x555f7e224180 msgr2=0x555f7e2fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f8203fff700 0 client.18132 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 shutdown_connections 2024-06-06T08:06:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 1 -- 172.21.15.87:0/4142818085 wait complete. 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f82354fbb80 20 client.18132 trim_cache size 0 max 16384 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 60, 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "d85bb652": { 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 60, 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:06:59.473 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 143.3, 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:59.474 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 msgr2=0x7f82140a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 0x7f82140a5250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82240ffe10 tx=0x7f821000a300 comp rx=0 tx=0).stop 2024-06-06T08:06:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 msgr2=0x7f82241999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.472+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9c0 tx=0x7f821800ecd0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 shutdown_connections 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8224106750 0x7f82241a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f82140a2d90 0x7f82140a5250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8224101df0 0x7f82241999b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 --2- 172.21.15.87:0/2419954911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82241027a0 0x7f8224199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 >> 172.21.15.87:0/2419954911 conn(0x7f8224077a00 msgr2=0x7f8224104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 shutdown_connections 2024-06-06T08:06:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:06:59.473+0000 7f822ac78700 1 -- 172.21.15.87:0/2419954911 wait complete. 2024-06-06T08:07:00.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69603 12.92 MB/sec execute 182 sec latency 21916.783 ms 2024-06-06T08:07:00.347 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd85bb652', total seconds elapsed: 29.6 2024-06-06T08:07:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:00 smithi181 ceph-mon[36790]: pgmap v601: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 190 op/s 2024-06-06T08:07:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:00 smithi087 ceph-mon[32276]: pgmap v601: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 190 op/s 2024-06-06T08:07:00.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:00 smithi195 ceph-mon[37040]: pgmap v601: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 190 op/s 2024-06-06T08:07:01.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 69893 12.96 MB/sec execute 183 sec latency 327.120 ms 2024-06-06T08:07:02.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70032 12.92 MB/sec execute 184 sec latency 367.026 ms 2024-06-06T08:07:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:02 smithi181 ceph-mon[36790]: pgmap v602: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-06-06T08:07:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:02 smithi181 ceph-mon[36790]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T08:07:02.561 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:02 smithi195 ceph-mon[37040]: pgmap v602: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-06-06T08:07:02.561 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:02 smithi195 ceph-mon[37040]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T08:07:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:02 smithi087 ceph-mon[32276]: pgmap v602: 129 pgs: 129 active+clean; 479 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-06-06T08:07:02.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:02 smithi087 ceph-mon[32276]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T08:07:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.860+0000 7fd970dfb640 1 -- 172.21.15.87:0/3820022084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c101290 msgr2=0x7fd96c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.860+0000 7fd970dfb640 1 --2- 172.21.15.87:0/3820022084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c101290 0x7fd96c10d750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd96000fb10 tx=0x7fd960032990 comp rx=0 tx=0).stop 2024-06-06T08:07:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.861+0000 7fd970dfb640 1 -- 172.21.15.87:0/3820022084 shutdown_connections 2024-06-06T08:07:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.861+0000 7fd970dfb640 1 --2- 172.21.15.87:0/3820022084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c101290 0x7fd96c10d750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.861+0000 7fd970dfb640 1 --2- 172.21.15.87:0/3820022084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd96c1008d0 0x7fd96c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.861+0000 7fd970dfb640 1 --2- 172.21.15.87:0/3820022084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c106890 0x7fd96c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.861+0000 7fd970dfb640 1 -- 172.21.15.87:0/3820022084 >> 172.21.15.87:0/3820022084 conn(0x7fd96c077a20 msgr2=0x7fd96c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.865+0000 7fd970dfb640 1 -- 172.21.15.87:0/3820022084 shutdown_connections 2024-06-06T08:07:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.868+0000 7fd970dfb640 1 -- 172.21.15.87:0/3820022084 wait complete. 2024-06-06T08:07:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 Processor -- start 2024-06-06T08:07:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 -- start start 2024-06-06T08:07:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd96c1008d0 0x7fd96c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c106890 0x7fd96c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06bdf0 con 0x7fd96c101290 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06bc70 con 0x7fd96c106890 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd970dfb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06bf70 con 0x7fd96c1008d0 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd969d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd969d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54934/0 (socket says 172.21.15.87:54934) 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.869+0000 7fd969d9b640 1 -- 172.21.15.87:0/1191099388 learned_addr learned my addr 172.21.15.87:0/1191099388 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.871+0000 7fd969d9b640 1 -- 172.21.15.87:0/1191099388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd96c1008d0 msgr2=0x7fd96c0731c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.871+0000 7fd969d9b640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd96c1008d0 0x7fd96c0731c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.871+0000 7fd969d9b640 1 -- 172.21.15.87:0/1191099388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c106890 msgr2=0x7fd96c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.871+0000 7fd969d9b640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c106890 0x7fd96c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.871+0000 7fd969d9b640 1 -- 172.21.15.87:0/1191099388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c06ddb0 con 0x7fd96c101290 2024-06-06T08:07:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.872+0000 7fd969d9b640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00bfd0 tx=0x7fd95c00ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.879+0000 7fd95b7fe640 1 -- 172.21.15.87:0/1191099388 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c019070 con 0x7fd96c101290 2024-06-06T08:07:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.879+0000 7fd95b7fe640 1 -- 172.21.15.87:0/1191099388 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95c014410 con 0x7fd96c101290 2024-06-06T08:07:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.879+0000 7fd95b7fe640 1 -- 172.21.15.87:0/1191099388 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c013610 con 0x7fd96c101290 2024-06-06T08:07:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.879+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c06e0a0 con 0x7fd96c101290 2024-06-06T08:07:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.879+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c1bc1b0 con 0x7fd96c101290 2024-06-06T08:07:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.884+0000 7fd973bdf740 1 Processor -- start 2024-06-06T08:07:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 -- start start 2024-06-06T08:07:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c86e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018c89740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556018be04f0 con 0x556018c7ca90 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556018b7e870 con 0x556018c87350 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.886+0000 7fd973bdf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556018bdc4b0 con 0x556018c7c170 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.887+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.887+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:38514/0 (socket says 172.21.15.87:38514) 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.887+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2763834896 learned_addr learned my addr 172.21.15.87:0/2763834896 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.889+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2763834896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 msgr2=0x556018c89740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.889+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018c89740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.889+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2763834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 msgr2=0x556018c86e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.889+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c86e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.889+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2763834896 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556018b9a9b0 con 0x556018c7c170 2024-06-06T08:07:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.891+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd960032eb0 tx=0x7fd9600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19611fb3094c1e6d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.892+0000 7fd958ff9640 1 -- 172.21.15.87:0/2763834896 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96003b030 con 0x556018c7c170 2024-06-06T08:07:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.893+0000 7fd958ff9640 1 -- 172.21.15.87:0/2763834896 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd960017070 con 0x556018c7c170 2024-06-06T08:07:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.893+0000 7fd95b7fe640 1 -- 172.21.15.87:0/1191099388 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd95c007a70 con 0x7fd96c101290 2024-06-06T08:07:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.894+0000 7fd973bdf740 1 -- 172.21.15.87:0/2763834896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 msgr2=0x556018c7c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.894+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd960032eb0 tx=0x7fd9600046c0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.894+0000 7fd95b7fe640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 0x7fd9540a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.894+0000 7fd95b7fe640 1 -- 172.21.15.87:0/1191099388 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9671+0+0 (secure 0 0 0) 0x7fd95c0ceae0 con 0x7fd96c101290 2024-06-06T08:07:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 0x7fd9540a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd973bdf740 1 -- 172.21.15.87:0/2763834896 shutdown_connections 2024-06-06T08:07:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018c89740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2763834896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c7c550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.895+0000 7fd973bdf740 1 -- 172.21.15.87:0/2763834896 >> 172.21.15.87:0/2763834896 conn(0x556018c78680 msgr2=0x556018c78a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.897+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 0x7fd9540a5440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd960018800 tx=0x7fd960004400 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.898+0000 7fd973bdf740 1 -- 172.21.15.87:0/2763834896 shutdown_connections 2024-06-06T08:07:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.901+0000 7fd973bdf740 1 -- 172.21.15.87:0/2763834896 wait complete. 2024-06-06T08:07:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.901+0000 7fd973bdf740 1 -- start start 2024-06-06T08:07:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.901+0000 7fd973bdf740 1 Processor -- start 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c84170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7ca90 0x556018c846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556018c89ec0 con 0x556018c87350 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556018b69ae0 con 0x556018c7ca90 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556018c8a040 con 0x556018c7c170 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd973bdf740 10 client.0 _ll_register_callbacks cb 0x556018ba29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54944/0 (socket says 172.21.15.87:54944) 2024-06-06T08:07:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.904+0000 7fd96ad9d640 1 -- 172.21.15.87:0/3450938188 learned_addr learned my addr 172.21.15.87:0/3450938188 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96a59c640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 -- 172.21.15.87:0/3450938188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 msgr2=0x556018c84170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c84170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 -- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7ca90 msgr2=0x556018c846b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7ca90 0x556018c846b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 -- 172.21.15.87:0/3450938188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560188f4210 con 0x556018c87350 2024-06-06T08:07:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.905+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd960005940 tx=0x7fd960004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea33a1481ea07c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.906+0000 7fd951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.906+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96003b040 con 0x556018c87350 2024-06-06T08:07:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.906+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9671+0+0 (secure 0 0 0) 0x7fd960003cc0 con 0x556018c87350 2024-06-06T08:07:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.908+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd960017070 con 0x556018c87350 2024-06-06T08:07:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.908+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd960003ec0 con 0x556018c87350 2024-06-06T08:07:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.908+0000 7fd973bdf740 10 client.18141 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.908+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5560189c0620 con 0x556018c87350 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96004c960 con 0x556018c87350 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd9600406f0 con 0x556018c87350 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 10 client.18141 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 10 client.18141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 20 client.18141 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018c97340 0x556018c99780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 4 client.18141 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.909+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd960017070 con 0x556018c97340 2024-06-06T08:07:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.911+0000 7fd96a59c640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018c97340 0x556018c99780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.924+0000 7fd96a59c640 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018c97340 0x556018c99780 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbc17792e65790 server_cookie=85a53b4eaf1ea4d2 in_seq=0 out_seq=0 2024-06-06T08:07:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.924+0000 7fd951ffb640 10 client.18141 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.940+0000 7fd951ffb640 1 -- 172.21.15.87:0/3450938188 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd960017070 con 0x556018c97340 2024-06-06T08:07:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.940+0000 7fd951ffb640 10 client.18141 handle_command_reply: tid=0 2024-06-06T08:07:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.940+0000 7fd973bdf740 1 client.18141 shutdown 2024-06-06T08:07:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.944+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 msgr2=0x556018c84bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.944+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd960005940 tx=0x7fd960004190 comp rx=0 tx=0).stop 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 shutdown_connections 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018c97340 0x556018c99780 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c87350 0x556018c84bf0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7ca90 0x556018c846b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 --2- 172.21.15.87:0/3450938188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c84170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 >> 172.21.15.87:0/3450938188 conn(0x556018c78680 msgr2=0x556018c7e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.946+0000 7fd951ffb640 0 client.18141 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.949+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 shutdown_connections 2024-06-06T08:07:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.950+0000 7fd973bdf740 1 -- 172.21.15.87:0/3450938188 wait complete. 2024-06-06T08:07:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.950+0000 7fd973bdf740 20 client.18141 trim_cache size 0 max 16384 2024-06-06T08:07:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.983+0000 7fd973bdf740 1 Processor -- start 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 -- start start 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 0x556018c7eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c87350 0x556018c9f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c012c50 con 0x556018c7ca90 2024-06-06T08:07:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c010520 con 0x556018c7c170 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd973bdf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c00e390 con 0x556018c87350 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.984+0000 7fd96a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 0x556018c7eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd96a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 0x556018c7eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:39962/0 (socket says 172.21.15.87:39962) 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd96a59c640 1 -- 172.21.15.87:0/2377748879 learned_addr learned my addr 172.21.15.87:0/2377748879 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c87350 0x556018c9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 -- 172.21.15.87:0/2377748879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c87350 msgr2=0x556018c9f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c87350 0x556018c9f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 -- 172.21.15.87:0/2377748879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 msgr2=0x556018c7eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 0x556018c7eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 -- 172.21.15.87:0/2377748879 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556018c7efe0 con 0x556018c7ca90 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.985+0000 7fd969d9b640 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c84a10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0cbd50 tx=0x7fd95c09a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964645ef6625eb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.986+0000 7fd953fff640 1 -- 172.21.15.87:0/2377748879 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c09eed0 con 0x556018c7ca90 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.986+0000 7fd953fff640 1 -- 172.21.15.87:0/2377748879 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95c09be50 con 0x556018c7ca90 2024-06-06T08:07:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.986+0000 7fd953fff640 1 -- 172.21.15.87:0/2377748879 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c099f70 con 0x556018c7ca90 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 -- 172.21.15.87:0/2377748879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 msgr2=0x556018c84a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c84a10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0cbd50 tx=0x7fd95c09a6e0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 -- 172.21.15.87:0/2377748879 shutdown_connections 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c87350 0x556018c9f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c84a10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2377748879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c7c170 0x556018c7eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.987+0000 7fd973bdf740 1 -- 172.21.15.87:0/2377748879 >> 172.21.15.87:0/2377748879 conn(0x556018c78680 msgr2=0x556018c86230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.991+0000 7fd973bdf740 1 -- 172.21.15.87:0/2377748879 shutdown_connections 2024-06-06T08:07:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.995+0000 7fd973bdf740 1 -- 172.21.15.87:0/2377748879 wait complete. 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.995+0000 7fd973bdf740 1 -- start start 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.995+0000 7fd973bdf740 1 Processor -- start 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.995+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c8a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c8ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c00e9e0 con 0x556018c7ca90 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c00fe50 con 0x556018c87350 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c00f0b0 con 0x556018c7c170 2024-06-06T08:07:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.996+0000 7fd973bdf740 10 client.0 _ll_register_callbacks cb 0x7fd960005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.997+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.997+0000 7fd96ad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:39972/0 (socket says 172.21.15.87:39972) 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.997+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2801313889 learned_addr learned my addr 172.21.15.87:0/2801313889 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.997+0000 7fd96a59c640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c8a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.998+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2801313889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 msgr2=0x556018c8a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.998+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c8a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.998+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2801313889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 msgr2=0x556018c8ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.998+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c8ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:02.998+0000 7fd96ad9d640 1 -- 172.21.15.87:0/2801313889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556018d00580 con 0x556018c87350 2024-06-06T08:07:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.000+0000 7fd96ad9d640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd960032eb0 tx=0x7fd960004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fce4efa440f7360 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.000+0000 7fd951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:07:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.005+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96003b040 con 0x556018c87350 2024-06-06T08:07:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.005+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd95c09be50 con 0x556018c87350 2024-06-06T08:07:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.005+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9671+0+0 (secure 0 0 0) 0x7fd960040870 con 0x556018c87350 2024-06-06T08:07:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.007+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd96000f990 con 0x556018c87350 2024-06-06T08:07:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.012+0000 7fd973bdf740 10 client.27086 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.012+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556018d01870 con 0x556018c87350 2024-06-06T08:07:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.014+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96006e070 con 0x556018c87350 2024-06-06T08:07:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.014+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd960061420 con 0x556018c87350 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 10 client.27086 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 10 client.27086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 20 client.27086 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018d01f30 0x556018d04350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 4 client.27086 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:07:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.016+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556018bf3f10 con 0x556018d01f30 2024-06-06T08:07:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.025+0000 7fd96a59c640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018d01f30 0x556018d04350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.029+0000 7fd96a59c640 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018d01f30 0x556018d04350 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cc14197348eae8 server_cookie=198413b2f642a691 in_seq=0 out_seq=0 2024-06-06T08:07:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.029+0000 7fd951ffb640 10 client.27086 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:03.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70129 12.87 MB/sec execute 185 sec latency 882.395 ms 2024-06-06T08:07:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.647+0000 7fd951ffb640 1 -- 172.21.15.87:0/2801313889 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556018bf3f10 con 0x556018d01f30 2024-06-06T08:07:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd951ffb640 10 client.27086 handle_command_reply: tid=0 2024-06-06T08:07:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 client.27086 shutdown 2024-06-06T08:07:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 msgr2=0x556018d00010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd960032eb0 tx=0x7fd960004390 comp rx=0 tx=0).stop 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 shutdown_connections 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556018d01f30 0x556018d04350 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556018c87350 0x556018d00010 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556018c7ca90 0x556018c8ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 --2- 172.21.15.87:0/2801313889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556018c7c170 0x556018c8a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 >> 172.21.15.87:0/2801313889 conn(0x556018c78680 msgr2=0x556018c76f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd951ffb640 0 client.27086 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 shutdown_connections 2024-06-06T08:07:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.648+0000 7fd973bdf740 1 -- 172.21.15.87:0/2801313889 wait complete. 2024-06-06T08:07:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.649+0000 7fd973bdf740 20 client.27086 trim_cache size 0 max 16384 2024-06-06T08:07:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:07:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:07:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.649+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 msgr2=0x7fd9540a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.649+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 0x7fd9540a5440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd960018800 tx=0x7fd960004400 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 msgr2=0x7fd96c06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00bfd0 tx=0x7fd95c00ae10 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 shutdown_connections 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd9540a2f80 0x7fd9540a5440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd96c106890 0x7fd96c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c101290 0x7fd96c06d280 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 --2- 172.21.15.87:0/1191099388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd96c1008d0 0x7fd96c0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 >> 172.21.15.87:0/1191099388 conn(0x7fd96c077a20 msgr2=0x7fd96c1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 shutdown_connections 2024-06-06T08:07:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:03.650+0000 7fd970dfb640 1 -- 172.21.15.87:0/1191099388 wait complete. 2024-06-06T08:07:04.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70195 12.83 MB/sec execute 186 sec latency 1075.380 ms 2024-06-06T08:07:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:04 smithi181 ceph-mon[36790]: pgmap v604: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:04 smithi087 ceph-mon[32276]: pgmap v604: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:04.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:04 smithi195 ceph-mon[37040]: pgmap v604: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:05.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70302 12.80 MB/sec execute 187 sec latency 1412.046 ms 2024-06-06T08:07:06.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70466 12.75 MB/sec execute 188 sec latency 1060.887 ms 2024-06-06T08:07:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:06 smithi181 ceph-mon[36790]: pgmap v605: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:06 smithi087 ceph-mon[32276]: pgmap v605: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:06.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:06 smithi195 ceph-mon[37040]: pgmap v605: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 481 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-06-06T08:07:07.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70679 12.76 MB/sec execute 189 sec latency 971.101 ms 2024-06-06T08:07:08.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 70889 12.78 MB/sec execute 190 sec latency 609.159 ms 2024-06-06T08:07:08.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:08 smithi181 ceph-mon[36790]: pgmap v606: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 477 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-06-06T08:07:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:08 smithi087 ceph-mon[32276]: pgmap v606: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 477 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-06-06T08:07:08.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:08 smithi195 ceph-mon[37040]: pgmap v606: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 477 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-06-06T08:07:09.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 71078 12.79 MB/sec execute 191 sec latency 462.375 ms 2024-06-06T08:07:10.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 71373 12.83 MB/sec execute 192 sec latency 431.090 ms 2024-06-06T08:07:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:10 smithi181 ceph-mon[36790]: pgmap v607: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:10 smithi087 ceph-mon[32276]: pgmap v607: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:10.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:10 smithi195 ceph-mon[37040]: pgmap v607: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:11.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 71584 12.81 MB/sec execute 193 sec latency 223.490 ms 2024-06-06T08:07:12.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 71843 12.81 MB/sec execute 194 sec latency 243.159 ms 2024-06-06T08:07:12.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:12 smithi181 ceph-mon[36790]: pgmap v608: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:12 smithi087 ceph-mon[32276]: pgmap v608: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:12 smithi195 ceph-mon[37040]: pgmap v608: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 474 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-06-06T08:07:13.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 72184 12.81 MB/sec execute 195 sec latency 178.477 ms 2024-06-06T08:07:14.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 72475 12.83 MB/sec execute 196 sec latency 241.190 ms 2024-06-06T08:07:14.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:14 smithi181 ceph-mon[36790]: pgmap v609: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2024-06-06T08:07:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:14 smithi087 ceph-mon[32276]: pgmap v609: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2024-06-06T08:07:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:14 smithi195 ceph-mon[37040]: pgmap v609: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2024-06-06T08:07:15.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 72795 12.83 MB/sec execute 197 sec latency 202.145 ms 2024-06-06T08:07:16.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 73185 12.86 MB/sec execute 198 sec latency 232.171 ms 2024-06-06T08:07:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:15 smithi195 ceph-mon[37040]: pgmap v610: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-06-06T08:07:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:15 smithi195 ceph-mon[37040]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T08:07:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:15 smithi181 ceph-mon[36790]: pgmap v610: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-06-06T08:07:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:15 smithi181 ceph-mon[36790]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T08:07:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:15 smithi087 ceph-mon[32276]: pgmap v610: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-06-06T08:07:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:15 smithi087 ceph-mon[32276]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T08:07:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 -- 172.21.15.87:0/2395340209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a4101290 msgr2=0x7f81a410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/2395340209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a4101290 0x7f81a410d750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f819c009be0 tx=0x7f819c02a6c0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 -- 172.21.15.87:0/2395340209 shutdown_connections 2024-06-06T08:07:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/2395340209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a4101290 0x7f81a410d750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/2395340209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a41008d0 0x7f81a4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/2395340209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 -- 172.21.15.87:0/2395340209 >> 172.21.15.87:0/2395340209 conn(0x7f81a4077a20 msgr2=0x7f81a4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 -- 172.21.15.87:0/2395340209 shutdown_connections 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 -- 172.21.15.87:0/2395340209 wait complete. 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.754+0000 7f81ab2fc640 1 Processor -- start 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 -- start start 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a41008d0 0x7f81a40fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4110c60 con 0x7f81a4106890 2024-06-06T08:07:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4110ae0 con 0x7f81a41008d0 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81ab2fc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4110de0 con 0x7f81a4101290 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a8897640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53514/0 (socket says 172.21.15.87:53514) 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9098640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a41008d0 0x7f81a40fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a8897640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:36042/0 (socket says 172.21.15.87:36042) 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 -- 172.21.15.87:0/278878129 learned_addr learned my addr 172.21.15.87:0/278878129 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 -- 172.21.15.87:0/278878129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 msgr2=0x7f81a40fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 -- 172.21.15.87:0/278878129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a41008d0 msgr2=0x7f81a40fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a41008d0 0x7f81a40fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 -- 172.21.15.87:0/278878129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41a66c0 con 0x7f81a4106890 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a8897640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.755+0000 7f81a9899640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f819c034040 tx=0x7f819c030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.756+0000 7f819a7fc640 1 -- 172.21.15.87:0/278878129 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c040020 con 0x7f81a4106890 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.756+0000 7f819a7fc640 1 -- 172.21.15.87:0/278878129 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c003e50 con 0x7f81a4106890 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.756+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41a6950 con 0x7f81a4106890 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.756+0000 7f819a7fc640 1 -- 172.21.15.87:0/278878129 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c036040 con 0x7f81a4106890 2024-06-06T08:07:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.756+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a6e90 con 0x7f81a4106890 2024-06-06T08:07:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.758+0000 7f819a7fc640 1 -- 172.21.15.87:0/278878129 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f819c043020 con 0x7f81a4106890 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 Processor -- start 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 -- start start 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f819a7fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099ef6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564099e504f0 con 0x564099eec170 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564099dee870 con 0x564099ef7350 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81ae007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564099e4c4b0 con 0x564099eeca90 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53530/0 (socket says 172.21.15.87:53530) 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 -- 172.21.15.87:0/1695238681 learned_addr learned my addr 172.21.15.87:0/1695238681 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a9899640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a9098640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f819a7fc640 1 -- 172.21.15.87:0/278878129 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f819c0f03a0 con 0x7f81a4106890 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099ef6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 -- 172.21.15.87:0/1695238681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 msgr2=0x564099ef6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099ef6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 -- 172.21.15.87:0/1695238681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 msgr2=0x564099ef9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a8897640 1 -- 172.21.15.87:0/1695238681 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564099e0a9b0 con 0x564099eec170 2024-06-06T08:07:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.759+0000 7f81a9899640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f81a9098640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 0x7f81900a52e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005fd0 tx=0x7f81a0009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f81a8897640 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8194013c00 tx=0x7f8194009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ede4a1ccc2ffd5a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f818f7fe640 1 -- 172.21.15.87:0/1695238681 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194013db0 con 0x564099eec170 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f818f7fe640 1 -- 172.21.15.87:0/1695238681 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194013170 con 0x564099eec170 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f818f7fe640 1 -- 172.21.15.87:0/1695238681 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194010d80 con 0x564099eec170 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.760+0000 7f81ae007740 1 -- 172.21.15.87:0/1695238681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 msgr2=0x564099eec550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8194013c00 tx=0x7f8194009b50 comp rx=0 tx=0).stop 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- 172.21.15.87:0/1695238681 shutdown_connections 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099ef6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- 172.21.15.87:0/1695238681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eec550 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- 172.21.15.87:0/1695238681 >> 172.21.15.87:0/1695238681 conn(0x564099ee8680 msgr2=0x564099ee8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- 172.21.15.87:0/1695238681 shutdown_connections 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- 172.21.15.87:0/1695238681 wait complete. 2024-06-06T08:07:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- start start 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 Processor -- start 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099ef4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.761+0000 7f81ae007740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564099ef9f20 con 0x564099eeca90 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564099dd9ae0 con 0x564099ef7350 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564099efa0a0 con 0x564099eec170 2024-06-06T08:07:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 10 client.0 _ll_register_callbacks cb 0x564099e129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53546/0 (socket says 172.21.15.87:53546) 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 -- 172.21.15.87:0/1047268047 learned_addr learned my addr 172.21.15.87:0/1047268047 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a9098640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099ef4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a9899640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 -- 172.21.15.87:0/1047268047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 msgr2=0x564099ef4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099ef4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 -- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 msgr2=0x564099ef4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 -- 172.21.15.87:0/1047268047 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564099b64210 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81a8897640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8194010d60 tx=0x7f8194010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f99e2e32197cb65 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f818cff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194010840 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f8194005050 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8194013170 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8194005250 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 10 client.18156 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.762+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564099c30620 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194013170 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f8194029470 con 0x564099eeca90 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 10 client.18156 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 20 client.18156 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f07350 0x564099f09740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 4 client.18156 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564099e63f10 con 0x564099f07350 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81a9098640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f07350 0x564099f09740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f818cff9640 10 client.18156 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.763+0000 7f81a9098640 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f07350 0x564099f09740 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6dff3095633ea0 server_cookie=96c979322f444650 in_seq=0 out_seq=0 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.767+0000 7f818cff9640 1 -- 172.21.15.87:0/1047268047 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564099e63f10 con 0x564099f07350 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.767+0000 7f818cff9640 10 client.18156 handle_command_reply: tid=0 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.767+0000 7f81ae007740 1 client.18156 shutdown 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 msgr2=0x564099ef47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8194010d60 tx=0x7f8194010d90 comp rx=0 tx=0).stop 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 shutdown_connections 2024-06-06T08:07:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f07350 0x564099f09740 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099ef4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eeca90 0x564099ef47c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 --2- 172.21.15.87:0/1047268047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099ef4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f818cff9640 0 client.18156 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 >> 172.21.15.87:0/1047268047 conn(0x564099ee8680 msgr2=0x564099eee3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 shutdown_connections 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 1 -- 172.21.15.87:0/1047268047 wait complete. 2024-06-06T08:07:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.768+0000 7f81ae007740 20 client.18156 trim_cache size 0 max 16384 2024-06-06T08:07:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 Processor -- start 2024-06-06T08:07:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 -- start start 2024-06-06T08:07:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099eeeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099eeca90 0x564099ef61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b3a20 con 0x564099ef7350 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b12f0 con 0x564099eeca90 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81ae007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41af160 con 0x564099eec170 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53560/0 (socket says 172.21.15.87:53560) 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 -- 172.21.15.87:0/1217167023 learned_addr learned my addr 172.21.15.87:0/1217167023 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a8897640 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099eeca90 0x564099ef61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9098640 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099eeeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 -- 172.21.15.87:0/1217167023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 msgr2=0x564099eeeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099eeeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 -- 172.21.15.87:0/1217167023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099eeca90 msgr2=0x564099ef61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099eeca90 0x564099ef61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 -- 172.21.15.87:0/1217167023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564099eef070 con 0x564099ef7350 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.785+0000 7f81a9899640 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b8e70 tx=0x7f819c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c142de3916f1e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f818effd640 1 -- 172.21.15.87:0/1217167023 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0bc4a0 con 0x564099ef7350 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f818effd640 1 -- 172.21.15.87:0/1217167023 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c0c0850 con 0x564099ef7350 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f818effd640 1 -- 172.21.15.87:0/1217167023 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0d5a60 con 0x564099ef7350 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 -- 172.21.15.87:0/1217167023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 msgr2=0x564099f0ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b8e70 tx=0x7f819c04c000 comp rx=0 tx=0).stop 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 -- 172.21.15.87:0/1217167023 shutdown_connections 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099ef7350 0x564099f0ee80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099eeca90 0x564099ef61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 --2- 172.21.15.87:0/1217167023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eec170 0x564099eeeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 -- 172.21.15.87:0/1217167023 >> 172.21.15.87:0/1217167023 conn(0x564099ee8680 msgr2=0x564099ef5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 -- 172.21.15.87:0/1217167023 shutdown_connections 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.786+0000 7f81ae007740 1 -- 172.21.15.87:0/1217167023 wait complete. 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 -- start start 2024-06-06T08:07:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 Processor -- start 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099efacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099efb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41af7b0 con 0x564099eec170 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b0c20 con 0x564099ef7350 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41afe80 con 0x564099eeca90 2024-06-06T08:07:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81ae007740 10 client.0 _ll_register_callbacks cb 0x7f819c0b8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53572/0 (socket says 172.21.15.87:53572) 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 -- 172.21.15.87:0/2148793424 learned_addr learned my addr 172.21.15.87:0/2148793424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a8897640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099efacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9899640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099efb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 -- 172.21.15.87:0/2148793424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 msgr2=0x564099efacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099efacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 -- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 msgr2=0x564099efb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099efb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.787+0000 7f81a9098640 1 -- 172.21.15.87:0/2148793424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564099efb900 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81a9098640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x564099f09df0 tx=0x7f81a0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51fec33711c0c7d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a000e540 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f81a0009040 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819c0c0850 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81a0036050 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 10 client.18162 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564099f71dc0 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c0c0850 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f81a001ebc0 con 0x564099eec170 2024-06-06T08:07:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 10 client.18162 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 20 client.18162 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f72540 0x564099f74960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 4 client.18162 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.788+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564099e5dc20 con 0x564099f72540 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.789+0000 7f81a8897640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f72540 0x564099f74960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.789+0000 7f81a8897640 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f72540 0x564099f74960 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ee6de407291b94 server_cookie=85c18154af1ff4ab in_seq=0 out_seq=0 2024-06-06T08:07:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:16.789+0000 7f818cff9640 10 client.18162 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:17.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 73453 12.87 MB/sec execute 199 sec latency 752.183 ms 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f818cff9640 1 -- 172.21.15.87:0/2148793424 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564099e5dc20 con 0x564099f72540 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f818cff9640 10 client.18162 handle_command_reply: tid=0 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 client.18162 shutdown 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 msgr2=0x564099eff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x564099f09df0 tx=0x7f81a0013040 comp rx=0 tx=0).stop 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 shutdown_connections 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564099f72540 0x564099f74960 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564099ef7350 0x564099efb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564099eeca90 0x564099efacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 --2- 172.21.15.87:0/2148793424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564099eec170 0x564099eff960 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.378+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 >> 172.21.15.87:0/2148793424 conn(0x564099ee8680 msgr2=0x564099ef6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f818cff9640 0 client.18162 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 shutdown_connections 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ae007740 1 -- 172.21.15.87:0/2148793424 wait complete. 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ae007740 20 client.18162 trim_cache size 0 max 16384 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 msgr2=0x7f81900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 0x7f81900a52e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005fd0 tx=0x7f81a0009260 comp rx=0 tx=0).stop 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 msgr2=0x7f81a40fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f819c034040 tx=0x7f819c030c50 comp rx=0 tx=0).stop 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 shutdown_connections 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81a4106890 0x7f81a40fe2e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81a4101290 0x7f81a40fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 --2- 172.21.15.87:0/278878129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a41008d0 0x7f81a40fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.379+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 >> 172.21.15.87:0/278878129 conn(0x7f81a4077a20 msgr2=0x7f81a4069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.380+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 shutdown_connections 2024-06-06T08:07:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:17.380+0000 7f81ab2fc640 1 -- 172.21.15.87:0/278878129 wait complete. 2024-06-06T08:07:18.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 73782 12.92 MB/sec execute 200 sec latency 1294.211 ms 2024-06-06T08:07:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:17 smithi195 ceph-mon[37040]: pgmap v612: 129 pgs: 129 active+clean; 490 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2024-06-06T08:07:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:17 smithi181 ceph-mon[36790]: pgmap v612: 129 pgs: 129 active+clean; 490 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2024-06-06T08:07:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:17 smithi087 ceph-mon[32276]: pgmap v612: 129 pgs: 129 active+clean; 490 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.4 MiB/s wr, 383 op/s 2024-06-06T08:07:19.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 74102 12.96 MB/sec execute 201 sec latency 197.853 ms 2024-06-06T08:07:20.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 74370 12.96 MB/sec execute 202 sec latency 211.026 ms 2024-06-06T08:07:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:19 smithi195 ceph-mon[37040]: pgmap v613: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:20.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:19 smithi181 ceph-mon[36790]: pgmap v613: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:19 smithi087 ceph-mon[32276]: pgmap v613: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:21.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 74714 12.96 MB/sec execute 203 sec latency 333.805 ms 2024-06-06T08:07:22.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:21 smithi087 ceph-mon[32276]: pgmap v614: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:22.179 INFO:tasks.workunit.client.0.smithi087.stdout: 10 75030 12.99 MB/sec execute 204 sec latency 238.122 ms 2024-06-06T08:07:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:21 smithi195 ceph-mon[37040]: pgmap v614: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:22.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:21 smithi181 ceph-mon[36790]: pgmap v614: 129 pgs: 129 active+clean; 509 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 411 op/s 2024-06-06T08:07:23.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 75383 13.01 MB/sec execute 205 sec latency 231.605 ms 2024-06-06T08:07:24.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 75807 13.08 MB/sec execute 206 sec latency 172.653 ms 2024-06-06T08:07:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:23 smithi195 ceph-mon[37040]: pgmap v615: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:23 smithi181 ceph-mon[36790]: pgmap v615: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:23 smithi087 ceph-mon[32276]: pgmap v615: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:25.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 76172 13.13 MB/sec execute 207 sec latency 172.854 ms 2024-06-06T08:07:26.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 76531 13.16 MB/sec execute 208 sec latency 318.700 ms 2024-06-06T08:07:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:25 smithi195 ceph-mon[37040]: pgmap v616: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:25 smithi181 ceph-mon[36790]: pgmap v616: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:25 smithi087 ceph-mon[32276]: pgmap v616: 129 pgs: 129 active+clean; 505 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-06-06T08:07:27.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 76775 13.14 MB/sec execute 209 sec latency 233.974 ms 2024-06-06T08:07:28.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 77033 13.12 MB/sec execute 210 sec latency 236.136 ms 2024-06-06T08:07:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:27 smithi195 ceph-mon[37040]: pgmap v617: 129 pgs: 129 active+clean; 508 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-06-06T08:07:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:27 smithi181 ceph-mon[36790]: pgmap v617: 129 pgs: 129 active+clean; 508 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-06-06T08:07:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:27 smithi087 ceph-mon[32276]: pgmap v617: 129 pgs: 129 active+clean; 508 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-06-06T08:07:29.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 77405 13.15 MB/sec execute 211 sec latency 254.830 ms 2024-06-06T08:07:30.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 77808 13.23 MB/sec execute 212 sec latency 197.472 ms 2024-06-06T08:07:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:29 smithi195 ceph-mon[37040]: pgmap v618: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-06T08:07:30.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:29 smithi181 ceph-mon[36790]: pgmap v618: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-06T08:07:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:29 smithi087 ceph-mon[32276]: pgmap v618: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-06T08:07:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.480+0000 7fa376d1d640 1 -- 172.21.15.87:0/3867560603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 msgr2=0x7fa37010e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.480+0000 7fa376d1d640 1 --2- 172.21.15.87:0/3867560603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37010e410 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa368009ca0 tx=0x7fa36802a9e0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- 172.21.15.87:0/3867560603 shutdown_connections 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- 172.21.15.87:0/3867560603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37010e410 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- 172.21.15.87:0/3867560603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 0x7fa37010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- 172.21.15.87:0/3867560603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa370103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- 172.21.15.87:0/3867560603 >> 172.21.15.87:0/3867560603 conn(0x7fa370077a20 msgr2=0x7fa370077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- 172.21.15.87:0/3867560603 shutdown_connections 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- 172.21.15.87:0/3867560603 wait complete. 2024-06-06T08:07:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 Processor -- start 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- start start 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa3701981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 0x7fa370198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa370111d70 con 0x7fa37010bf70 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa370111bf0 con 0x7fa370103550 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa376d1d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa370111ef0 con 0x7fa370102c30 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa3752ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa3752ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39870/0 (socket says 172.21.15.87:39870) 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa3752ba640 1 -- 172.21.15.87:0/2695269555 learned_addr learned my addr 172.21.15.87:0/2695269555 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa374ab9640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa3701981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.481+0000 7fa367fff640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 0x7fa370198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 -- 172.21.15.87:0/2695269555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 msgr2=0x7fa3701981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa3701981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 -- 172.21.15.87:0/2695269555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 msgr2=0x7fa370198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 0x7fa370198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 -- 172.21.15.87:0/2695269555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37019d170 con 0x7fa37010bf70 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa374ab9640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa3701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa3752ba640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa368006010 tx=0x7fa368003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa365ffb640 1 -- 172.21.15.87:0/2695269555 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368047070 con 0x7fa37010bf70 2024-06-06T08:07:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37019d400 con 0x7fa37010bf70 2024-06-06T08:07:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa365ffb640 1 -- 172.21.15.87:0/2695269555 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36802fb20 con 0x7fa37010bf70 2024-06-06T08:07:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa365ffb640 1 -- 172.21.15.87:0/2695269555 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368033050 con 0x7fa37010bf70 2024-06-06T08:07:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.482+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3701a4cf0 con 0x7fa37010bf70 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.485+0000 7fa365ffb640 1 -- 172.21.15.87:0/2695269555 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa36804a070 con 0x7fa37010bf70 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa365ffb640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 0x7fa3580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 Processor -- start 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 -- start start 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa365ffb640 1 -- 172.21.15.87:0/2695269555 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fa3680f02e0 con 0x7fa37010bf70 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa374ab9640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 0x7fa3580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e95a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5575e94fe4f0 con 0x5575e95a5350 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5575e949c870 con 0x5575e959a170 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa379a21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5575e94fa4b0 con 0x5575e959aa90 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa374ab9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa374ab9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:34434/0 (socket says 172.21.15.87:34434) 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa374ab9640 1 -- 172.21.15.87:0/1842951880 learned_addr learned my addr 172.21.15.87:0/1842951880 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa3752ba640 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e95a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa367fff640 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.486+0000 7fa374ab9640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 0x7fa3580a52e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009c70 tx=0x7fa36c005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 -- 172.21.15.87:0/1842951880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 msgr2=0x5575e95a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 -- 172.21.15.87:0/1842951880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 msgr2=0x5575e95a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e95a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 -- 172.21.15.87:0/1842951880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e94b89b0 con 0x5575e959a170 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa374ab9640 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa36c012500 tx=0x7fa36c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4e6674831798d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa356ffd640 1 -- 172.21.15.87:0/1842951880 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c01f620 con 0x5575e959a170 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa356ffd640 1 -- 172.21.15.87:0/1842951880 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c01fca0 con 0x5575e959a170 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.487+0000 7fa356ffd640 1 -- 172.21.15.87:0/1842951880 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c027dc0 con 0x5575e959a170 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- 172.21.15.87:0/1842951880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 msgr2=0x5575e959a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa36c012500 tx=0x7fa36c014000 comp rx=0 tx=0).stop 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- 172.21.15.87:0/1842951880 shutdown_connections 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e95a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 --2- 172.21.15.87:0/1842951880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e959a550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- 172.21.15.87:0/1842951880 >> 172.21.15.87:0/1842951880 conn(0x5575e9596680 msgr2=0x5575e9596a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- 172.21.15.87:0/1842951880 shutdown_connections 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- 172.21.15.87:0/1842951880 wait complete. 2024-06-06T08:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 -- start start 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.488+0000 7fa379a21740 1 Processor -- start 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e95a5350 0x5575e95a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5575e95a7f80 con 0x5575e959a170 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5575e9487ae0 con 0x5575e95a5350 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5575e95a8100 con 0x5575e959aa90 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa379a21740 10 client.0 _ll_register_callbacks cb 0x5575e94c09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39886/0 (socket says 172.21.15.87:39886) 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 -- 172.21.15.87:0/3743627463 learned_addr learned my addr 172.21.15.87:0/3743627463 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa367fff640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa3752ba640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e95a5350 0x5575e95a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 -- 172.21.15.87:0/3743627463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 msgr2=0x5575e95a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 -- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e95a5350 msgr2=0x5575e95a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e95a5350 0x5575e95a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 -- 172.21.15.87:0/3743627463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e9212210 con 0x5575e959a170 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa367fff640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa374ab9640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009650 tx=0x7fa36c027630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1384a23c8394181 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.489+0000 7fa34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c037060 con 0x5575e959a170 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fa36c005260 con 0x5575e959a170 2024-06-06T08:07:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa36c01fca0 con 0x5575e959a170 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa36c03e050 con 0x5575e959a170 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 10 client.18174 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5575e92de620 con 0x5575e959a170 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c01fca0 con 0x5575e959a170 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fa36c004750 con 0x5575e959a170 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 10 client.18174 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 10 client.18174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 20 client.18174 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e95b54a0 0x5575e95b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 4 client.18174 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e9511f10 con 0x5575e95b54a0 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.490+0000 7fa367fff640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e95b54a0 0x5575e95b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.491+0000 7fa34ffff640 10 client.18174 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.491+0000 7fa367fff640 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e95b54a0 0x5575e95b7890 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c51fb42a34dfa9 server_cookie=a8d45737f605988e in_seq=0 out_seq=0 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa34ffff640 1 -- 172.21.15.87:0/3743627463 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5575e9511f10 con 0x5575e95b54a0 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa34ffff640 10 client.18174 handle_command_reply: tid=0 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 client.18174 shutdown 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 msgr2=0x5575e95a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009650 tx=0x7fa36c027630 comp rx=0 tx=0).stop 2024-06-06T08:07:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 shutdown_connections 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e95b54a0 0x5575e95b7890 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e95a5350 0x5575e95a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959aa90 0x5575e95a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa34ffff640 0 client.18174 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 --2- 172.21.15.87:0/3743627463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959a170 0x5575e95a2360 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 >> 172.21.15.87:0/3743627463 conn(0x5575e9596680 msgr2=0x5575e959c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 shutdown_connections 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 1 -- 172.21.15.87:0/3743627463 wait complete. 2024-06-06T08:07:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.494+0000 7fa379a21740 20 client.18174 trim_cache size 0 max 16384 2024-06-06T08:07:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 Processor -- start 2024-06-06T08:07:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 -- start start 2024-06-06T08:07:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e9595660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e95a5350 0x5575e95bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701b1880 con 0x5575e959aa90 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701af150 con 0x5575e959a170 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.511+0000 7fa379a21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701acfc0 con 0x5575e95a5350 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39894/0 (socket says 172.21.15.87:39894) 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 -- 172.21.15.87:0/1667304594 learned_addr learned my addr 172.21.15.87:0/1667304594 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa3752ba640 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e95a5350 0x5575e95bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa374ab9640 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e9595660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 -- 172.21.15.87:0/1667304594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e95a5350 msgr2=0x5575e95bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e95a5350 0x5575e95bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 -- 172.21.15.87:0/1667304594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 msgr2=0x5575e9595660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e9595660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 -- 172.21.15.87:0/1667304594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e95b11e0 con 0x5575e959aa90 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa367fff640 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x5575e95a1bc0 tx=0x7fa360005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4163383f2c77c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa3567fc640 1 -- 172.21.15.87:0/1667304594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360004030 con 0x5575e959aa90 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa3567fc640 1 -- 172.21.15.87:0/1667304594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36000fcd0 con 0x5575e959aa90 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.512+0000 7fa3567fc640 1 -- 172.21.15.87:0/1667304594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360003b50 con 0x5575e959aa90 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- 172.21.15.87:0/1667304594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 msgr2=0x5575e95bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x5575e95a1bc0 tx=0x7fa360005f90 comp rx=0 tx=0).stop 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- 172.21.15.87:0/1667304594 shutdown_connections 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e95a5350 0x5575e95bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e959aa90 0x5575e95bce30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 --2- 172.21.15.87:0/1667304594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959a170 0x5575e9595660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- 172.21.15.87:0/1667304594 >> 172.21.15.87:0/1667304594 conn(0x5575e9596680 msgr2=0x5575e9594f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- 172.21.15.87:0/1667304594 shutdown_connections 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- 172.21.15.87:0/1667304594 wait complete. 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 -- start start 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.513+0000 7fa379a21740 1 Processor -- start 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959a170 0x5575e959f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 0x5575e959f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ad610 con 0x5575e95a5350 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701aea80 con 0x5575e959aa90 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701adce0 con 0x5575e959a170 2024-06-06T08:07:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa379a21740 10 client.0 _ll_register_callbacks cb 0x7fa3680b6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa3752ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa3752ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39904/0 (socket says 172.21.15.87:39904) 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 0x5575e959f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa3752ba640 1 -- 172.21.15.87:0/790605887 learned_addr learned my addr 172.21.15.87:0/790605887 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa374ab9640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959a170 0x5575e959f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 -- 172.21.15.87:0/790605887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959a170 msgr2=0x5575e959f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959a170 0x5575e959f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 -- 172.21.15.87:0/790605887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 msgr2=0x5575e959ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 -- 172.21.15.87:0/790605887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e961eeb0 con 0x5575e959aa90 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa3752ba640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa367fff640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 0x5575e959f9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5575e95b1400 tx=0x7fa360009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f42817d717dfead server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.514+0000 7fa34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360037060 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa36000fcd0 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fa36000f7f0 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa36003e050 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 10 client.27122 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5575e96201a0 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36000fcd0 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fa360004730 con 0x5575e959aa90 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 10 client.27122 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 20 client.27122 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e9620870 0x5575e9622c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 4 client.27122 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575e950bc20 con 0x5575e9620870 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.515+0000 7fa374ab9640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e9620870 0x5575e9622c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.516+0000 7fa34ffff640 10 client.27122 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:30.516+0000 7fa374ab9640 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e9620870 0x5575e9622c90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae36dbceee5936ba server_cookie=620fc6d76b512bfa in_seq=0 out_seq=0 2024-06-06T08:07:31.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 77919 13.18 MB/sec execute 213 sec latency 529.443 ms 2024-06-06T08:07:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa34ffff640 1 -- 172.21.15.87:0/790605887 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575e950bc20 con 0x5575e9620870 2024-06-06T08:07:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa34ffff640 10 client.27122 handle_command_reply: tid=0 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 client.27122 shutdown 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 msgr2=0x5575e959f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 0x5575e959f9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5575e95b1400 tx=0x7fa360009c30 comp rx=0 tx=0).stop 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 shutdown_connections 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5575e9620870 0x5575e9622c90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5575e95a5350 0x5575e959ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5575e959aa90 0x5575e959f9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 --2- 172.21.15.87:0/790605887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5575e959a170 0x5575e959f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa34ffff640 0 client.27122 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 >> 172.21.15.87:0/790605887 conn(0x5575e9596680 msgr2=0x5575e95b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 shutdown_connections 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.247+0000 7fa379a21740 1 -- 172.21.15.87:0/790605887 wait complete. 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.248+0000 7fa379a21740 20 client.27122 trim_cache size 0 max 16384 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:07:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:07:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.248+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 msgr2=0x7fa3580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.248+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 0x7fa3580a52e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009c70 tx=0x7fa36c005f90 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 msgr2=0x7fa37019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa368006010 tx=0x7fa368003ba0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 shutdown_connections 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa3580a2e20 0x7fa3580a52e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37010bf70 0x7fa37019ca40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa370103550 0x7fa370198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 --2- 172.21.15.87:0/2695269555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa370102c30 0x7fa3701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 >> 172.21.15.87:0/2695269555 conn(0x7fa370077a20 msgr2=0x7fa3700fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 shutdown_connections 2024-06-06T08:07:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:31.249+0000 7fa376d1d640 1 -- 172.21.15.87:0/2695269555 wait complete. 2024-06-06T08:07:32.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 78260 13.19 MB/sec execute 214 sec latency 621.824 ms 2024-06-06T08:07:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:31 smithi195 ceph-mon[37040]: pgmap v619: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-06-06T08:07:32.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:31 smithi181 ceph-mon[36790]: pgmap v619: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-06-06T08:07:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:31 smithi087 ceph-mon[32276]: pgmap v619: 129 pgs: 129 active+clean; 522 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-06-06T08:07:33.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 78590 13.23 MB/sec execute 215 sec latency 198.535 ms 2024-06-06T08:07:34.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:33 smithi087 ceph-mon[32276]: pgmap v620: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-06-06T08:07:34.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 78973 13.24 MB/sec execute 216 sec latency 968.520 ms 2024-06-06T08:07:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:33 smithi195 ceph-mon[37040]: pgmap v620: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-06-06T08:07:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:33 smithi181 ceph-mon[36790]: pgmap v620: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-06-06T08:07:35.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 79265 13.26 MB/sec execute 217 sec latency 339.753 ms 2024-06-06T08:07:36.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 79598 13.28 MB/sec execute 218 sec latency 247.553 ms 2024-06-06T08:07:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:35 smithi195 ceph-mon[37040]: pgmap v621: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-06-06T08:07:36.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:35 smithi181 ceph-mon[36790]: pgmap v621: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-06-06T08:07:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:35 smithi087 ceph-mon[32276]: pgmap v621: 129 pgs: 129 active+clean; 513 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-06-06T08:07:37.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 80026 13.33 MB/sec execute 219 sec latency 163.285 ms 2024-06-06T08:07:38.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 80425 13.37 MB/sec execute 220 sec latency 195.772 ms 2024-06-06T08:07:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:37 smithi195 ceph-mon[37040]: pgmap v622: 129 pgs: 129 active+clean; 517 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:07:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:37 smithi181 ceph-mon[36790]: pgmap v622: 129 pgs: 129 active+clean; 517 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:07:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:37 smithi087 ceph-mon[32276]: pgmap v622: 129 pgs: 129 active+clean; 517 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:07:39.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 80736 13.41 MB/sec execute 221 sec latency 195.275 ms 2024-06-06T08:07:40.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 80990 13.41 MB/sec execute 222 sec latency 393.406 ms 2024-06-06T08:07:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:39 smithi195 ceph-mon[37040]: pgmap v623: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 542 op/s 2024-06-06T08:07:40.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:39 smithi181 ceph-mon[36790]: pgmap v623: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 542 op/s 2024-06-06T08:07:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:39 smithi087 ceph-mon[32276]: pgmap v623: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 542 op/s 2024-06-06T08:07:41.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 81225 13.40 MB/sec execute 223 sec latency 270.537 ms 2024-06-06T08:07:41.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:40 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:07:41.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:40 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:07:41.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:40 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:07:42.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 81484 13.40 MB/sec execute 224 sec latency 238.284 ms 2024-06-06T08:07:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:41 smithi195 ceph-mon[37040]: pgmap v624: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-06T08:07:42.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:41 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:07:42.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:41 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:07:42.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:41 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:07:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:41 smithi181 ceph-mon[36790]: pgmap v624: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-06T08:07:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:41 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:07:42.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:41 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:07:42.307 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:41 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:07:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:41 smithi087 ceph-mon[32276]: pgmap v624: 129 pgs: 129 active+clean; 530 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-06T08:07:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:41 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:07:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:41 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:07:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:41 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:07:43.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 81798 13.40 MB/sec execute 225 sec latency 261.740 ms 2024-06-06T08:07:43.619 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T08:07:43.619 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-06T08:07:43.620 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:07:43.881 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:07:44.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 82100 13.43 MB/sec execute 226 sec latency 187.657 ms 2024-06-06T08:07:44.204 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:43 smithi087 ceph-mon[32276]: pgmap v625: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 569 op/s 2024-06-06T08:07:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:43 smithi195 ceph-mon[37040]: pgmap v625: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 569 op/s 2024-06-06T08:07:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:43 smithi181 ceph-mon[36790]: pgmap v625: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 569 op/s 2024-06-06T08:07:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 -- 172.21.15.87:0/4123336820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc110570 msgr2=0x7fdedc108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 --2- 172.21.15.87:0/4123336820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc110570 0x7fdedc108ee0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fded0009890 tx=0x7fded002a410 comp rx=0 tx=0).stop 2024-06-06T08:07:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 -- 172.21.15.87:0/4123336820 shutdown_connections 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 --2- 172.21.15.87:0/4123336820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 0x7fdedc11d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 --2- 172.21.15.87:0/4123336820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc110570 0x7fdedc108ee0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 --2- 172.21.15.87:0/4123336820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.371+0000 7fdee1c37640 1 -- 172.21.15.87:0/4123336820 >> 172.21.15.87:0/4123336820 conn(0x7fdedc06ca70 msgr2=0x7fdedc06ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.372+0000 7fdee1c37640 1 -- 172.21.15.87:0/4123336820 shutdown_connections 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.372+0000 7fdee1c37640 1 -- 172.21.15.87:0/4123336820 wait complete. 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.372+0000 7fdee1c37640 1 Processor -- start 2024-06-06T08:07:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 -- start start 2024-06-06T08:07:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 0x7fdedc1a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc120ba0 con 0x7fdedc1ab920 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc120a20 con 0x7fdedc11b180 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdee1c37640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc120d20 con 0x7fdedc110190 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41700/0 (socket says 172.21.15.87:41700) 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 -- 172.21.15.87:0/3826618739 learned_addr learned my addr 172.21.15.87:0/3826618739 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedaffd640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 0x7fdedc1a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 -- 172.21.15.87:0/3826618739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 msgr2=0x7fdedc1a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 -- 172.21.15.87:0/3826618739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 msgr2=0x7fdedc1a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 0x7fdedc1a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.373+0000 7fdedbfff640 1 -- 172.21.15.87:0/3826618739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc1ac410 con 0x7fdedc1ab920 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fdedbfff640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e970 tx=0x7fdecc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fded8ff9640 1 -- 172.21.15.87:0/3826618739 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00ccc0 con 0x7fdedc1ab920 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fded8ff9640 1 -- 172.21.15.87:0/3826618739 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdecc005500 con 0x7fdedc1ab920 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1ac700 con 0x7fdedc1ab920 2024-06-06T08:07:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fded8ff9640 1 -- 172.21.15.87:0/3826618739 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc010690 con 0x7fdedc1ab920 2024-06-06T08:07:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.374+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc1b3ec0 con 0x7fdedc1ab920 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.377+0000 7fded8ff9640 1 -- 172.21.15.87:0/3826618739 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdecc0046c0 con 0x7fdedc1ab920 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 Processor -- start 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 -- start start 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7582e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7583350 0x55c3f7585740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c3f74dc4f0 con 0x55c3f7578170 2024-06-06T08:07:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3f747a870 con 0x55c3f7578a90 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdee4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3f74d84b0 con 0x55c3f7583350 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fded8ff9640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 0x7fdec40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7582e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41710/0 (socket says 172.21.15.87:41710) 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedaffd640 1 -- 172.21.15.87:0/188561150 learned_addr learned my addr 172.21.15.87:0/188561150 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedaffd640 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7583350 0x55c3f7585740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fdedbfff640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 0x7fdec40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.378+0000 7fded8ff9640 1 -- 172.21.15.87:0/3826618739 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fdecc0cfb70 con 0x7fdedc1ab920 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 -- 172.21.15.87:0/188561150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7583350 msgr2=0x55c3f7585740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7583350 0x55c3f7585740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 -- 172.21.15.87:0/188561150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 msgr2=0x55c3f7582e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7582e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 -- 172.21.15.87:0/188561150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f74969b0 con 0x55c3f7578170 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedaffd640 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fded0005b60 tx=0x7fded0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b085dccd86ccab4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdedbfff640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 0x7fdec40a52e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0ccf50 tx=0x7fdecc0c6800 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdec1ffb640 1 -- 172.21.15.87:0/188561150 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00303a0 con 0x55c3f7578170 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdec1ffb640 1 -- 172.21.15.87:0/188561150 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0005040 con 0x55c3f7578170 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.379+0000 7fdec1ffb640 1 -- 172.21.15.87:0/188561150 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0004a40 con 0x55c3f7578170 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- 172.21.15.87:0/188561150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 msgr2=0x55c3f7578550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fded0005b60 tx=0x7fded0005a90 comp rx=0 tx=0).stop 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- 172.21.15.87:0/188561150 shutdown_connections 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7583350 0x55c3f7585740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7582e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 --2- 172.21.15.87:0/188561150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7578170 0x55c3f7578550 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- 172.21.15.87:0/188561150 >> 172.21.15.87:0/188561150 conn(0x55c3f7574680 msgr2=0x55c3f7574a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- 172.21.15.87:0/188561150 shutdown_connections 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- 172.21.15.87:0/188561150 wait complete. 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.380+0000 7fdee4a16740 1 -- start start 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 Processor -- start 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7580320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7580860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c3f7585ec0 con 0x55c3f7583350 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3f7465ae0 con 0x55c3f7578a90 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3f7586040 con 0x55c3f7578170 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdee4a16740 10 client.0 _ll_register_callbacks cb 0x55c3f749e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7580320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41716/0 (socket says 172.21.15.87:41716) 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdedaffd640 1 -- 172.21.15.87:0/1977277280 learned_addr learned my addr 172.21.15.87:0/1977277280 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.381+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7580860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 -- 172.21.15.87:0/1977277280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 msgr2=0x55c3f7580320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7580320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 -- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 msgr2=0x55c3f7580860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7580860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 -- 172.21.15.87:0/1977277280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f71f0210 con 0x55c3f7583350 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fded0005790 tx=0x7fded002f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f384238303b665 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00303a0 con 0x55c3f7583350 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fded0030540 con 0x55c3f7583350 2024-06-06T08:07:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.382+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0005040 con 0x55c3f7583350 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fded0030740 con 0x55c3f7583350 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 10 client.18192 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c3f72bc620 con 0x55c3f7583350 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0005040 con 0x55c3f7583350 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fded00039c0 con 0x55c3f7583350 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 10 client.18192 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 20 client.18192 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f75956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 4 client.18192 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.383+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3f74eff10 con 0x55c3f75932f0 2024-06-06T08:07:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.384+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f75956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.384+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f75956e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519110b6cf7dd42a server_cookie=388cbc795d2164ed in_seq=0 out_seq=0 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.384+0000 7fdebaffd640 10 client.18192 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdebaffd640 1 -- 172.21.15.87:0/1977277280 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3f74eff10 con 0x55c3f75932f0 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdebaffd640 10 client.18192 handle_command_reply: tid=0 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 client.18192 shutdown 2024-06-06T08:07:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 msgr2=0x55c3f7580da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fded0005790 tx=0x7fded002f360 comp rx=0 tx=0).stop 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 shutdown_connections 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f75956e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7580da0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdebaffd640 0 client.18192 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7580860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1977277280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7580320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 >> 172.21.15.87:0/1977277280 conn(0x55c3f7574680 msgr2=0x55c3f757a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 shutdown_connections 2024-06-06T08:07:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.387+0000 7fdee4a16740 1 -- 172.21.15.87:0/1977277280 wait complete. 2024-06-06T08:07:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.388+0000 7fdee4a16740 20 client.18192 trim_cache size 0 max 16384 2024-06-06T08:07:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 Processor -- start 2024-06-06T08:07:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 -- start start 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7582260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7598c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1c06e0 con 0x55c3f7583350 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1bdfb0 con 0x55c3f7578a90 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.404+0000 7fdee4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1bbe20 con 0x55c3f7578170 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7598c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:59582/0 (socket says 172.21.15.87:59582) 2024-06-06T08:07:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedb7fe640 1 -- 172.21.15.87:0/1957775771 learned_addr learned my addr 172.21.15.87:0/1957775771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7582260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 -- 172.21.15.87:0/1957775771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 msgr2=0x55c3f7582260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7582260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 -- 172.21.15.87:0/1957775771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 msgr2=0x55c3f7598700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 -- 172.21.15.87:0/1957775771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f7575410 con 0x55c3f7583350 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdedaffd640 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7598c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fded0003fd0 tx=0x7fded0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c26575ae1f2cb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdec17fa640 1 -- 172.21.15.87:0/1957775771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0062070 con 0x55c3f7583350 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdec17fa640 1 -- 172.21.15.87:0/1957775771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0043040 con 0x55c3f7583350 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.405+0000 7fdec17fa640 1 -- 172.21.15.87:0/1957775771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00035f0 con 0x55c3f7583350 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 -- 172.21.15.87:0/1957775771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 msgr2=0x55c3f7598c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7598c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fded0003fd0 tx=0x7fded0000bc0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 -- 172.21.15.87:0/1957775771 shutdown_connections 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7598c40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578a90 0x55c3f7598700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 --2- 172.21.15.87:0/1957775771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578170 0x55c3f7582260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 -- 172.21.15.87:0/1957775771 >> 172.21.15.87:0/1957775771 conn(0x55c3f7574680 msgr2=0x55c3f7581b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 -- 172.21.15.87:0/1957775771 shutdown_connections 2024-06-06T08:07:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.406+0000 7fdee4a16740 1 -- 172.21.15.87:0/1957775771 wait complete. 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 -- start start 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 Processor -- start 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 0x55c3f75966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 0x55c3f7596c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7597170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1bc470 con 0x55c3f7583350 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 0x55c3f75966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 0x55c3f7596c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdedaffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 0x55c3f75966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:59598/0 (socket says 172.21.15.87:59598) 2024-06-06T08:07:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdedaffd640 1 -- 172.21.15.87:0/4220382239 learned_addr learned my addr 172.21.15.87:0/4220382239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1bd8e0 con 0x55c3f7578170 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.407+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1bcb40 con 0x55c3f7578a90 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedaffd640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7597170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdee4a16740 10 client.0 _ll_register_callbacks cb 0x7fded400e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 -- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 msgr2=0x55c3f75966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 0x55c3f75966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 -- 172.21.15.87:0/4220382239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 msgr2=0x55c3f7597170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7597170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f75fce80 con 0x55c3f7578a90 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0043040 con 0x55c3f7578a90 2024-06-06T08:07:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdedb7fe640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 0x55c3f7596c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c3f7578eb0 tx=0x7fded4003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50033b55484d3fe3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdebaffd640 10 client.27301 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdebaffd640 1 -- 172.21.15.87:0/4220382239 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4027070 con 0x55c3f7578a90 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdebaffd640 1 -- 172.21.15.87:0/4220382239 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fded4004b20 con 0x55c3f7578a90 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdebaffd640 1 -- 172.21.15.87:0/4220382239 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fded4004d60 con 0x55c3f7578a90 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdee4a16740 10 client.27301 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.408+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=55}) -- 0x55c3f75fe070 con 0x55c3f7578a90 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdebaffd640 1 -- 172.21.15.87:0/4220382239 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fded400b040 con 0x55c3f7578a90 2024-06-06T08:07:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 10 client.27301 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 10 client.27301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 20 client.27301 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f76006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 4 client.27301 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3f74e9c20 con 0x55c3f75932f0 2024-06-06T08:07:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.409+0000 7fdedaffd640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f76006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.410+0000 7fdebaffd640 10 client.27301 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:44.410+0000 7fdedaffd640 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f76006f0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1dcc16ec0e78be server_cookie=943a44c0aa08c4bb in_seq=0 out_seq=0 2024-06-06T08:07:45.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 82195 13.39 MB/sec execute 227 sec latency 597.968 ms 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.588+0000 7fdebaffd640 1 -- 172.21.15.87:0/4220382239 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3f74e9c20 con 0x55c3f75932f0 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.588+0000 7fdebaffd640 10 client.27301 handle_command_reply: tid=0 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.588+0000 7fdee4a16740 1 client.27301 shutdown 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 msgr2=0x55c3f7596c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 0x55c3f7596c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c3f7578eb0 tx=0x7fded4003d40 comp rx=0 tx=0).stop 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 shutdown_connections 2024-06-06T08:07:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c3f75932f0 0x55c3f76006f0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c3f7583350 0x55c3f7597170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3f7578a90 0x55c3f7596c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 --2- 172.21.15.87:0/4220382239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3f7578170 0x55c3f75966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 >> 172.21.15.87:0/4220382239 conn(0x55c3f7574680 msgr2=0x55c3f75823e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdebaffd640 0 client.27301 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 shutdown_connections 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 1 -- 172.21.15.87:0/4220382239 wait complete. 2024-06-06T08:07:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.589+0000 7fdee4a16740 20 client.27301 trim_cache size 0 max 16384 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 msgr2=0x7fdec40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 0x7fdec40a52e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0ccf50 tx=0x7fdecc0c6800 comp rx=0 tx=0).stop 2024-06-06T08:07:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 msgr2=0x7fdedc1abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e970 tx=0x7fdecc00ee30 comp rx=0 tx=0).stop 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 shutdown_connections 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdec40a2e20 0x7fdec40a52e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdedc1ab920 0x7fdedc1abdd0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc11b180 0x7fdedc1a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 --2- 172.21.15.87:0/3826618739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdedc110190 0x7fdedc1a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 >> 172.21.15.87:0/3826618739 conn(0x7fdedc06ca70 msgr2=0x7fdedc10de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 shutdown_connections 2024-06-06T08:07:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:45.590+0000 7fdee1c37640 1 -- 172.21.15.87:0/3826618739 wait complete. 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 -- 172.21.15.87:0/2854711995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 msgr2=0x7ff9d41029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 --2- 172.21.15.87:0/2854711995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d41029d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009990 tx=0x7ff9c4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 -- 172.21.15.87:0/2854711995 shutdown_connections 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 --2- 172.21.15.87:0/2854711995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d41084f0 0x7ff9d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 --2- 172.21.15.87:0/2854711995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d4102f10 0x7ff9d410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 --2- 172.21.15.87:0/2854711995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d41029d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 -- 172.21.15.87:0/2854711995 >> 172.21.15.87:0/2854711995 conn(0x7ff9d4079680 msgr2=0x7ff9d4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 -- 172.21.15.87:0/2854711995 shutdown_connections 2024-06-06T08:07:45.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.857+0000 7ff9db418700 1 -- 172.21.15.87:0/2854711995 wait complete. 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 Processor -- start 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 -- start start 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d4102f10 0x7ff9d419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 0x7ff9d419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d41127c0 con 0x7ff9d4102550 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4112640 con 0x7ff9d4102f10 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9db418700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4112940 con 0x7ff9d41084f0 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9d91b4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:45.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9d91b4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41734/0 (socket says 172.21.15.87:41734) 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9d91b4700 1 -- 172.21.15.87:0/3703960283 learned_addr learned my addr 172.21.15.87:0/3703960283 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9d99b5700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 0x7ff9d419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.858+0000 7ff9d89b3700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d4102f10 0x7ff9d419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 -- 172.21.15.87:0/3703960283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 msgr2=0x7ff9d419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 0x7ff9d419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 -- 172.21.15.87:0/3703960283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d4102f10 msgr2=0x7ff9d419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d4102f10 0x7ff9d419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 -- 172.21.15.87:0/3703960283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d419fe90 con 0x7ff9d4102550 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d99b5700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 0x7ff9d419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9d91b4700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e9a0 tx=0x7ff9d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000ca10 con 0x7ff9d4102550 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d000cbb0 con 0x7ff9d4102550 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0018680 con 0x7ff9d4102550 2024-06-06T08:07:45.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d41a0180 con 0x7ff9d4102550 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.859+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d41a7a00 con 0x7ff9d4102550 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.860+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d4069180 con 0x7ff9d4102550 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.862+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff9d0010430 con 0x7ff9d4102550 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.863+0000 7ff9ca7fc700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 0x7ff9c00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.864+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7ff9d00cff20 con 0x7ff9d4102550 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.864+0000 7ff9d89b3700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 0x7ff9c00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.864+0000 7ff9d89b3700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 0x7ff9c00a52d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4101630 tx=0x7ff9c4005fd0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:45.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:45.868+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff9d009c6c0 con 0x7ff9d4102550 2024-06-06T08:07:46.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:45 smithi087 ceph-mon[32276]: pgmap v626: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-06-06T08:07:46.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 82357 13.35 MB/sec execute 228 sec latency 798.899 ms 2024-06-06T08:07:46.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.189+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9d419c1e0 con 0x7ff9d4102550 2024-06-06T08:07:46.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.190+0000 7ff9ca7fc700 1 -- 172.21.15.87:0/3703960283 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7ff9d009bfc0 con 0x7ff9d4102550 2024-06-06T08:07:46.189 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:07:46.190 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 msgr2=0x7ff9c00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 0x7ff9c00a52d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4101630 tx=0x7ff9c4005fd0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 msgr2=0x7ff9d419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e9a0 tx=0x7ff9d000ecb0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 shutdown_connections 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff9c00a2e10 0x7ff9c00a52d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d41084f0 0x7ff9d419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9d4102550 0x7ff9d419ae80 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 --2- 172.21.15.87:0/3703960283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d4102f10 0x7ff9d419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:46.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 >> 172.21.15.87:0/3703960283 conn(0x7ff9d4079680 msgr2=0x7ff9d4074950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:46.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 shutdown_connections 2024-06-06T08:07:46.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:46.192+0000 7ff9db418700 1 -- 172.21.15.87:0/3703960283 wait complete. 2024-06-06T08:07:46.193 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:07:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:45 smithi195 ceph-mon[37040]: pgmap v626: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-06-06T08:07:46.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:45 smithi181 ceph-mon[36790]: pgmap v626: 129 pgs: 129 active+clean; 549 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-06-06T08:07:46.888 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-06T08:07:46.888 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-06T08:07:47.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:46 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3703960283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:07:47.173 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:07:47.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 82712 13.34 MB/sec execute 229 sec latency 205.424 ms 2024-06-06T08:07:47.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:46 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3703960283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:07:47.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:46 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3703960283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:07:48.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83099 13.39 MB/sec execute 230 sec latency 175.923 ms 2024-06-06T08:07:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:47 smithi195 ceph-mon[37040]: pgmap v627: 129 pgs: 129 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 350 op/s 2024-06-06T08:07:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:47 smithi181 ceph-mon[36790]: pgmap v627: 129 pgs: 129 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 350 op/s 2024-06-06T08:07:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:47 smithi087 ceph-mon[32276]: pgmap v627: 129 pgs: 129 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 350 op/s 2024-06-06T08:07:49.180 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83523 13.44 MB/sec execute 231 sec latency 205.286 ms 2024-06-06T08:07:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 -- 172.21.15.87:0/1068124544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 msgr2=0x7fbaec1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/1068124544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec1019d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbae80099c0 tx=0x7fbae8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 -- 172.21.15.87:0/1068124544 shutdown_connections 2024-06-06T08:07:49.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/1068124544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec1066c0 0x7fbaec10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/1068124544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec1019d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/1068124544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec101fa0 0x7fbaec105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 -- 172.21.15.87:0/1068124544 >> 172.21.15.87:0/1068124544 conn(0x7fbaec078330 msgr2=0x7fbaec078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:49.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 -- 172.21.15.87:0/1068124544 shutdown_connections 2024-06-06T08:07:49.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.489+0000 7fbaf3b22700 1 -- 172.21.15.87:0/1068124544 wait complete. 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 Processor -- start 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 -- start start 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec101fa0 0x7fbaec19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec1066c0 0x7fbaec198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec112800 con 0x7fbaec1015f0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec112680 con 0x7fbaec101fa0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf3b22700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec112980 con 0x7fbaec1066c0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48744/0 (socket says 172.21.15.87:48744) 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf18be700 1 -- 172.21.15.87:0/3655730792 learned_addr learned my addr 172.21.15.87:0/3655730792 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.490+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec101fa0 0x7fbaec19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf20bf700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec1066c0 0x7fbaec198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 -- 172.21.15.87:0/3655730792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec1066c0 msgr2=0x7fbaec198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec1066c0 0x7fbaec198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 -- 172.21.15.87:0/3655730792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec101fa0 msgr2=0x7fbaec19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec101fa0 0x7fbaec19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 -- 172.21.15.87:0/3655730792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec199180 con 0x7fbaec1015f0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbae8006010 tx=0x7fbae8003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbae2ffd700 1 -- 172.21.15.87:0/3655730792 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8003680 con 0x7fbaec1015f0 2024-06-06T08:07:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec199410 con 0x7fbaec1015f0 2024-06-06T08:07:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbae2ffd700 1 -- 172.21.15.87:0/3655730792 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8003e50 con 0x7fbaec1015f0 2024-06-06T08:07:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbae2ffd700 1 -- 172.21.15.87:0/3655730792 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae800fe10 con 0x7fbaec1015f0 2024-06-06T08:07:49.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.491+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a56e0 con 0x7fbaec1015f0 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.493+0000 7fbae2ffd700 1 -- 172.21.15.87:0/3655730792 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbae800f460 con 0x7fbaec1015f0 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.494+0000 7fbae2ffd700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 0x7fbad80a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.494+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 0x7fbad80a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.494+0000 7fbae2ffd700 1 -- 172.21.15.87:0/3655730792 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fbae80d0540 con 0x7fbaec1015f0 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.495+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 0x7fbad80a5490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ffc90 tx=0x7fbadc009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.495+0000 7fbafe3a5b80 1 Processor -- start 2024-06-06T08:07:49.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 -- start start 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c19100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5b0b0f0 con 0x55f9c5c1e590 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5b699e0 con 0x55f9c5c18ce0 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5ad15f0 con 0x55f9c5c1eeb0 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48758/0 (socket says 172.21.15.87:48758) 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 -- 172.21.15.87:0/1700318509 learned_addr learned my addr 172.21.15.87:0/1700318509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf20bf700 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c19100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 -- 172.21.15.87:0/1700318509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1eeb0 msgr2=0x55f9c5c187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 -- 172.21.15.87:0/1700318509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 msgr2=0x55f9c5c19100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c19100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 -- 172.21.15.87:0/1700318509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c5ab8a60 con 0x55f9c5c1e590 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.496+0000 7fbaf18be700 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbae80990a0 tx=0x7fbae809c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4e7cb8028491ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbad7fff700 1 -- 172.21.15.87:0/1700318509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae80870d0 con 0x55f9c5c1e590 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbad7fff700 1 -- 172.21.15.87:0/1700318509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae809cdd0 con 0x55f9c5c1e590 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbad7fff700 1 -- 172.21.15.87:0/1700318509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae80d0160 con 0x55f9c5c1e590 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/1700318509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 msgr2=0x55f9c5c1e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbae80990a0 tx=0x7fbae809c5d0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/1700318509 shutdown_connections 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c19100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c1e970 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/1700318509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/1700318509 >> 172.21.15.87:0/1700318509 conn(0x55f9c5b45210 msgr2=0x55f9c5c13a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/1700318509 shutdown_connections 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.497+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/1700318509 wait complete. 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 -- start start 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 Processor -- start 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c22420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5b0a150 con 0x55f9c5c1e590 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5b659a0 con 0x55f9c5c1eeb0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5c25640 con 0x55f9c5c18ce0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbafe3a5b80 10 client.0 _ll_register_callbacks cb 0x55f9c5aff740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.498+0000 7fbaf10bd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48764/0 (socket says 172.21.15.87:48764) 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 -- 172.21.15.87:0/3206286880 learned_addr learned my addr 172.21.15.87:0/3206286880 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf20bf700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c22420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 -- 172.21.15.87:0/3206286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c18ce0 msgr2=0x55f9c5c24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 -- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 msgr2=0x55f9c5c22420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c22420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 -- 172.21.15.87:0/3206286880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c5b2f8e0 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbadc004460 tx=0x7fbadc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40866c22f0323241 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004830 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae809cdd0 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fbadc0049d0 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbadc004bd0 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbafe3a5b80 10 client.18216 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.499+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f9c5c2fd50 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae809cdd0 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fbadc026030 con 0x55f9c5c1e590 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 10 client.18216 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 10 client.18216 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:07:49.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 20 client.18216 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c34720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 4 client.18216 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c5b76ee0 con 0x55f9c5c32330 2024-06-06T08:07:49.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.500+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c34720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.501+0000 7fbad57fa700 10 client.18216 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.501+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c34720 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ded9f719f5b7be server_cookie=63b4612c9fdf8ae1 in_seq=0 out_seq=0 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbad57fa700 1 -- 172.21.15.87:0/3206286880 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9c5b76ee0 con 0x55f9c5c32330 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbad57fa700 10 client.18216 handle_command_reply: tid=0 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 client.18216 shutdown 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 msgr2=0x55f9c5c21ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbadc004460 tx=0x7fbadc0047c0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 shutdown_connections 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c34720 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c22420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbad57fa700 0 client.18216 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3206286880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1e590 0x55f9c5c21ee0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.503+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 >> 172.21.15.87:0/3206286880 conn(0x55f9c5b45210 msgr2=0x55f9c5c123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.504+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 shutdown_connections 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.504+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3206286880 wait complete. 2024-06-06T08:07:49.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.504+0000 7fbafe3a5b80 20 client.18216 trim_cache size 0 max 16384 2024-06-06T08:07:49.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.528+0000 7fbafe3a5b80 1 Processor -- start 2024-06-06T08:07:49.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.528+0000 7fbafe3a5b80 1 -- start start 2024-06-06T08:07:49.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c24a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1b0f20 con 0x55f9c5c18ce0 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ae7b0 con 0x55f9c5c1eeb0 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ac620 con 0x55f9c5c1e590 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf20bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c24a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48772/0 (socket says 172.21.15.87:48772) 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 -- 172.21.15.87:0/2029036991 learned_addr learned my addr 172.21.15.87:0/2029036991 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:49.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 -- 172.21.15.87:0/2029036991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 msgr2=0x55f9c5c244f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c244f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 -- 172.21.15.87:0/2029036991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 msgr2=0x55f9c5c24a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c24a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 -- 172.21.15.87:0/2029036991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c5c20450 con 0x55f9c5c18ce0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbaf18be700 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fbae809c3d0 tx=0x7fbae800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82fb337a695bda server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.529+0000 7fbad77fe700 1 -- 172.21.15.87:0/2029036991 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae809cdd0 con 0x55f9c5c18ce0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbad77fe700 1 -- 172.21.15.87:0/2029036991 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae809c6d0 con 0x55f9c5c18ce0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbad77fe700 1 -- 172.21.15.87:0/2029036991 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae80d0160 con 0x55f9c5c18ce0 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/2029036991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 msgr2=0x55f9c5c187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fbae809c3d0 tx=0x7fbae800f870 comp rx=0 tx=0).stop 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/2029036991 shutdown_connections 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c24a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c187a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/2029036991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.530+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/2029036991 >> 172.21.15.87:0/2029036991 conn(0x55f9c5b45210 msgr2=0x55f9c5c12c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/2029036991 shutdown_connections 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/2029036991 wait complete. 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- start start 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 Processor -- start 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c2a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c2add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1acc70 con 0x55f9c5c1eeb0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ae0e0 con 0x55f9c5c18ce0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.531+0000 7fbafe3a5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ad340 con 0x55f9c5c1e590 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbafe3a5b80 10 client.0 _ll_register_callbacks cb 0x55f9c5aff740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbaf18be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c2a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbaf20bf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbaf20bf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48786/0 (socket says 172.21.15.87:48786) 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbaf20bf700 1 -- 172.21.15.87:0/3761792689 learned_addr learned my addr 172.21.15.87:0/3761792689 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.532+0000 7fbaf18be700 1 -- 172.21.15.87:0/3761792689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 msgr2=0x55f9c5c2add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c2add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c2add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf18be700 1 -- 172.21.15.87:0/3761792689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 msgr2=0x55f9c5c27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf18be700 1 -- 172.21.15.87:0/3761792689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c5c27e20 con 0x55f9c5c18ce0 2024-06-06T08:07:49.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf20bf700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c2add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbaf18be700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c2a890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbae80cde70 tx=0x7fbae80096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c560d7019c5a1b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae80b9070 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.533+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7fbae8020370 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae809c6d0 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbae8020570 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 10 client.27164 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f9c5c9d810 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae80d0160 con 0x55f9c5c18ce0 2024-06-06T08:07:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fbae8063c80 con 0x55f9c5c18ce0 2024-06-06T08:07:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 10 client.27164 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 10 client.27164 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.534+0000 7fbafe3a5b80 20 client.27164 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c9fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbafe3a5b80 4 client.27164 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fbae809c6d0 con 0x55f9c5c32330 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c9fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbad57fa700 10 client.27164 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:49.535+0000 7fbaf10bd700 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c9fbb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6176dbd9ff0793f server_cookie=9e2f8078887b7bc8 in_seq=0 out_seq=0 2024-06-06T08:07:50.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83739 13.46 MB/sec execute 232 sec latency 589.919 ms 2024-06-06T08:07:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:49 smithi195 ceph-mon[37040]: pgmap v628: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-06-06T08:07:50.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:49 smithi181 ceph-mon[36790]: pgmap v628: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-06-06T08:07:50.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:49 smithi087 ceph-mon[32276]: pgmap v628: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-06-06T08:07:51.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.083+0000 7fbad57fa700 1 -- 172.21.15.87:0/3761792689 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fbae809c6d0 con 0x55f9c5c32330 2024-06-06T08:07:51.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.083+0000 7fbad57fa700 10 client.27164 handle_command_reply: tid=0 2024-06-06T08:07:51.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.083+0000 7fbafe3a5b80 1 client.27164 shutdown 2024-06-06T08:07:51.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 msgr2=0x55f9c5c2a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:51.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c2a890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbae80cde70 tx=0x7fbae80096e0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 shutdown_connections 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f9c5c32330 0x55f9c5c9fbb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9c5c18ce0 0x55f9c5c2a890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9c5c1eeb0 0x55f9c5c27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbad57fa700 0 client.27164 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 --2- 172.21.15.87:0/3761792689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c5c1e590 0x55f9c5c2add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 >> 172.21.15.87:0/3761792689 conn(0x55f9c5b45210 msgr2=0x55f9c5c21790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 shutdown_connections 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 1 -- 172.21.15.87:0/3761792689 wait complete. 2024-06-06T08:07:51.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.084+0000 7fbafe3a5b80 20 client.27164 trim_cache size 0 max 16384 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 62, 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "bbd4b373": { 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 62, 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:07:51.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:07:51.155 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 msgr2=0x7fbad80a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 0x7fbad80a5490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ffc90 tx=0x7fbadc009260 comp rx=0 tx=0).stop 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 msgr2=0x7fbaec19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbae8006010 tx=0x7fbae8003a90 comp rx=0 tx=0).stop 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 shutdown_connections 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fbad80a2fd0 0x7fbad80a5490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaec1066c0 0x7fbaec198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaec1015f0 0x7fbaec19e810 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 --2- 172.21.15.87:0/3655730792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaec101fa0 0x7fbaec19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 >> 172.21.15.87:0/3655730792 conn(0x7fbaec078330 msgr2=0x7fbaec100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 shutdown_connections 2024-06-06T08:07:51.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:07:51.085+0000 7fbaf3b22700 1 -- 172.21.15.87:0/3655730792 wait complete. 2024-06-06T08:07:51.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.40 MB/sec execute 233 sec latency 1376.905 ms 2024-06-06T08:07:51.759 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bbd4b373', quiesce took 8.1 seconds. Will release after: 21.9 2024-06-06T08:07:52.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:51 smithi087 ceph-mon[32276]: pgmap v629: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-06-06T08:07:52.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.34 MB/sec execute 234 sec latency 2377.012 ms 2024-06-06T08:07:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:51 smithi195 ceph-mon[37040]: pgmap v629: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-06-06T08:07:52.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:51 smithi181 ceph-mon[36790]: pgmap v629: 129 pgs: 129 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-06-06T08:07:53.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.29 MB/sec execute 235 sec latency 3377.097 ms 2024-06-06T08:07:54.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.23 MB/sec execute 236 sec latency 4377.211 ms 2024-06-06T08:07:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:53 smithi195 ceph-mon[37040]: pgmap v630: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-06-06T08:07:54.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:53 smithi181 ceph-mon[36790]: pgmap v630: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-06-06T08:07:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:53 smithi087 ceph-mon[32276]: pgmap v630: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-06-06T08:07:55.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.18 MB/sec execute 237 sec latency 5377.305 ms 2024-06-06T08:07:56.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.12 MB/sec execute 238 sec latency 6377.373 ms 2024-06-06T08:07:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:55 smithi195 ceph-mon[37040]: pgmap v631: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-06-06T08:07:56.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:55 smithi181 ceph-mon[36790]: pgmap v631: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-06-06T08:07:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:55 smithi087 ceph-mon[32276]: pgmap v631: 129 pgs: 129 active+clean; 542 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-06-06T08:07:57.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.06 MB/sec execute 239 sec latency 7377.480 ms 2024-06-06T08:07:58.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:57 smithi087 ceph-mon[32276]: pgmap v632: 129 pgs: 129 active+clean; 543 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-06-06T08:07:58.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 13.01 MB/sec execute 240 sec latency 8377.578 ms 2024-06-06T08:07:58.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:57 smithi195 ceph-mon[37040]: pgmap v632: 129 pgs: 129 active+clean; 543 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-06-06T08:07:58.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:57 smithi181 ceph-mon[36790]: pgmap v632: 129 pgs: 129 active+clean; 543 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 -- 172.21.15.87:0/3959894471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1298102c70 msgr2=0x7f1298103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 --2- 172.21.15.87:0/3959894471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1298102c70 0x7f1298103050 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1288009900 tx=0x7f128802a480 comp rx=0 tx=0).stop 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 -- 172.21.15.87:0/3959894471 shutdown_connections 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 --2- 172.21.15.87:0/3959894471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f129810bfb0 0x7f129810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 --2- 172.21.15.87:0/3959894471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 0x7f129810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 --2- 172.21.15.87:0/3959894471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1298102c70 0x7f1298103050 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 -- 172.21.15.87:0/3959894471 >> 172.21.15.87:0/3959894471 conn(0x7f1298077a20 msgr2=0x7f1298077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 -- 172.21.15.87:0/3959894471 shutdown_connections 2024-06-06T08:07:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.690+0000 7f129cd09640 1 -- 172.21.15.87:0/3959894471 wait complete. 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 Processor -- start 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 -- start start 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 0x7f12981008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 0x7f1298100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1298111d60 con 0x7f129810bfb0 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1298111be0 con 0x7f1298102c70 2024-06-06T08:07:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129cd09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1298111ee0 con 0x7f1298103590 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54802/0 (socket says 172.21.15.87:54802) 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 -- 172.21.15.87:0/2810515460 learned_addr learned my addr 172.21.15.87:0/2810515460 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129659c640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 0x7f12981008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f128ffff640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 0x7f1298100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 -- 172.21.15.87:0/2810515460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 msgr2=0x7f1298100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 0x7f1298100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 -- 172.21.15.87:0/2810515460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 msgr2=0x7f12981008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 0x7f12981008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f1296d9d640 1 -- 172.21.15.87:0/2810515460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12981a6660 con 0x7f129810bfb0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.691+0000 7f129659c640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 0x7f12981008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f1296d9d640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f129000cce0 tx=0x7f129000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f128f7fe640 1 -- 172.21.15.87:0/2810515460 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129000f450 con 0x7f129810bfb0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f128f7fe640 1 -- 172.21.15.87:0/2810515460 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129000fad0 con 0x7f129810bfb0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f128f7fe640 1 -- 172.21.15.87:0/2810515460 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290018c10 con 0x7f129810bfb0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981a6950 con 0x7f129810bfb0 2024-06-06T08:07:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.692+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a6e50 con 0x7f129810bfb0 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.694+0000 7f128f7fe640 1 -- 172.21.15.87:0/2810515460 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f129000f5f0 con 0x7f129810bfb0 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.694+0000 7f129fa0f740 1 Processor -- start 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 -- start start 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f128f7fe640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 0x7f12840a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cc5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5603e7c1f4f0 con 0x5603e7cc6350 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5603e7bbd870 con 0x5603e7cbb170 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129fa0f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5603e7c1b4b0 con 0x5603e7cbba90 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f129659c640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 0x7f12840a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54810/0 (socket says 172.21.15.87:54810) 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 -- 172.21.15.87:0/1047945688 learned_addr learned my addr 172.21.15.87:0/1047945688 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f128f7fe640 1 -- 172.21.15.87:0/2810515460 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f12900cff10 con 0x7f129810bfb0 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f128ffff640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f128ffff640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cc5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 -- 172.21.15.87:0/1047945688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 msgr2=0x5603e7cc5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cc5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 -- 172.21.15.87:0/1047945688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 msgr2=0x5603e7cbb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cbb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f1296d9d640 1 -- 172.21.15.87:0/1047945688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e7bd99b0 con 0x5603e7cc6350 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.695+0000 7f128ffff640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cc5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.696+0000 7f129659c640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 0x7f12840a5290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1288005ab0 tx=0x7f1288005a20 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.696+0000 7f1296d9d640 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f129001f000 tx=0x7f12900186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9bff1c36a5e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.696+0000 7f128cff9640 1 -- 172.21.15.87:0/1047945688 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900aed30 con 0x5603e7cc6350 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.696+0000 7f128cff9640 1 -- 172.21.15.87:0/1047945688 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12900bc0e0 con 0x5603e7cc6350 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.696+0000 7f128cff9640 1 -- 172.21.15.87:0/1047945688 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900aeb50 con 0x5603e7cc6350 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- 172.21.15.87:0/1047945688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 msgr2=0x5603e7cc8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f129001f000 tx=0x7f12900186b0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- 172.21.15.87:0/1047945688 shutdown_connections 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cc8740 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cc5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1047945688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cbb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- 172.21.15.87:0/1047945688 >> 172.21.15.87:0/1047945688 conn(0x5603e7cb7680 msgr2=0x5603e7cb7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- 172.21.15.87:0/1047945688 shutdown_connections 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- 172.21.15.87:0/1047945688 wait complete. 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- start start 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 Processor -- start 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbba90 0x5603e7cc3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 0x5603e7cc3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5603e7cc8f80 con 0x5603e7cbb170 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5603e7ba8ae0 con 0x5603e7cbba90 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.697+0000 7f129fa0f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5603e7cc9100 con 0x5603e7cc6350 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129fa0f740 10 client.0 _ll_register_callbacks cb 0x5603e7be19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54824/0 (socket says 172.21.15.87:54824) 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 -- 172.21.15.87:0/2336935814 learned_addr learned my addr 172.21.15.87:0/2336935814 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f128ffff640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbba90 0x5603e7cc3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f1296d9d640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 0x5603e7cc3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 -- 172.21.15.87:0/2336935814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 msgr2=0x5603e7cc3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 0x5603e7cc3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 -- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbba90 msgr2=0x5603e7cc3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbba90 0x5603e7cc3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 -- 172.21.15.87:0/2336935814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e7933210 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f1296d9d640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 0x5603e7cc3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129659c640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1288033000 tx=0x7f1288009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64382a7ab5faa5f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f127dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128804b070 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f1288003af0 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12900bc0e0 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1288057050 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129fa0f740 10 client.18234 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.698+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5603e79ff620 con 0x5603e7cbb170 2024-06-06T08:07:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12900bc0e0 con 0x5603e7cbb170 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f128803b430 con 0x5603e7cbb170 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 10 client.18234 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 20 client.18234 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7cd65a0 0x5603e7cd8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 4 client.18234 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e7c32f10 con 0x5603e7cd65a0 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.699+0000 7f128ffff640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7cd65a0 0x5603e7cd8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.700+0000 7f128ffff640 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7cd65a0 0x5603e7cd8990 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7e2d2fcd98fec1 server_cookie=c11dad660252a810 in_seq=0 out_seq=0 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.700+0000 7f127dffb640 10 client.18234 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.700+0000 7f127dffb640 1 -- 172.21.15.87:0/2336935814 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603e7c32f10 con 0x5603e7cd65a0 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.700+0000 7f127dffb640 10 client.18234 handle_command_reply: tid=0 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.700+0000 7f129fa0f740 1 client.18234 shutdown 2024-06-06T08:07:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 msgr2=0x5603e7cc32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1288033000 tx=0x7f1288009260 comp rx=0 tx=0).stop 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 shutdown_connections 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7cd65a0 0x5603e7cd8990 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cc6350 0x5603e7cc3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbba90 0x5603e7cc3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f127dffb640 0 client.18234 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 --2- 172.21.15.87:0/2336935814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7cc32f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 >> 172.21.15.87:0/2336935814 conn(0x5603e7cb7680 msgr2=0x5603e7cbd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 shutdown_connections 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 1 -- 172.21.15.87:0/2336935814 wait complete. 2024-06-06T08:07:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.701+0000 7f129fa0f740 20 client.18234 trim_cache size 0 max 16384 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.716+0000 7f129fa0f740 1 Processor -- start 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.716+0000 7f129fa0f740 1 -- start start 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.716+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cc3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cb6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129fa0f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12981b3ab0 con 0x5603e7cc6350 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129fa0f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f12981b1380 con 0x5603e7cbb170 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129fa0f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12981af1f0 con 0x5603e7cbba90 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54826/0 (socket says 172.21.15.87:54826) 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 -- 172.21.15.87:0/77389344 learned_addr learned my addr 172.21.15.87:0/77389344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f128ffff640 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cb6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f129659c640 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cc3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 -- 172.21.15.87:0/77389344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 msgr2=0x5603e7cb6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cb6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 -- 172.21.15.87:0/77389344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 msgr2=0x5603e7cc3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cc3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 -- 172.21.15.87:0/77389344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e7cc1840 con 0x5603e7cc6350 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f1296d9d640 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f129001f000 tx=0x7f12900a4c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dd4f33938397c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f127ffff640 1 -- 172.21.15.87:0/77389344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129000f8e0 con 0x5603e7cc6350 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f127ffff640 1 -- 172.21.15.87:0/77389344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129009c010 con 0x5603e7cc6350 2024-06-06T08:07:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.717+0000 7f127ffff640 1 -- 172.21.15.87:0/77389344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290098680 con 0x5603e7cc6350 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- 172.21.15.87:0/77389344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 msgr2=0x5603e7cde230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f129001f000 tx=0x7f12900a4c80 comp rx=0 tx=0).stop 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- 172.21.15.87:0/77389344 shutdown_connections 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cc6350 0x5603e7cde230 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7cb6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 --2- 172.21.15.87:0/77389344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cbb170 0x5603e7cc3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- 172.21.15.87:0/77389344 >> 172.21.15.87:0/77389344 conn(0x5603e7cb7680 msgr2=0x5603e7cb5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- 172.21.15.87:0/77389344 shutdown_connections 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- 172.21.15.87:0/77389344 wait complete. 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 -- start start 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.718+0000 7f129fa0f740 1 Processor -- start 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7d39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cc6350 0x5603e7cc98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12981af840 con 0x5603e7cbb170 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f12981b0cb0 con 0x5603e7cc6350 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12981aff10 con 0x5603e7cbba90 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 10 client.0 _ll_register_callbacks cb 0x7f128803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54828/0 (socket says 172.21.15.87:54828) 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 -- 172.21.15.87:0/1447297324 learned_addr learned my addr 172.21.15.87:0/1447297324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f128ffff640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7d39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f1296d9d640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cc6350 0x5603e7cc98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 -- 172.21.15.87:0/1447297324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 msgr2=0x5603e7d39a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7d39a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 -- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cc6350 msgr2=0x5603e7cc98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:07:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cc6350 0x5603e7cc98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 -- 172.21.15.87:0/1447297324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e7cc9fc0 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129659c640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1288009230 tx=0x7f1288031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cbd83dba980d43 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f127dffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128804b040 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9583+0+0 (secure 0 0 0) 0x7f128803ba70 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129009c010 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f128806d050 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 10 client.18240 fetch_fsmap learned FSMap version 55 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.719+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5603e7d40cb0 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129009c010 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1288050670 con 0x5603e7cbb170 2024-06-06T08:07:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 10 client.18240 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 10 client.18240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 20 client.18240 populate_metadata read hostname 'smithi087' 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7d41430 0x5603e7d43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 4 client.18240 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603e7c2cc20 con 0x5603e7d41430 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.720+0000 7f128ffff640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7d41430 0x5603e7d43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.721+0000 7f127dffb640 10 client.18240 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:58.721+0000 7f128ffff640 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7d41430 0x5603e7d43850 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b57f5bbca2e552c server_cookie=c1442f8dc211505 in_seq=0 out_seq=0 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f127dffb640 1 -- 172.21.15.87:0/1447297324 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603e7c2cc20 con 0x5603e7d41430 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f127dffb640 10 client.18240 handle_command_reply: tid=0 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 client.18240 shutdown 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 msgr2=0x5603e7d39500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1288009230 tx=0x7f1288031b60 comp rx=0 tx=0).stop 2024-06-06T08:07:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 shutdown_connections 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5603e7d41430 0x5603e7d43850 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603e7cc6350 0x5603e7cc98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603e7cbba90 0x5603e7d39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 --2- 172.21.15.87:0/1447297324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5603e7cbb170 0x5603e7d39500 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 >> 172.21.15.87:0/1447297324 conn(0x5603e7cb7680 msgr2=0x5603e7cc53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f127dffb640 0 client.18240 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 shutdown_connections 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 1 -- 172.21.15.87:0/1447297324 wait complete. 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.072+0000 7f129fa0f740 20 client.18240 trim_cache size 0 max 16384 2024-06-06T08:07:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:07:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:07:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:07:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 msgr2=0x7f12840a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 0x7f12840a5290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1288005ab0 tx=0x7f1288005a20 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 msgr2=0x7f1298101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f129000cce0 tx=0x7f129000a620 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 shutdown_connections 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f12840a2dd0 0x7f12840a5290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f129810bfb0 0x7f1298101340 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1298103590 0x7f1298100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 --2- 172.21.15.87:0/2810515460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1298102c70 0x7f12981008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 >> 172.21.15.87:0/2810515460 conn(0x7f1298077a20 msgr2=0x7f12980fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 shutdown_connections 2024-06-06T08:07:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:07:59.073+0000 7f129cd09640 1 -- 172.21.15.87:0/2810515460 wait complete. 2024-06-06T08:07:59.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.96 MB/sec execute 241 sec latency 9377.680 ms 2024-06-06T08:08:00.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.90 MB/sec execute 242 sec latency 10377.720 ms 2024-06-06T08:08:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:07:59 smithi195 ceph-mon[37040]: pgmap v633: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-06-06T08:08:00.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:07:59 smithi181 ceph-mon[36790]: pgmap v633: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-06-06T08:08:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:07:59 smithi087 ceph-mon[32276]: pgmap v633: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-06-06T08:08:01.181 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.85 MB/sec execute 243 sec latency 11377.768 ms 2024-06-06T08:08:02.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.80 MB/sec execute 244 sec latency 12377.840 ms 2024-06-06T08:08:02.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:01 smithi195 ceph-mon[37040]: pgmap v634: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 213 op/s 2024-06-06T08:08:02.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:01 smithi181 ceph-mon[36790]: pgmap v634: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 213 op/s 2024-06-06T08:08:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:01 smithi087 ceph-mon[32276]: pgmap v634: 129 pgs: 129 active+clean; 542 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 213 op/s 2024-06-06T08:08:03.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.74 MB/sec execute 245 sec latency 13377.912 ms 2024-06-06T08:08:04.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.69 MB/sec execute 246 sec latency 14377.969 ms 2024-06-06T08:08:04.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:03 smithi195 ceph-mon[37040]: pgmap v635: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-06-06T08:08:04.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:03 smithi181 ceph-mon[36790]: pgmap v635: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-06-06T08:08:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:03 smithi087 ceph-mon[32276]: pgmap v635: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 396 op/s 2024-06-06T08:08:05.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.64 MB/sec execute 247 sec latency 15378.010 ms 2024-06-06T08:08:06.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.59 MB/sec execute 248 sec latency 16378.054 ms 2024-06-06T08:08:06.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:05 smithi195 ceph-mon[37040]: pgmap v636: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-06-06T08:08:06.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:05 smithi181 ceph-mon[36790]: pgmap v636: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-06-06T08:08:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:05 smithi087 ceph-mon[32276]: pgmap v636: 129 pgs: 129 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-06-06T08:08:07.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.54 MB/sec execute 249 sec latency 17378.102 ms 2024-06-06T08:08:07.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:06 smithi195 ceph-mon[37040]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T08:08:07.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:06 smithi181 ceph-mon[36790]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T08:08:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:06 smithi087 ceph-mon[32276]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T08:08:08.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.49 MB/sec execute 250 sec latency 18378.147 ms 2024-06-06T08:08:08.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:07 smithi195 ceph-mon[37040]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-06-06T08:08:08.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:07 smithi181 ceph-mon[36790]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-06-06T08:08:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:07 smithi087 ceph-mon[32276]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-06-06T08:08:09.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.44 MB/sec execute 251 sec latency 19378.191 ms 2024-06-06T08:08:10.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.39 MB/sec execute 252 sec latency 20378.235 ms 2024-06-06T08:08:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:09 smithi195 ceph-mon[37040]: pgmap v639: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:10.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:09 smithi181 ceph-mon[36790]: pgmap v639: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:09 smithi087 ceph-mon[32276]: pgmap v639: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:11.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.34 MB/sec execute 253 sec latency 21378.281 ms 2024-06-06T08:08:12.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.29 MB/sec execute 254 sec latency 22378.322 ms 2024-06-06T08:08:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.215+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/1653297356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 msgr2=0x7f7b54106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.215+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/1653297356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54106c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009be0 tx=0x7f7b5002a6d0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.221+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/1653297356 shutdown_connections 2024-06-06T08:08:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.221+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/1653297356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 0x7f7b5410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.221+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/1653297356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541008d0 0x7f7b54100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.221+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/1653297356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54106c70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.221+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/1653297356 >> 172.21.15.87:0/1653297356 conn(0x7f7b54077a20 msgr2=0x7f7b54077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.227+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/1653297356 shutdown_connections 2024-06-06T08:08:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.228+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/1653297356 wait complete. 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 Processor -- start 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 -- start start 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541008d0 0x7f7b5410d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 0x7f7b54109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54110ca0 con 0x7f7b541008d0 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54110b20 con 0x7f7b54106890 2024-06-06T08:08:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.229+0000 7f7b5bbe2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54110e20 con 0x7f7b54101290 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56018/0 (socket says 172.21.15.87:56018) 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 -- 172.21.15.87:0/2261124208 learned_addr learned my addr 172.21.15.87:0/2261124208 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5917d640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 0x7f7b54109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 -- 172.21.15.87:0/2261124208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 msgr2=0x7f7b54109570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 0x7f7b54109570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 -- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541008d0 msgr2=0x7f7b5410d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541008d0 0x7f7b5410d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.230+0000 7f7b5a17f640 1 -- 172.21.15.87:0/2261124208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5410a370 con 0x7f7b54106890 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.231+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00d750 tx=0x7f7b4c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.233+0000 7f7b4affd640 1 -- 172.21.15.87:0/2261124208 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c017070 con 0x7f7b54106890 2024-06-06T08:08:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.233+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b541aae40 con 0x7f7b54106890 2024-06-06T08:08:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.233+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541ab2a0 con 0x7f7b54106890 2024-06-06T08:08:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.234+0000 7f7b4affd640 1 -- 172.21.15.87:0/2261124208 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c005500 con 0x7f7b54106890 2024-06-06T08:08:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.235+0000 7f7b4affd640 1 -- 172.21.15.87:0/2261124208 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004d60 con 0x7f7b54106890 2024-06-06T08:08:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.239+0000 7f7b4affd640 1 -- 172.21.15.87:0/2261124208 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7b4c009bd0 con 0x7f7b54106890 2024-06-06T08:08:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.241+0000 7f7b5e9c5740 1 Processor -- start 2024-06-06T08:08:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.242+0000 7f7b5e9c5740 1 -- start start 2024-06-06T08:08:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.242+0000 7f7b4affd640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 0x7f7b400a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b4affd640 1 -- 172.21.15.87:0/2261124208 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9671+0+0 (secure 0 0 0) 0x7f7b4c0cf220 con 0x7f7b54106890 2024-06-06T08:08:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af352a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af3534e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 0x560af3537740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560af348e4f0 con 0x560af352aa90 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560af342c870 con 0x560af3535350 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.243+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560af348a4b0 con 0x560af352a170 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.244+0000 7f7b5917d640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 0x7f7b400a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.244+0000 7f7b5917d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af3534e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.244+0000 7f7b5917d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af3534e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44386/0 (socket says 172.21.15.87:44386) 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.244+0000 7f7b5917d640 1 -- 172.21.15.87:0/1031553870 learned_addr learned my addr 172.21.15.87:0/1031553870 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.244+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 0x560af3537740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 -- 172.21.15.87:0/1031553870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 msgr2=0x560af352a550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:08:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af352a550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 -- 172.21.15.87:0/1031553870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 msgr2=0x560af3534e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af3534e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 -- 172.21.15.87:0/1031553870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af34489b0 con 0x560af3535350 2024-06-06T08:08:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.245+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 0x560af3537740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0cc490 tx=0x7f7b4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743bf0554550f8b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.246+0000 7f7b5917d640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 0x7f7b400a5310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b44016560 tx=0x7f7b44009870 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.246+0000 7f7b3ffff640 1 -- 172.21.15.87:0/1031553870 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c020040 con 0x560af3535350 2024-06-06T08:08:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.246+0000 7f7b3ffff640 1 -- 172.21.15.87:0/1031553870 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c09b260 con 0x560af3535350 2024-06-06T08:08:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.246+0000 7f7b3ffff640 1 -- 172.21.15.87:0/1031553870 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0ccbe0 con 0x560af3535350 2024-06-06T08:08:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/1031553870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 msgr2=0x560af3537740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 0x560af3537740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0cc490 tx=0x7f7b4c004730 comp rx=0 tx=0).stop 2024-06-06T08:08:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/1031553870 shutdown_connections 2024-06-06T08:08:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af3535350 0x560af3537740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af3534e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/1031553870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af352a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/1031553870 >> 172.21.15.87:0/1031553870 conn(0x560af3526680 msgr2=0x560af3526a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/1031553870 shutdown_connections 2024-06-06T08:08:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.249+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/1031553870 wait complete. 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.250+0000 7f7b5e9c5740 1 -- start start 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.250+0000 7f7b5e9c5740 1 Processor -- start 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 0x560af35322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af3532830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560af3537ec0 con 0x560af352a170 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560af3417ae0 con 0x560af352aa90 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560af3538040 con 0x560af3535350 2024-06-06T08:08:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5e9c5740 10 client.0 _ll_register_callbacks cb 0x560af34509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:44744/0 (socket says 172.21.15.87:44744) 2024-06-06T08:08:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5a17f640 1 -- 172.21.15.87:0/3274605076 learned_addr learned my addr 172.21.15.87:0/3274605076 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.251+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 0x560af35322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 -- 172.21.15.87:0/3274605076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 msgr2=0x560af3532d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 -- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 msgr2=0x560af3532830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af3532830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 -- 172.21.15.87:0/3274605076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af31a2210 con 0x560af352a170 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 0x560af35322f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009b30 tx=0x7f7b50030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6759525094f2af06 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.252+0000 7f7b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.256+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50040020 con 0x560af352a170 2024-06-06T08:08:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.257+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9671+0+0 (secure 0 0 0) 0x7f7b50003970 con 0x560af352a170 2024-06-06T08:08:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.257+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b4c09b260 con 0x560af352a170 2024-06-06T08:08:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.257+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7b5004b050 con 0x560af352a170 2024-06-06T08:08:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.257+0000 7f7b5e9c5740 10 client.18246 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.257+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560af326e620 con 0x560af352a170 2024-06-06T08:08:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c09b260 con 0x560af352a170 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7b500046a0 con 0x560af352a170 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 10 client.18246 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 10 client.18246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 20 client.18246 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af3545940 0x560af3547dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 4 client.18246 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.258+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af34a1f10 con 0x560af3545940 2024-06-06T08:08:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.259+0000 7f7b5917d640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af3545940 0x560af3547dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:12 smithi195 ceph-mon[37040]: pgmap v640: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.261+0000 7f7b5917d640 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af3545940 0x560af3547dc0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e9f87271c5201f server_cookie=13a76eff5c332a2b in_seq=0 out_seq=0 2024-06-06T08:08:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.261+0000 7f7b3d7fa640 10 client.18246 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.262+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3274605076 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560af34a1f10 con 0x560af3545940 2024-06-06T08:08:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.262+0000 7f7b3d7fa640 10 client.18246 handle_command_reply: tid=0 2024-06-06T08:08:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.263+0000 7f7b5e9c5740 1 client.18246 shutdown 2024-06-06T08:08:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.264+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 msgr2=0x560af35322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.264+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 0x560af35322f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009b30 tx=0x7f7b50030c50 comp rx=0 tx=0).stop 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 shutdown_connections 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af3545940 0x560af3547dc0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3532d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af3532830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3274605076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352a170 0x560af35322f0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 >> 172.21.15.87:0/3274605076 conn(0x560af3526680 msgr2=0x560af352c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b3d7fa640 0 client.18246 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 shutdown_connections 2024-06-06T08:08:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.265+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3274605076 wait complete. 2024-06-06T08:08:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.269+0000 7f7b5e9c5740 20 client.18246 trim_cache size 0 max 16384 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.298+0000 7f7b5e9c5740 1 Processor -- start 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.298+0000 7f7b5e9c5740 1 -- start start 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.298+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af352cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3540850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38012c40 con 0x560af352aa90 2024-06-06T08:08:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38010510 con 0x560af352a170 2024-06-06T08:08:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3800e380 con 0x560af3535350 2024-06-06T08:08:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:56042/0 (socket says 172.21.15.87:56042) 2024-06-06T08:08:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 -- 172.21.15.87:0/2005742776 learned_addr learned my addr 172.21.15.87:0/2005742776 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3540850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5917d640 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af352cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 -- 172.21.15.87:0/2005742776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 msgr2=0x560af3540850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3540850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 -- 172.21.15.87:0/2005742776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 msgr2=0x560af352cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af352cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 -- 172.21.15.87:0/2005742776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af352b540 con 0x560af352a170 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.299+0000 7f7b5997e640 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b50002230 tx=0x7f7b50005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67611aa892e12b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.300+0000 7f7b3f7fe640 1 -- 172.21.15.87:0/2005742776 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50040040 con 0x560af352a170 2024-06-06T08:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.300+0000 7f7b3f7fe640 1 -- 172.21.15.87:0/2005742776 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50030ec0 con 0x560af352a170 2024-06-06T08:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.300+0000 7f7b3f7fe640 1 -- 172.21.15.87:0/2005742776 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50036040 con 0x560af352a170 2024-06-06T08:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/2005742776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 msgr2=0x560af3534b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b50002230 tx=0x7f7b50005bd0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/2005742776 shutdown_connections 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af3535350 0x560af3540850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af352aa90 0x560af352cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/2005742776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352a170 0x560af3534b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/2005742776 >> 172.21.15.87:0/2005742776 conn(0x560af3526680 msgr2=0x560af3540140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/2005742776 shutdown_connections 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/2005742776 wait complete. 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 -- start start 2024-06-06T08:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.301+0000 7f7b5e9c5740 1 Processor -- start 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.302+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af353d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.302+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af353dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.302+0000 7f7b5e9c5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.302+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3800e9d0 con 0x560af3535350 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.302+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3800fe40 con 0x560af352aa90 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5e9c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3800f0a0 con 0x560af352a170 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44416/0 (socket says 172.21.15.87:44416) 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 -- 172.21.15.87:0/3142355676 learned_addr learned my addr 172.21.15.87:0/3142355676 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5e9c5740 10 client.0 _ll_register_callbacks cb 0x7f7b50034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5917d640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af353dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af353d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 -- 172.21.15.87:0/3142355676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 msgr2=0x560af353d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af353d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 -- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 msgr2=0x560af353dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af353dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 -- 172.21.15.87:0/3142355676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af352d0c0 con 0x560af3535350 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.303+0000 7f7b5a17f640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c097c30 tx=0x7f7b4c099770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87865eea6f6bc5d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.304+0000 7f7b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.304+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c020040 con 0x560af3535350 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.304+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9671+0+0 (secure 0 0 0) 0x7f7b4c0ccde0 con 0x560af3535350 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.304+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b50030ec0 con 0x560af3535350 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c0b4230 con 0x560af3535350 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b5e9c5740 10 client.18252 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560af35c10a0 con 0x560af3535350 2024-06-06T08:08:12.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:12 smithi181 ceph-mon[36790]: pgmap v640: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c07c020 con 0x560af3535350 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7b4c0bb0b0 con 0x560af3535350 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.305+0000 7f7b5e9c5740 10 client.18252 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.306+0000 7f7b5e9c5740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.306+0000 7f7b5e9c5740 20 client.18252 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.306+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af35c1330 0x560af35c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.306+0000 7f7b5e9c5740 4 client.18252 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:08:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.306+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7b50030ec0 con 0x560af35c1330 2024-06-06T08:08:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.311+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af35c1330 0x560af35c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.312+0000 7f7b5997e640 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af35c1330 0x560af35c3740 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ae9954e8f1e922 server_cookie=74d7f33a47e4fe82 in_seq=0 out_seq=0 2024-06-06T08:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.313+0000 7f7b3d7fa640 10 client.18252 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:12 smithi087 ceph-mon[32276]: pgmap v640: 129 pgs: 21 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 55 active+clean; 527 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.503+0000 7f7b3d7fa640 1 -- 172.21.15.87:0/3142355676 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f7b50030ec0 con 0x560af35c1330 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.503+0000 7f7b3d7fa640 10 client.18252 handle_command_reply: tid=0 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.503+0000 7f7b5e9c5740 1 client.18252 shutdown 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 msgr2=0x560af352ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c097c30 tx=0x7f7b4c099770 comp rx=0 tx=0).stop 2024-06-06T08:08:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 shutdown_connections 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560af35c1330 0x560af35c3740 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560af3535350 0x560af352ed00 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b3d7fa640 0 client.18252 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560af352aa90 0x560af353dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 --2- 172.21.15.87:0/3142355676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560af352a170 0x560af353d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 >> 172.21.15.87:0/3142355676 conn(0x560af3526680 msgr2=0x560af35409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 shutdown_connections 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.504+0000 7f7b5e9c5740 1 -- 172.21.15.87:0/3142355676 wait complete. 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5e9c5740 20 client.18252 trim_cache size 0 max 16384 2024-06-06T08:08:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 msgr2=0x7f7b400a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 0x7f7b400a5310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b44016560 tx=0x7f7b44009870 comp rx=0 tx=0).stop 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 msgr2=0x7f7b54109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00d750 tx=0x7f7b4c00dc10 comp rx=0 tx=0).stop 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 shutdown_connections 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7b400a2e50 0x7f7b400a5310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b54106890 0x7f7b54109ab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b54101290 0x7f7b54109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 --2- 172.21.15.87:0/2261124208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b541008d0 0x7f7b5410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.505+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 >> 172.21.15.87:0/2261124208 conn(0x7f7b54077a20 msgr2=0x7f7b54103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.506+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 shutdown_connections 2024-06-06T08:08:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:12.506+0000 7f7b5bbe2640 1 -- 172.21.15.87:0/2261124208 wait complete. 2024-06-06T08:08:13.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.25 MB/sec execute 255 sec latency 23378.362 ms 2024-06-06T08:08:13.660 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:08:13.925 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:14.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.20 MB/sec execute 256 sec latency 24378.400 ms 2024-06-06T08:08:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:14 smithi195 ceph-mon[37040]: pgmap v641: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:14 smithi181 ceph-mon[36790]: pgmap v641: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:14 smithi087 ceph-mon[32276]: pgmap v641: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:15.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.15 MB/sec execute 257 sec latency 25378.438 ms 2024-06-06T08:08:16.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.10 MB/sec execute 258 sec latency 26378.502 ms 2024-06-06T08:08:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:16 smithi195 ceph-mon[37040]: pgmap v642: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:16.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:16 smithi181 ceph-mon[36790]: pgmap v642: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:16 smithi087 ceph-mon[32276]: pgmap v642: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-06-06T08:08:17.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.06 MB/sec execute 259 sec latency 27378.539 ms 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/373342704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db8106750 msgr2=0x7f8db810e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/373342704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db8106750 0x7f8db810e2d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b0d0 tx=0x7f8db000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/373342704 shutdown_connections 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/373342704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db8106750 0x7f8db810e2d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/373342704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8101df0 0x7f8db81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/373342704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db81027a0 0x7f8db8106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/373342704 >> 172.21.15.87:0/373342704 conn(0x7f8db8077a00 msgr2=0x7f8db80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/373342704 shutdown_connections 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.102+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/373342704 wait complete. 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 Processor -- start 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 -- start start 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 0x7f8db819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 0x7f8db819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 0x7f8db8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111690 con 0x7f8db81027a0 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111510 con 0x7f8db8101df0 2024-06-06T08:08:18.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dbcc7a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111810 con 0x7f8db8106750 2024-06-06T08:08:18.104 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:18 smithi087 ceph-mon[32276]: pgmap v643: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 515 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.8 MiB/s wr, 150 op/s 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8db659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 0x7f8db819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8db659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 0x7f8db819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:46256/0 (socket says 172.21.15.87:46256) 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8db659c700 1 -- 172.21.15.87:0/740102380 learned_addr learned my addr 172.21.15.87:0/740102380 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.103+0000 7f8dadd9b700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 0x7f8db819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 0x7f8db8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 -- 172.21.15.87:0/740102380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 msgr2=0x7f8db819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 0x7f8db819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 -- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 msgr2=0x7f8db819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 0x7f8db819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 -- 172.21.15.87:0/740102380 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db81990c0 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8dadd9b700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 0x7f8db819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.104+0000 7f8db6d9d700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 0x7f8db8198960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b820 tx=0x7f8db0004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.105+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0021020 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.105+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db8199350 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.105+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db0007b80 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.105+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db00052a0 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.105+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81a5670 con 0x7f8db8106750 2024-06-06T08:08:18.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.106+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db8069180 con 0x7f8db8106750 2024-06-06T08:08:18.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.107+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8db0024020 con 0x7f8db8106750 2024-06-06T08:08:18.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.108+0000 7f8daffff700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 0x7f8da40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:18.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.108+0000 7f8db659c700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 0x7f8da40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:18.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.108+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9671+0+0 (secure 0 0 0) 0x7f8db000f020 con 0x7f8db8106750 2024-06-06T08:08:18.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.108+0000 7f8db659c700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 0x7f8da40a52d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8db8199aa0 tx=0x7f8da800b480 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:18.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.112+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8db009d290 con 0x7f8db8106750 2024-06-06T08:08:18.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 12.01 MB/sec execute 260 sec latency 28378.577 ms 2024-06-06T08:08:18.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:18 smithi195 ceph-mon[37040]: pgmap v643: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 515 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.8 MiB/s wr, 150 op/s 2024-06-06T08:08:18.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:18 smithi181 ceph-mon[36790]: pgmap v643: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 515 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.8 MiB/s wr, 150 op/s 2024-06-06T08:08:18.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.363+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8db81a5840 con 0x7f8db8106750 2024-06-06T08:08:18.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.364+0000 7f8daffff700 1 -- 172.21.15.87:0/740102380 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7f8db0025e40 con 0x7f8db8106750 2024-06-06T08:08:18.364 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:08:18.364 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:08:18.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 msgr2=0x7f8da40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:18.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 0x7f8da40a52d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8db8199aa0 tx=0x7f8da800b480 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 msgr2=0x7f8db8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 0x7f8db8198960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b820 tx=0x7f8db0004ce0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 shutdown_connections 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8db8101df0 0x7f8db819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f8da40a2e10 0x7f8da40a52d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db8106750 0x7f8db8198960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 --2- 172.21.15.87:0/740102380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db81027a0 0x7f8db819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.367+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 >> 172.21.15.87:0/740102380 conn(0x7f8db8077a00 msgr2=0x7f8db8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.368+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 shutdown_connections 2024-06-06T08:08:18.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:18.368+0000 7f8dbcc7a700 1 -- 172.21.15.87:0/740102380 wait complete. 2024-06-06T08:08:18.369 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:08:19.029 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id bbd4b373 --release --await' 2024-06-06T08:08:19.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id bbd4b373 --release --await 2024-06-06T08:08:19.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 11.96 MB/sec execute 261 sec latency 29378.618 ms 2024-06-06T08:08:19.275 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:19.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:19 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/740102380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:19.347 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:19 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/740102380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:19.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:19 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/740102380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:20.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83747 11.92 MB/sec execute 262 sec latency 30378.665 ms 2024-06-06T08:08:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:20 smithi181 ceph-mon[36790]: pgmap v644: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.6 MiB/s wr, 181 op/s 2024-06-06T08:08:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:20 smithi181 ceph-mon[36790]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T08:08:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:20 smithi087 ceph-mon[32276]: pgmap v644: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.6 MiB/s wr, 181 op/s 2024-06-06T08:08:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:20 smithi087 ceph-mon[32276]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T08:08:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:20 smithi195 ceph-mon[37040]: pgmap v644: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 3.6 MiB/s wr, 181 op/s 2024-06-06T08:08:20.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:20 smithi195 ceph-mon[37040]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T08:08:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 -- 172.21.15.87:0/994875834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 msgr2=0x7f4594107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 --2- 172.21.15.87:0/994875834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594107780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b0d0 tx=0x7f458c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 -- 172.21.15.87:0/994875834 shutdown_connections 2024-06-06T08:08:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 --2- 172.21.15.87:0/994875834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102f10 0x7f459410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 --2- 172.21.15.87:0/994875834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594107780 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 --2- 172.21.15.87:0/994875834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102550 0x7f45941029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 -- 172.21.15.87:0/994875834 >> 172.21.15.87:0/994875834 conn(0x7f4594079650 msgr2=0x7f4594079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 -- 172.21.15.87:0/994875834 shutdown_connections 2024-06-06T08:08:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 -- 172.21.15.87:0/994875834 wait complete. 2024-06-06T08:08:21.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.063+0000 7f45992cc700 1 Processor -- start 2024-06-06T08:08:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 -- start start 2024-06-06T08:08:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102550 0x7f459419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102f10 0x7f459419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4594112780 con 0x7f45941073a0 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4594112600 con 0x7f4594102f10 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45992cc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4594112900 con 0x7f4594102550 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55718/0 (socket says 172.21.15.87:55718) 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45937fe700 1 -- 172.21.15.87:0/3301904656 learned_addr learned my addr 172.21.15.87:0/3301904656 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f45927fc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102f10 0x7f459419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.064+0000 7f4592ffd700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102550 0x7f459419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 -- 172.21.15.87:0/3301904656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102550 msgr2=0x7f459419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102550 0x7f459419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 -- 172.21.15.87:0/3301904656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102f10 msgr2=0x7f459419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102f10 0x7f459419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 -- 172.21.15.87:0/3301904656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45941991a0 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45937fe700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f458400c970 tx=0x7f458400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f458bfff700 1 -- 172.21.15.87:0/3301904656 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458400ce60 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f458bfff700 1 -- 172.21.15.87:0/3301904656 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4584004830 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f458bfff700 1 -- 172.21.15.87:0/3301904656 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840055a0 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4594199490 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.065+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45941a5750 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.067+0000 7f458bfff700 1 -- 172.21.15.87:0/3301904656 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f45840049e0 con 0x7f45941073a0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.068+0000 7f458bfff700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 0x7f45800a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.068+0000 7f4592ffd700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 0x7f45800a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.068+0000 7f458bfff700 1 -- 172.21.15.87:0/3301904656 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f45840cf640 con 0x7f45941073a0 2024-06-06T08:08:21.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.068+0000 7f4592ffd700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 0x7f45800a52d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f458c009850 tx=0x7f458c009780 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.068+0000 7f45a3b4fb80 1 Processor -- start 2024-06-06T08:08:21.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 -- start start 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 0x558229267020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229267560 0x558229260e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5582291b2910 con 0x5582292613d0 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55822915a820 con 0x558229266c40 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5582291ae8d0 con 0x558229267560 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55720/0 (socket says 172.21.15.87:55720) 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 -- 172.21.15.87:0/3649531192 learned_addr learned my addr 172.21.15.87:0/3649531192 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45927fc700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229267560 0x558229260e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f4592ffd700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 0x558229267020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 -- 172.21.15.87:0/3649531192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229267560 msgr2=0x558229260e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229267560 0x558229260e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 -- 172.21.15.87:0/3649531192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 msgr2=0x558229267020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 0x558229267020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f45937fe700 1 -- 172.21.15.87:0/3649531192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558229151050 con 0x5582292613d0 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.069+0000 7f4592ffd700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 0x558229267020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45937fe700 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f458409b6d0 tx=0x7f458409b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0b8ebdb962edce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45897fa700 1 -- 172.21.15.87:0/3649531192 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458401d040 con 0x5582292613d0 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45897fa700 1 -- 172.21.15.87:0/3649531192 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458409b740 con 0x5582292613d0 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45897fa700 1 -- 172.21.15.87:0/3649531192 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458409b9a0 con 0x5582292613d0 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/3649531192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 msgr2=0x558229261810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.070+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f458409b6d0 tx=0x7f458409b1a0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/3649531192 shutdown_connections 2024-06-06T08:08:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558229266c40 0x558229267020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x558229261810 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/3649531192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229267560 0x558229260e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/3649531192 >> 172.21.15.87:0/3649531192 conn(0x55822918bdc0 msgr2=0x55822925c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/3649531192 shutdown_connections 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/3649531192 wait complete. 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- start start 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 Processor -- start 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822926d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822926a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55822926dfd0 con 0x558229267560 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558229152cf0 con 0x5582292613d0 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55822926e150 con 0x558229266c40 2024-06-06T08:08:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45a3b4fb80 10 client.0 _ll_register_callbacks cb 0x55822914ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55726/0 (socket says 172.21.15.87:55726) 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f45937fe700 1 -- 172.21.15.87:0/4086116817 learned_addr learned my addr 172.21.15.87:0/4086116817 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.071+0000 7f4592ffd700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822926d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45927fc700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822926a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 -- 172.21.15.87:0/4086116817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 msgr2=0x55822926a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822926a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 -- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 msgr2=0x55822926d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822926d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 -- 172.21.15.87:0/4086116817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582291019d0 con 0x558229267560 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f4592ffd700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822926d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45937fe700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f45840a3e80 tx=0x7f458400af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2626fbaf7a1039aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840b8070 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f458409bba0 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458409b740 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f45840b3070 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45a3b4fb80 10 client.18270 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558229278850 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458409b740 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.072+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f458409ab90 con 0x558229267560 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 10 client.18270 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 10 client.18270 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 20 client.18270 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55822927ad20 0x55822927d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 4 client.18270 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:21.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582291bfe10 con 0x55822927ad20 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f4592ffd700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55822927ad20 0x55822927d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f4592ffd700 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55822927ad20 0x55822927d110 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e140737197b82c7f server_cookie=bb5ecba3ab9e96cb in_seq=0 out_seq=0 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.073+0000 7f457a7fc700 10 client.18270 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f457a7fc700 1 -- 172.21.15.87:0/4086116817 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582291bfe10 con 0x55822927ad20 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f457a7fc700 10 client.18270 handle_command_reply: tid=0 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 client.18270 shutdown 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 msgr2=0x55822926ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f45840a3e80 tx=0x7f458400af50 comp rx=0 tx=0).stop 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 shutdown_connections 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55822927ad20 0x55822927d110 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822926d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f457a7fc700 0 client.18270 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558229267560 0x55822926ab10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/4086116817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822926a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 >> 172.21.15.87:0/4086116817 conn(0x55822918bdc0 msgr2=0x55822925bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 shutdown_connections 2024-06-06T08:08:21.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/4086116817 wait complete. 2024-06-06T08:08:21.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.074+0000 7f45a3b4fb80 20 client.18270 trim_cache size 0 max 16384 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 Processor -- start 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 -- start start 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55822918bdc0 0x558229260e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x558229267940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45941b1090 con 0x5582292613d0 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45941ae920 con 0x55822918bdc0 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45941ac770 con 0x558229266c40 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45927fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45927fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55736/0 (socket says 172.21.15.87:55736) 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f45927fc700 1 -- 172.21.15.87:0/1603292768 learned_addr learned my addr 172.21.15.87:0/1603292768 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.094+0000 7f4592ffd700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55822918bdc0 0x558229260e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45937fe700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x558229267940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 -- 172.21.15.87:0/1603292768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 msgr2=0x558229267940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x558229267940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 -- 172.21.15.87:0/1603292768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55822918bdc0 msgr2=0x558229260e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55822918bdc0 0x558229260e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 -- 172.21.15.87:0/1603292768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822926cea0 con 0x5582292613d0 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45937fe700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x558229267940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f45927fc700 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f457c004580 tx=0x7f457c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39ebf0ee7ab50e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f4588ff9700 1 -- 172.21.15.87:0/1603292768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004830 con 0x5582292613d0 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f4588ff9700 1 -- 172.21.15.87:0/1603292768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c005020 con 0x5582292613d0 2024-06-06T08:08:21.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.095+0000 7f4588ff9700 1 -- 172.21.15.87:0/1603292768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c003920 con 0x5582292613d0 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/1603292768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 msgr2=0x55822925bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f457c004580 tx=0x7f457c00b9f0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/1603292768 shutdown_connections 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55822918bdc0 0x558229260e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x558229267940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/1603292768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582292613d0 0x55822925bff0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/1603292768 >> 172.21.15.87:0/1603292768 conn(0x55822926a190 msgr2=0x55822925b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/1603292768 shutdown_connections 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/1603292768 wait complete. 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- start start 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 Processor -- start 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822927eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822927f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45941acde0 con 0x55822918bdc0 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45941ae250 con 0x5582292613d0 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45941ad4b0 con 0x558229266c40 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f45a3b4fb80 10 client.0 _ll_register_callbacks cb 0x55822914ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.096+0000 7f4592ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822927eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f4592ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55742/0 (socket says 172.21.15.87:55742) 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f4592ffd700 1 -- 172.21.15.87:0/2059720618 learned_addr learned my addr 172.21.15.87:0/2059720618 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45937fe700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822927f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 -- 172.21.15.87:0/2059720618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 msgr2=0x55822927f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822927f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 -- 172.21.15.87:0/2059720618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 msgr2=0x558229280730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 -- 172.21.15.87:0/2059720618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822927f720 con 0x5582292613d0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f4592ffd700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45927fc700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822927eae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f457c00bdf0 tx=0x7f457c0055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8a5e91c4646a1b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f457a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004b20 con 0x5582292613d0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c005020 con 0x5582292613d0 2024-06-06T08:08:21.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f457c018040 con 0x5582292613d0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f457c004cc0 con 0x5582292613d0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45a3b4fb80 10 client.27212 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.097+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5582292ea960 con 0x5582292613d0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c005020 con 0x5582292613d0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f457c01c6f0 con 0x5582292613d0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 10 client.27212 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 10 client.27212 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 20 client.27212 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558229267560 0x5582292ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 4 client.27212 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bbd4b373", "release": true, "await": true}] 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bbd4b373", "release": true, "await": true}) -- 0x5582291ba390 con 0x558229267560 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f4592ffd700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558229267560 0x5582292ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f4592ffd700 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558229267560 0x5582292ecfd0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361a16a77d22fdb6 server_cookie=68b3a668b2174759 in_seq=0 out_seq=0 2024-06-06T08:08:21.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.098+0000 7f457a7fc700 10 client.27212 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f457a7fc700 1 -- 172.21.15.87:0/2059720618 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582291ba390 con 0x558229267560 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f457a7fc700 10 client.27212 handle_command_reply: tid=0 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 client.27212 shutdown 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 msgr2=0x55822927eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822927eae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f457c00bdf0 tx=0x7f457c0055b0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 shutdown_connections 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558229267560 0x5582292ecfd0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558229266c40 0x55822927f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55822918bdc0 0x558229280730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 --2- 172.21.15.87:0/2059720618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582292613d0 0x55822927eae0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 >> 172.21.15.87:0/2059720618 conn(0x55822926a190 msgr2=0x558229267940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f457a7fc700 0 client.27212 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 shutdown_connections 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 1 -- 172.21.15.87:0/2059720618 wait complete. 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.171+0000 7f45a3b4fb80 20 client.27212 trim_cache size 0 max 16384 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:08:21.172 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 64, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bbd4b373": { 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 64, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:08:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:21.174 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 msgr2=0x7f45800a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 0x7f45800a52d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f458c009850 tx=0x7f458c009780 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 msgr2=0x7f4594198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f458400c970 tx=0x7f458400cc80 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 shutdown_connections 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f45800a2e10 0x7f45800a52d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45941073a0 0x7f4594198a40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4594102550 0x7f459419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 --2- 172.21.15.87:0/3301904656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4594102f10 0x7f459419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 >> 172.21.15.87:0/3301904656 conn(0x7f4594079650 msgr2=0x7f4594100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.172+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 shutdown_connections 2024-06-06T08:08:21.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:21.173+0000 7f45992cc700 1 -- 172.21.15.87:0/3301904656 wait complete. 2024-06-06T08:08:21.182 INFO:tasks.workunit.client.0.smithi087.stdout: 10 83751 11.87 MB/sec execute 263 sec latency 31364.059 ms 2024-06-06T08:08:21.912 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bbd4b373', total seconds elapsed: 38.3 2024-06-06T08:08:21.912 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-06T08:08:21.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:08:22.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 84027 11.87 MB/sec execute 264 sec latency 30961.870 ms 2024-06-06T08:08:22.212 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:22.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:22 smithi181 ceph-mon[36790]: pgmap v646: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-06-06T08:08:22.336 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:22 smithi087 ceph-mon[32276]: pgmap v646: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-06-06T08:08:22.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:22 smithi195 ceph-mon[37040]: pgmap v646: 129 pgs: 129 active+clean; 510 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-06-06T08:08:23.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 84395 11.91 MB/sec execute 265 sec latency 183.853 ms 2024-06-06T08:08:24.124 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:24 smithi087 ceph-mon[32276]: pgmap v647: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:24.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 84742 11.95 MB/sec execute 266 sec latency 209.835 ms 2024-06-06T08:08:24.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:24 smithi181 ceph-mon[36790]: pgmap v647: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:24 smithi195 ceph-mon[37040]: pgmap v647: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:24.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.544+0000 7fb284892700 1 -- 172.21.15.87:0/1250185569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 msgr2=0x7fb280108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.544+0000 7fb284892700 1 --2- 172.21.15.87:0/1250185569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb280108f40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb270009990 tx=0x7fb270009ca0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 -- 172.21.15.87:0/1250185569 shutdown_connections 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 --2- 172.21.15.87:0/1250185569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28010eac0 0x7fb28010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 --2- 172.21.15.87:0/1250185569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb280109480 0x7fb280115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 --2- 172.21.15.87:0/1250185569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb280108f40 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 -- 172.21.15.87:0/1250185569 >> 172.21.15.87:0/1250185569 conn(0x7fb280075c60 msgr2=0x7fb2801072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 -- 172.21.15.87:0/1250185569 shutdown_connections 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.545+0000 7fb284892700 1 -- 172.21.15.87:0/1250185569 wait complete. 2024-06-06T08:08:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 Processor -- start 2024-06-06T08:08:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 -- start start 2024-06-06T08:08:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb28019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb280109480 0x7fb28019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb280118f40 con 0x7fb280108ac0 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb280118dc0 con 0x7fb280109480 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.546+0000 7fb284892700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb2801190c0 con 0x7fb28010eac0 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49938/0 (socket says 172.21.15.87:49938) 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 -- 172.21.15.87:0/240030099 learned_addr learned my addr 172.21.15.87:0/240030099 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27e59c700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb28019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 -- 172.21.15.87:0/240030099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb280109480 msgr2=0x7fb28019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb280109480 0x7fb28019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 -- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 msgr2=0x7fb28019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.547+0000 7fb27ed9d700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb28019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.548+0000 7fb27ed9d700 1 -- 172.21.15.87:0/240030099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2801a4440 con 0x7fb28010eac0 2024-06-06T08:08:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.549+0000 7fb27ed9d700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b780 tx=0x7fb27800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.550+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27800f840 con 0x7fb28010eac0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.550+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2801a4730 con 0x7fb28010eac0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.550+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28007ace0 con 0x7fb28010eac0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.550+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb278017400 con 0x7fb28010eac0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.550+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27800d630 con 0x7fb28010eac0 2024-06-06T08:08:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.552+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2801110c0 con 0x7fb28010eac0 2024-06-06T08:08:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.557+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb2780175b0 con 0x7fb28010eac0 2024-06-06T08:08:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.558+0000 7fb26f7fe700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 0x7fb2680a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.558+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fb2780cfda0 con 0x7fb28010eac0 2024-06-06T08:08:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.559+0000 7fb27e59c700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 0x7fb2680a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.559+0000 7fb27e59c700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 0x7fb2680a5420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb274009910 tx=0x7fb274008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.563+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb27801c060 con 0x7fb28010eac0 2024-06-06T08:08:24.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.921+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb280069180 con 0x7fb28010eac0 2024-06-06T08:08:24.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.922+0000 7fb26f7fe700 1 -- 172.21.15.87:0/240030099 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7fb278013030 con 0x7fb28010eac0 2024-06-06T08:08:24.923 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:08:24.923 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:08:24.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 msgr2=0x7fb2680a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:24.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 0x7fb2680a5420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb274009910 tx=0x7fb274008040 comp rx=0 tx=0).stop 2024-06-06T08:08:24.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 msgr2=0x7fb2801a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:24.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b780 tx=0x7fb27800bb40 comp rx=0 tx=0).stop 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 shutdown_connections 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb2680a2f60 0x7fb2680a5420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb28010eac0 0x7fb2801a3d00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb280108ac0 0x7fb28019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 --2- 172.21.15.87:0/240030099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb280109480 0x7fb28019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.925+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 >> 172.21.15.87:0/240030099 conn(0x7fb280075c60 msgr2=0x7fb280076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.926+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 shutdown_connections 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:24.926+0000 7fb284892700 1 -- 172.21.15.87:0/240030099 wait complete. 2024-06-06T08:08:24.926 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:08:25.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85052 11.96 MB/sec execute 267 sec latency 88.585 ms 2024-06-06T08:08:25.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:25 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/240030099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:25.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:25 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/240030099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:25.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:25 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/240030099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 -- 172.21.15.87:0/2682438576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 msgr2=0x7f2f70100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 --2- 172.21.15.87:0/2682438576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70100d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009930 tx=0x7f2f6002a420 comp rx=0 tx=0).stop 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 -- 172.21.15.87:0/2682438576 shutdown_connections 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 --2- 172.21.15.87:0/2682438576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f7010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 --2- 172.21.15.87:0/2682438576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70100d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 --2- 172.21.15.87:0/2682438576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 0x7f2f70106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 -- 172.21.15.87:0/2682438576 >> 172.21.15.87:0/2682438576 conn(0x7f2f700779f0 msgr2=0x7f2f70077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 -- 172.21.15.87:0/2682438576 shutdown_connections 2024-06-06T08:08:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.621+0000 7f2f7855a640 1 -- 172.21.15.87:0/2682438576 wait complete. 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 Processor -- start 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 -- start start 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 0x7f2f701a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70110cd0 con 0x7f2f701012c0 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70110b50 con 0x7f2f701068c0 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f7855a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70110e50 con 0x7f2f70100900 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55786/0 (socket says 172.21.15.87:55786) 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f75af5640 1 -- 172.21.15.87:0/1950756145 learned_addr learned my addr 172.21.15.87:0/1950756145 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f76af7640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 0x7f2f701a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 -- 172.21.15.87:0/1950756145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 msgr2=0x7f2f701a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 0x7f2f701a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 -- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 msgr2=0x7f2f70199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f762f6640 1 -- 172.21.15.87:0/1950756145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701a4610 con 0x7f2f70100900 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.622+0000 7f2f75af5640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70199050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9f0 tx=0x7f2f6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f677fe640 1 -- 172.21.15.87:0/1950756145 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00ccc0 con 0x7f2f70100900 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f677fe640 1 -- 172.21.15.87:0/1950756145 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c005020 con 0x7f2f70100900 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701a47e0 con 0x7f2f70100900 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f677fe640 1 -- 172.21.15.87:0/1950756145 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c004a00 con 0x7f2f70100900 2024-06-06T08:08:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.623+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701a4cb0 con 0x7f2f70100900 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.625+0000 7f2f677fe640 1 -- 172.21.15.87:0/1950756145 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2f6c0051d0 con 0x7f2f70100900 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 Processor -- start 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 -- start start 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5588ed4294f0 con 0x5588ed4cee90 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588ed3c7870 con 0x5588ed4d35a0 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f7b265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588ed4254b0 con 0x5588ed4cf840 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f677fe640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 0x7f2f5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f76af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49972/0 (socket says 172.21.15.87:49972) 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f762f6640 1 -- 172.21.15.87:0/4230044310 learned_addr learned my addr 172.21.15.87:0/4230044310 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f76af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55802/0 (socket says 172.21.15.87:55802) 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f75af5640 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.626+0000 7f2f76af7640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 0x7f2f5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f677fe640 1 -- 172.21.15.87:0/1950756145 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f2f6c0cf740 con 0x7f2f70100900 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 -- 172.21.15.87:0/4230044310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 msgr2=0x5588ed4d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 -- 172.21.15.87:0/4230044310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 msgr2=0x5588ed4cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 -- 172.21.15.87:0/4230044310 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ed3e39b0 con 0x5588ed4cf840 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f762f6640 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9c0 tx=0x7f2f6c0c63d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5965c75ceca9819 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f76af7640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 0x7f2f5c0a5260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009b80 tx=0x7f2f68009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f64bfe640 1 -- 172.21.15.87:0/4230044310 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c01c030 con 0x5588ed4cf840 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f64bfe640 1 -- 172.21.15.87:0/4230044310 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c09b840 con 0x5588ed4cf840 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.627+0000 7f2f64bfe640 1 -- 172.21.15.87:0/4230044310 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c099bd0 con 0x5588ed4cf840 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- 172.21.15.87:0/4230044310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 msgr2=0x5588ed4d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9c0 tx=0x7f2f6c0c63d0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- 172.21.15.87:0/4230044310 shutdown_connections 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d2f80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 --2- 172.21.15.87:0/4230044310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- 172.21.15.87:0/4230044310 >> 172.21.15.87:0/4230044310 conn(0x5588ed4c1130 msgr2=0x5588ed4c1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- 172.21.15.87:0/4230044310 shutdown_connections 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- 172.21.15.87:0/4230044310 wait complete. 2024-06-06T08:08:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 -- start start 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.628+0000 7f2f7b265740 1 Processor -- start 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5588ed4db750 con 0x5588ed4d35a0 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5588ed3b2ae0 con 0x5588ed4cee90 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5588ed4db8d0 con 0x5588ed4cf840 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:46350/0 (socket says 172.21.15.87:46350) 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49974/0 (socket says 172.21.15.87:49974) 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f762f6640 1 -- 172.21.15.87:0/3676437107 learned_addr learned my addr 172.21.15.87:0/3676437107 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f7b265740 10 client.0 _ll_register_callbacks cb 0x5588ed3eb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 -- 172.21.15.87:0/3676437107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 msgr2=0x5588ed4d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 -- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 msgr2=0x5588ed4d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f76af7640 1 -- 172.21.15.87:0/3676437107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ed13d210 con 0x5588ed4d35a0 2024-06-06T08:08:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.629+0000 7f2f762f6640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f76af7640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4d8cc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2f68011630 tx=0x7f2f68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8582c38a81deba07 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680047c0 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6c09b840 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f2f68004b20 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2f68004d20 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f7b265740 10 client.18294 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.630+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5588ed209620 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c09b840 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2f68027070 con 0x5588ed4d35a0 2024-06-06T08:08:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 10 client.18294 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 10 client.18294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 20 client.18294 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed4eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 4 client.18294 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.631+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ed43cf10 con 0x5588ed4e8d90 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.632+0000 7f2f762f6640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed4eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.632+0000 7f2f59ffb640 10 client.18294 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.632+0000 7f2f762f6640 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed4eb180 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9602dc2cd3bd54d server_cookie=99b1efb9fb1de2f6 in_seq=0 out_seq=0 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.634+0000 7f2f59ffb640 1 -- 172.21.15.87:0/3676437107 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588ed43cf10 con 0x5588ed4e8d90 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.634+0000 7f2f59ffb640 10 client.18294 handle_command_reply: tid=0 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 client.18294 shutdown 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 msgr2=0x5588ed4d8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4d8cc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2f68011630 tx=0x7f2f68004a40 comp rx=0 tx=0).stop 2024-06-06T08:08:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 shutdown_connections 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed4eb180 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f59ffb640 0 client.18294 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4d8cc0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 --2- 172.21.15.87:0/3676437107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cee90 0x5588ed4d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 >> 172.21.15.87:0/3676437107 conn(0x5588ed4c1130 msgr2=0x5588ed4ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.635+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 shutdown_connections 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.636+0000 7f2f7b265740 1 -- 172.21.15.87:0/3676437107 wait complete. 2024-06-06T08:08:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.636+0000 7f2f7b265740 20 client.18294 trim_cache size 0 max 16384 2024-06-06T08:08:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 Processor -- start 2024-06-06T08:08:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 -- start start 2024-06-06T08:08:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701af6e0 con 0x5588ed4cee90 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701acfb0 con 0x5588ed4d35a0 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f7b265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701aae20 con 0x5588ed4cf840 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55828/0 (socket says 172.21.15.87:55828) 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49986/0 (socket says 172.21.15.87:49986) 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 -- 172.21.15.87:0/1068791629 learned_addr learned my addr 172.21.15.87:0/1068791629 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f76af7640 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 -- 172.21.15.87:0/1068791629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 msgr2=0x5588ed4d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 -- 172.21.15.87:0/1068791629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 msgr2=0x5588ed4d9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.656+0000 7f2f762f6640 1 -- 172.21.15.87:0/1068791629 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ed4eb4f0 con 0x5588ed4cee90 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c097ca0 tx=0x7f2f6c066040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aed1b82ddc1de8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f5bfff640 1 -- 172.21.15.87:0/1068791629 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c01c040 con 0x5588ed4cee90 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f5bfff640 1 -- 172.21.15.87:0/1068791629 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c005560 con 0x5588ed4cee90 2024-06-06T08:08:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f5bfff640 1 -- 172.21.15.87:0/1068791629 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0cccf0 con 0x5588ed4cee90 2024-06-06T08:08:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 -- 172.21.15.87:0/1068791629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 msgr2=0x5588ed4d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c097ca0 tx=0x7f2f6c066040 comp rx=0 tx=0).stop 2024-06-06T08:08:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 -- 172.21.15.87:0/1068791629 shutdown_connections 2024-06-06T08:08:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4d35a0 0x5588ed4d9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cf840 0x5588ed4d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.657+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1068791629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4cee90 0x5588ed4d2f70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- 172.21.15.87:0/1068791629 >> 172.21.15.87:0/1068791629 conn(0x5588ed4c1130 msgr2=0x5588ed4d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- 172.21.15.87:0/1068791629 shutdown_connections 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- 172.21.15.87:0/1068791629 wait complete. 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- start start 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 Processor -- start 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 0x5588ed4e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cf840 0x5588ed4e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701ab470 con 0x5588ed4d35a0 2024-06-06T08:08:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701ac8e0 con 0x5588ed4cf840 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701abb40 con 0x5588ed4cee90 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f7b265740 10 client.0 _ll_register_callbacks cb 0x7f2f68009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f76af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f762f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 0x5588ed4e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f75af5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cf840 0x5588ed4e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f76af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55830/0 (socket says 172.21.15.87:55830) 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.658+0000 7f2f76af7640 1 -- 172.21.15.87:0/1886007926 learned_addr learned my addr 172.21.15.87:0/1886007926 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 -- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cf840 msgr2=0x5588ed4e1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cf840 0x5588ed4e1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 -- 172.21.15.87:0/1886007926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 msgr2=0x5588ed4f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 -- 172.21.15.87:0/1886007926 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ed4f08b0 con 0x5588ed4cee90 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f76af7640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f762f6640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 0x5588ed4e0dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0a3f80 tx=0x7f2f6c005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c66ee3ac1db1d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:08:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0b3070 con 0x5588ed4cee90 2024-06-06T08:08:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f2f6c0cccf0 con 0x5588ed4cee90 2024-06-06T08:08:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6c005560 con 0x5588ed4cee90 2024-06-06T08:08:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2f6c01c050 con 0x5588ed4cee90 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 10 client.27394 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5588ed4e0630 con 0x5588ed4cee90 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c005560 con 0x5588ed4cee90 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f2f6c0a4020 con 0x5588ed4cee90 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 10 client.27394 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 20 client.27394 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed555a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 4 client.27394 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.659+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588ed436c20 con 0x5588ed4e8d90 2024-06-06T08:08:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.660+0000 7f2f75af5640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed555a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.660+0000 7f2f59ffb640 10 client.27394 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:25.660+0000 7f2f75af5640 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed555a50 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119c3aca093eb01b server_cookie=7980095d23cf5267 in_seq=0 out_seq=0 2024-06-06T08:08:25.940 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-06T08:08:25.941 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-06T08:08:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f59ffb640 1 -- 172.21.15.87:0/1886007926 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588ed436c20 con 0x5588ed4e8d90 2024-06-06T08:08:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f59ffb640 10 client.27394 handle_command_reply: tid=0 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 client.27394 shutdown 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 msgr2=0x5588ed4e0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 0x5588ed4e0dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0a3f80 tx=0x7f2f6c005ab0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 shutdown_connections 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5588ed4e8d90 0x5588ed555a50 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5588ed4d35a0 0x5588ed4f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5588ed4cf840 0x5588ed4e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 --2- 172.21.15.87:0/1886007926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5588ed4cee90 0x5588ed4e0dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f59ffb640 0 client.27394 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.073+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 >> 172.21.15.87:0/1886007926 conn(0x5588ed4c1130 msgr2=0x5588ed4d5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.074+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 shutdown_connections 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.074+0000 7f2f7b265740 1 -- 172.21.15.87:0/1886007926 wait complete. 2024-06-06T08:08:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.074+0000 7f2f7b265740 20 client.27394 trim_cache size 0 max 16384 2024-06-06T08:08:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:08:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:08:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:08:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 msgr2=0x7f2f5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 0x7f2f5c0a5260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009b80 tx=0x7f2f68009500 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 msgr2=0x7f2f70199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70199050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9f0 tx=0x7f2f6c00edf0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 shutdown_connections 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2f5c0a2da0 0x7f2f5c0a5260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f701068c0 0x7f2f701a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f701012c0 0x7f2f70199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 --2- 172.21.15.87:0/1950756145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f70100900 0x7f2f70199050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 >> 172.21.15.87:0/1950756145 conn(0x7f2f700779f0 msgr2=0x7f2f701033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 shutdown_connections 2024-06-06T08:08:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:26.075+0000 7f2f7855a640 1 -- 172.21.15.87:0/1950756145 wait complete. 2024-06-06T08:08:26.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85312 11.98 MB/sec execute 268 sec latency 305.987 ms 2024-06-06T08:08:26.248 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:26.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:26 smithi181 ceph-mon[36790]: pgmap v648: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:26 smithi087 ceph-mon[32276]: pgmap v648: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:26 smithi195 ceph-mon[37040]: pgmap v648: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-06-06T08:08:27.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85605 12.00 MB/sec execute 269 sec latency 292.929 ms 2024-06-06T08:08:28.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85769 11.97 MB/sec execute 270 sec latency 295.366 ms 2024-06-06T08:08:28.255 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:28 smithi087 ceph-mon[32276]: pgmap v649: 129 pgs: 129 active+clean; 530 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-06-06T08:08:28.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:28 smithi181 ceph-mon[36790]: pgmap v649: 129 pgs: 129 active+clean; 530 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-06-06T08:08:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:28 smithi195 ceph-mon[37040]: pgmap v649: 129 pgs: 129 active+clean; 530 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 -- 172.21.15.87:0/1358487717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 msgr2=0x7fa040105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 --2- 172.21.15.87:0/1358487717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa040105fe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa0340099c0 tx=0x7fa034009cd0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 -- 172.21.15.87:0/1358487717 shutdown_connections 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 --2- 172.21.15.87:0/1358487717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401015f0 0x7fa0401019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 --2- 172.21.15.87:0/1358487717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401066c0 0x7fa04010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 --2- 172.21.15.87:0/1358487717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa040105fe0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 -- 172.21.15.87:0/1358487717 >> 172.21.15.87:0/1358487717 conn(0x7fa040078330 msgr2=0x7fa040078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:28.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 -- 172.21.15.87:0/1358487717 shutdown_connections 2024-06-06T08:08:28.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.584+0000 7fa044dc5700 1 -- 172.21.15.87:0/1358487717 wait complete. 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 Processor -- start 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 -- start start 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401015f0 0x7fa04019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401066c0 0x7fa0401989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401116d0 con 0x7fa040101fa0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa040111550 con 0x7fa0401066c0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa044dc5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa040111850 con 0x7fa0401015f0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa03dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa03dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42300/0 (socket says 172.21.15.87:42300) 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa03ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401066c0 0x7fa0401989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa03dd9b700 1 -- 172.21.15.87:0/2542653159 learned_addr learned my addr 172.21.15.87:0/2542653159 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.585+0000 7fa03e59c700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401015f0 0x7fa04019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 -- 172.21.15.87:0/2542653159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401015f0 msgr2=0x7fa04019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401015f0 0x7fa04019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 -- 172.21.15.87:0/2542653159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401066c0 msgr2=0x7fa0401989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401066c0 0x7fa0401989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 -- 172.21.15.87:0/2542653159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa040199130 con 0x7fa040101fa0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa034005f50 tx=0x7fa034003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa02f7fe700 1 -- 172.21.15.87:0/2542653159 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034003680 con 0x7fa040101fa0 2024-06-06T08:08:28.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0401993c0 con 0x7fa040101fa0 2024-06-06T08:08:28.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa02f7fe700 1 -- 172.21.15.87:0/2542653159 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0340055a0 con 0x7fa040101fa0 2024-06-06T08:08:28.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa02f7fe700 1 -- 172.21.15.87:0/2542653159 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400fce0 con 0x7fa040101fa0 2024-06-06T08:08:28.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.586+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0401997c0 con 0x7fa040101fa0 2024-06-06T08:08:28.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.589+0000 7fa02f7fe700 1 -- 172.21.15.87:0/2542653159 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa03402b3f0 con 0x7fa040101fa0 2024-06-06T08:08:28.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.590+0000 7fa02f7fe700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 0x7fa0280a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.590+0000 7fa02f7fe700 1 -- 172.21.15.87:0/2542653159 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fa0340d0660 con 0x7fa040101fa0 2024-06-06T08:08:28.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.590+0000 7fa03e59c700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 0x7fa0280a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.590+0000 7fa04f648b80 1 Processor -- start 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03e59c700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 0x7fa0280a5510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa040199b10 tx=0x7fa030009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 -- start start 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db4080ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db408108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55db407619a0 con 0x55db40815cd0 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db407098b0 con 0x55db408165f0 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa04f648b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db4075d960 con 0x55db40810460 2024-06-06T08:08:28.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42306/0 (socket says 172.21.15.87:42306) 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03e59c700 1 -- 172.21.15.87:0/1855423396 learned_addr learned my addr 172.21.15.87:0/1855423396 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db408108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.591+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db4080ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 -- 172.21.15.87:0/1855423396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 msgr2=0x55db408108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db408108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 -- 172.21.15.87:0/1855423396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 msgr2=0x55db4080ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db4080ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 -- 172.21.15.87:0/1855423396 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db407000e0 con 0x55db40815cd0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa030011d50 tx=0x7fa030014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a859f372b35cd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa02cff9700 1 -- 172.21.15.87:0/1855423396 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000eb20 con 0x55db40815cd0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa02cff9700 1 -- 172.21.15.87:0/1855423396 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030004500 con 0x55db40815cd0 2024-06-06T08:08:28.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.592+0000 7fa02cff9700 1 -- 172.21.15.87:0/1855423396 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030026480 con 0x55db40815cd0 2024-06-06T08:08:28.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- 172.21.15.87:0/1855423396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 msgr2=0x55db408160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa030011d50 tx=0x7fa030014000 comp rx=0 tx=0).stop 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- 172.21.15.87:0/1855423396 shutdown_connections 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db408108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40815cd0 0x55db408160b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1855423396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db4080ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- 172.21.15.87:0/1855423396 >> 172.21.15.87:0/1855423396 conn(0x55db4073ae50 msgr2=0x55db4080b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- 172.21.15.87:0/1855423396 shutdown_connections 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- 172.21.15.87:0/1855423396 wait complete. 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.593+0000 7fa04f648b80 1 -- start start 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 Processor -- start 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40810460 0x55db4081c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40819690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55db4081d060 con 0x55db408165f0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db40701d80 con 0x55db40810460 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db4081d1e0 con 0x55db40815cd0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa04f648b80 10 client.0 _ll_register_callbacks cb 0x55db406fbab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42310/0 (socket says 172.21.15.87:42310) 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 -- 172.21.15.87:0/3026128288 learned_addr learned my addr 172.21.15.87:0/3026128288 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40819690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03e59c700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40810460 0x55db4081c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 -- 172.21.15.87:0/3026128288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 msgr2=0x55db40819690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40819690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 -- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40810460 msgr2=0x55db4081c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40810460 0x55db4081c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.594+0000 7fa03ed9d700 1 -- 172.21.15.87:0/3026128288 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db406b0a60 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.595+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa038011f60 tx=0x7fa038014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7990efea4ee405a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.595+0000 7fa025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.595+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038014ba0 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fa038014d40 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa030004500 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa038003a00 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa04f648b80 10 client.18312 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db40827a40 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030004500 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fa038007500 con 0x55db408165f0 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa04f648b80 10 client.18312 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.596+0000 7fa04f648b80 10 client.18312 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa04f648b80 20 client.18312 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:28.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db4082a380 0x55db4082c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa04f648b80 4 client.18312 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:28.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db4076eea0 con 0x55db4082a380 2024-06-06T08:08:28.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa03e59c700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db4082a380 0x55db4082c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa025ffb700 10 client.18312 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:28.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.597+0000 7fa03e59c700 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db4082a380 0x55db4082c800 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ce56f950272697 server_cookie=1342d287430c8714 in_seq=0 out_seq=0 2024-06-06T08:08:28.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.601+0000 7fa025ffb700 1 -- 172.21.15.87:0/3026128288 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db4076eea0 con 0x55db4082a380 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.601+0000 7fa025ffb700 10 client.18312 handle_command_reply: tid=0 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.601+0000 7fa04f648b80 1 client.18312 shutdown 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.601+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 msgr2=0x55db40819bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa038011f60 tx=0x7fa038014600 comp rx=0 tx=0).stop 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 shutdown_connections 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db4082a380 0x55db4082c800 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40810460 0x55db4081c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db40819bd0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa025ffb700 0 client.18312 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 --2- 172.21.15.87:0/3026128288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40819690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 >> 172.21.15.87:0/3026128288 conn(0x55db4073ae50 msgr2=0x55db4080ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:28.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 shutdown_connections 2024-06-06T08:08:28.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 1 -- 172.21.15.87:0/3026128288 wait complete. 2024-06-06T08:08:28.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.602+0000 7fa04f648b80 20 client.18312 trim_cache size 0 max 16384 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 Processor -- start 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 -- start start 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40832610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db408169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aee40 con 0x55db40810460 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ac6d0 con 0x55db408165f0 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.625+0000 7fa04f648b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aa540 con 0x55db40815cd0 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42318/0 (socket says 172.21.15.87:42318) 2024-06-06T08:08:28.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 -- 172.21.15.87:0/1650356318 learned_addr learned my addr 172.21.15.87:0/1650356318 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40832610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db408169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 -- 172.21.15.87:0/1650356318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 msgr2=0x55db40832610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40832610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 -- 172.21.15.87:0/1650356318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 msgr2=0x55db408169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db408169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 -- 172.21.15.87:0/1650356318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db40817c00 con 0x55db40810460 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa03e59c700 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa0300117d0 tx=0x7fa030003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef174711061c054c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa027fff700 1 -- 172.21.15.87:0/1650356318 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030003890 con 0x55db40810460 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa027fff700 1 -- 172.21.15.87:0/1650356318 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030003b90 con 0x55db40810460 2024-06-06T08:08:28.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.626+0000 7fa027fff700 1 -- 172.21.15.87:0/1650356318 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030005510 con 0x55db40810460 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 -- 172.21.15.87:0/1650356318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 msgr2=0x55db408320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa0300117d0 tx=0x7fa030003b20 comp rx=0 tx=0).stop 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 -- 172.21.15.87:0/1650356318 shutdown_connections 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db408165f0 0x55db408169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db40810460 0x55db408320d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 --2- 172.21.15.87:0/1650356318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40815cd0 0x55db40832610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 -- 172.21.15.87:0/1650356318 >> 172.21.15.87:0/1650356318 conn(0x55db4073ae50 msgr2=0x55db40817f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 -- 172.21.15.87:0/1650356318 shutdown_connections 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.627+0000 7fa04f648b80 1 -- 172.21.15.87:0/1650356318 wait complete. 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 -- start start 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 Processor -- start 2024-06-06T08:08:28.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db4081f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40815cd0 0x55db4081f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aab90 con 0x55db408165f0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ac000 con 0x55db40815cd0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ab260 con 0x55db40810460 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa04f648b80 10 client.0 _ll_register_callbacks cb 0x55db406fbab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa03ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa03e59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db4081f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa03ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42324/0 (socket says 172.21.15.87:42324) 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa03ed9d700 1 -- 172.21.15.87:0/283860299 learned_addr learned my addr 172.21.15.87:0/283860299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.628+0000 7fa03dd9b700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40815cd0 0x55db4081f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 -- 172.21.15.87:0/283860299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 msgr2=0x55db4081f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db4081f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 -- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40815cd0 msgr2=0x55db4081f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:28.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40815cd0 0x55db4081f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 -- 172.21.15.87:0/283860299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db40895180 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa03ed9d700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e40 tx=0x7fa038009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faa2a15faba30f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038007d30 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fa038003da0 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa030003b90 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa038039050 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa04f648b80 10 client.18318 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55db40896460 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.629+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa038014ba0 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fa038016490 con 0x55db408165f0 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 10 client.18318 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:28.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 10 client.18318 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 20 client.18318 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db40896bb0 0x55db40898fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 4 client.18318 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fa030003b90 con 0x55db40896bb0 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa03e59c700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db40896bb0 0x55db40898fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa025ffb700 10 client.18318 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:28.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:28.630+0000 7fa03e59c700 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db40896bb0 0x55db40898fd0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19997dc6f1e7e9a server_cookie=6dd96b9602e852b in_seq=0 out_seq=0 2024-06-06T08:08:29.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85902 11.95 MB/sec execute 271 sec latency 546.117 ms 2024-06-06T08:08:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.467+0000 7fa025ffb700 1 -- 172.21.15.87:0/283860299 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa030003b90 con 0x55db40896bb0 2024-06-06T08:08:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.467+0000 7fa025ffb700 10 client.18318 handle_command_reply: tid=0 2024-06-06T08:08:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 client.18318 shutdown 2024-06-06T08:08:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 msgr2=0x55db4081fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:29.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e40 tx=0x7fa038009e70 comp rx=0 tx=0).stop 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 shutdown_connections 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55db40896bb0 0x55db40898fd0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55db408165f0 0x55db4081fef0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db40810460 0x55db4081f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 --2- 172.21.15.87:0/283860299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db40815cd0 0x55db4081f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 >> 172.21.15.87:0/283860299 conn(0x55db4073ae50 msgr2=0x55db408169d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa025ffb700 0 client.18318 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 shutdown_connections 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 1 -- 172.21.15.87:0/283860299 wait complete. 2024-06-06T08:08:29.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.468+0000 7fa04f648b80 20 client.18318 trim_cache size 0 max 16384 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 66, 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "79878835": { 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 66, 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:29.472 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:08:29.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 msgr2=0x7fa0280a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:29.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 0x7fa0280a5510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa040199b10 tx=0x7fa030009260 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 msgr2=0x7fa04019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa034005f50 tx=0x7fa034003a90 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 shutdown_connections 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0401066c0 0x7fa0401989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fa0280a3050 0x7fa0280a5510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0401015f0 0x7fa04019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 --2- 172.21.15.87:0/2542653159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa040101fa0 0x7fa04019ed50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 >> 172.21.15.87:0/2542653159 conn(0x7fa040078330 msgr2=0x7fa04006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 shutdown_connections 2024-06-06T08:08:29.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:29.469+0000 7fa044dc5700 1 -- 172.21.15.87:0/2542653159 wait complete. 2024-06-06T08:08:30.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:30 smithi087 ceph-mon[32276]: pgmap v650: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-06-06T08:08:30.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.90 MB/sec execute 272 sec latency 1546.181 ms 2024-06-06T08:08:30.297 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '79878835', quiesce took 8.4 seconds. Will release after: 21.6 2024-06-06T08:08:30.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:30 smithi181 ceph-mon[36790]: pgmap v650: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-06-06T08:08:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:30 smithi195 ceph-mon[37040]: pgmap v650: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-06-06T08:08:31.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.86 MB/sec execute 273 sec latency 2546.281 ms 2024-06-06T08:08:32.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.82 MB/sec execute 274 sec latency 3546.379 ms 2024-06-06T08:08:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:32 smithi181 ceph-mon[36790]: pgmap v651: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-06-06T08:08:32.311 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:32 smithi195 ceph-mon[37040]: pgmap v651: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-06-06T08:08:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:32 smithi087 ceph-mon[32276]: pgmap v651: 129 pgs: 129 active+clean; 533 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-06-06T08:08:33.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.77 MB/sec execute 275 sec latency 4546.414 ms 2024-06-06T08:08:34.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.73 MB/sec execute 276 sec latency 5546.458 ms 2024-06-06T08:08:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:34 smithi181 ceph-mon[36790]: pgmap v652: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:08:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:34 smithi087 ceph-mon[32276]: pgmap v652: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:08:34.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:34 smithi195 ceph-mon[37040]: pgmap v652: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.1 MiB/s wr, 379 op/s 2024-06-06T08:08:35.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.69 MB/sec execute 277 sec latency 6546.558 ms 2024-06-06T08:08:36.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.65 MB/sec execute 278 sec latency 7546.655 ms 2024-06-06T08:08:36.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:36 smithi181 ceph-mon[36790]: pgmap v653: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-06-06T08:08:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:36 smithi087 ceph-mon[32276]: pgmap v653: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-06-06T08:08:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:36 smithi195 ceph-mon[37040]: pgmap v653: 129 pgs: 129 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-06-06T08:08:37.183 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.61 MB/sec execute 279 sec latency 8546.735 ms 2024-06-06T08:08:38.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.56 MB/sec execute 280 sec latency 9546.800 ms 2024-06-06T08:08:38.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:38 smithi181 ceph-mon[36790]: pgmap v654: 129 pgs: 129 active+clean; 544 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 321 op/s 2024-06-06T08:08:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:38 smithi087 ceph-mon[32276]: pgmap v654: 129 pgs: 129 active+clean; 544 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 321 op/s 2024-06-06T08:08:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:38 smithi195 ceph-mon[37040]: pgmap v654: 129 pgs: 129 active+clean; 544 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 321 op/s 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 -- 172.21.15.87:0/3902538305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de806abb0 msgr2=0x7f4de810e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 --2- 172.21.15.87:0/3902538305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de806abb0 0x7f4de810e830 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009ca0 tx=0x7f4de002a8c0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 -- 172.21.15.87:0/3902538305 shutdown_connections 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 --2- 172.21.15.87:0/3902538305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de806abb0 0x7f4de810e830 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 --2- 172.21.15.87:0/3902538305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806a210 0x7f4de806a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.179+0000 7f4dedf90640 1 --2- 172.21.15.87:0/3902538305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de81079c0 0x7f4de8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.180+0000 7f4dedf90640 1 -- 172.21.15.87:0/3902538305 >> 172.21.15.87:0/3902538305 conn(0x7f4de80775f0 msgr2=0x7f4de8077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.180+0000 7f4dedf90640 1 -- 172.21.15.87:0/3902538305 shutdown_connections 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.180+0000 7f4dedf90640 1 -- 172.21.15.87:0/3902538305 wait complete. 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.180+0000 7f4dedf90640 1 Processor -- start 2024-06-06T08:08:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.180+0000 7f4dedf90640 1 -- start start 2024-06-06T08:08:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de806a210 0x7f4de8100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 0x7f4de8101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111db0 con 0x7f4de81079c0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111c30 con 0x7f4de806a210 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4dedf90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111f30 con 0x7f4de806abb0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35228/0 (socket says 172.21.15.87:35228) 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 -- 172.21.15.87:0/2755524913 learned_addr learned my addr 172.21.15.87:0/2755524913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de77fe640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de806a210 0x7f4de8100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 0x7f4de8101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 -- 172.21.15.87:0/2755524913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 msgr2=0x7f4de8101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 0x7f4de8101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 -- 172.21.15.87:0/2755524913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de806a210 msgr2=0x7f4de8100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de806a210 0x7f4de8100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 -- 172.21.15.87:0/2755524913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de81a6630 con 0x7f4de81079c0 2024-06-06T08:08:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 0x7f4de8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de7fff640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4de0031a20 tx=0x7f4de0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de4ff9640 1 -- 172.21.15.87:0/2755524913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0003c30 con 0x7f4de81079c0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.181+0000 7f4de4ff9640 1 -- 172.21.15.87:0/2755524913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0003dd0 con 0x7f4de81079c0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.182+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de81a68c0 con 0x7f4de81079c0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.182+0000 7f4de4ff9640 1 -- 172.21.15.87:0/2755524913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de002fcf0 con 0x7f4de81079c0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.182+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a6e10 con 0x7f4de81079c0 2024-06-06T08:08:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.184+0000 7f4de4ff9640 1 -- 172.21.15.87:0/2755524913 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4de0004050 con 0x7f4de81079c0 2024-06-06T08:08:39.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.52 MB/sec execute 281 sec latency 10546.849 ms 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.184+0000 7f4df0d7f740 1 Processor -- start 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.184+0000 7f4df0d7f740 1 -- start start 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.184+0000 7f4de4ff9640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 0x7f4dd00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 0x7f4dd00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563ecea2e4f0 con 0x563eceaca170 2024-06-06T08:08:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ece9cc870 con 0x563eceacaa90 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563ecea2a4b0 con 0x563ecead5350 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de4ff9640 1 -- 172.21.15.87:0/2755524913 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f4de00f0ab0 con 0x7f4de81079c0 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35244/0 (socket says 172.21.15.87:35244) 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 -- 172.21.15.87:0/3501985906 learned_addr learned my addr 172.21.15.87:0/3501985906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de7fff640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 -- 172.21.15.87:0/3501985906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 msgr2=0x563ecead7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 -- 172.21.15.87:0/3501985906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 msgr2=0x563ecead4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 -- 172.21.15.87:0/3501985906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ece9e89b0 con 0x563eceaca170 2024-06-06T08:08:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 0x7f4dd00a5260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4de81026e0 tx=0x7f4ddc00b700 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc90 tx=0x7f4dd80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5af4c7c7b9b5f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4dcdffb640 1 -- 172.21.15.87:0/3501985906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd801c480 con 0x563eceaca170 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4dcdffb640 1 -- 172.21.15.87:0/3501985906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd801cb00 con 0x563eceaca170 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.185+0000 7f4dcdffb640 1 -- 172.21.15.87:0/3501985906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011bb0 con 0x563eceaca170 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3501985906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 msgr2=0x563eceaca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc90 tx=0x7f4dd80107a0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3501985906 shutdown_connections 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3501985906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaca550 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3501985906 >> 172.21.15.87:0/3501985906 conn(0x563eceac6680 msgr2=0x563eceac6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3501985906 shutdown_connections 2024-06-06T08:08:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.186+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3501985906 wait complete. 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 -- start start 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 Processor -- start 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563ecead7f80 con 0x563eceaca170 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ece9b7ae0 con 0x563eceacaa90 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563ecead8100 con 0x563ecead5350 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4df0d7f740 10 client.0 _ll_register_callbacks cb 0x563ece9f09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35260/0 (socket says 172.21.15.87:35260) 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 -- 172.21.15.87:0/4008496980 learned_addr learned my addr 172.21.15.87:0/4008496980 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de7fff640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 -- 172.21.15.87:0/4008496980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 msgr2=0x563ecead2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 -- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 msgr2=0x563ecead2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de77fe640 1 -- 172.21.15.87:0/4008496980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ece742210 con 0x563eceaca170 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.187+0000 7f4de7fff640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4de77fe640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8011890 tx=0x7f4dd801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85f1a725609bb71 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011930 con 0x563eceaca170 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f4dd8011ad0 con 0x563eceaca170 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd801cb00 con 0x563eceaca170 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4dd8011cd0 con 0x563eceaca170 2024-06-06T08:08:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 10 client.18330 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563ece80e620 con 0x563eceaca170 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd801cb00 con 0x563eceaca170 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f4dd8004ad0 con 0x563eceaca170 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 10 client.18330 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 10 client.18330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 20 client.18330 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceae53d0 0x563eceae77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 4 client.18330 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.188+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ecea41f10 con 0x563eceae53d0 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.189+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceae53d0 0x563eceae77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.189+0000 7f4de6ffd640 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceae53d0 0x563eceae77c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae16668d48f6cd2a server_cookie=6e3f05b21e25406 in_seq=0 out_seq=0 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.189+0000 7f4dc6ffd640 10 client.18330 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/4008496980 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ecea41f10 con 0x563eceae53d0 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4dc6ffd640 10 client.18330 handle_command_reply: tid=0 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 client.18330 shutdown 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 msgr2=0x563ecead2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8011890 tx=0x7f4dd801c6b0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 shutdown_connections 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceae53d0 0x563eceae77c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563ecead2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4dc6ffd640 0 client.18330 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/4008496980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563ecead2350 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 >> 172.21.15.87:0/4008496980 conn(0x563eceac6680 msgr2=0x563eceacc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 shutdown_connections 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 1 -- 172.21.15.87:0/4008496980 wait complete. 2024-06-06T08:08:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.190+0000 7f4df0d7f740 20 client.18330 trim_cache size 0 max 16384 2024-06-06T08:08:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.205+0000 7f4df0d7f740 1 Processor -- start 2024-06-06T08:08:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.205+0000 7f4df0d7f740 1 -- start start 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.205+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563eceaed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563eceacbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81b3a70 con 0x563eceaca170 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81b1340 con 0x563eceacaa90 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81af1b0 con 0x563ecead5350 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35262/0 (socket says 172.21.15.87:35262) 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 -- 172.21.15.87:0/3868649725 learned_addr learned my addr 172.21.15.87:0/3868649725 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 -- 172.21.15.87:0/3868649725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 msgr2=0x563eceacbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563eceacbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 -- 172.21.15.87:0/3868649725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 msgr2=0x563eceaed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563eceaed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 -- 172.21.15.87:0/3868649725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eceac64b0 con 0x563eceaca170 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4de77fe640 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8020040 tx=0x7f4dd80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a84afc7b04e9514 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4dcd7fa640 1 -- 172.21.15.87:0/3868649725 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800b820 con 0x563eceaca170 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4dcd7fa640 1 -- 172.21.15.87:0/3868649725 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80115c0 con 0x563eceaca170 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.206+0000 7f4dcd7fa640 1 -- 172.21.15.87:0/3868649725 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8027c40 con 0x563eceaca170 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3868649725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 msgr2=0x563eceaecf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8020040 tx=0x7f4dd80107a0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3868649725 shutdown_connections 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563ecead5350 0x563eceacbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563eceaed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/3868649725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563eceaca170 0x563eceaecf00 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3868649725 >> 172.21.15.87:0/3868649725 conn(0x563eceac6680 msgr2=0x563eceacd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3868649725 shutdown_connections 2024-06-06T08:08:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- 172.21.15.87:0/3868649725 wait complete. 2024-06-06T08:08:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 -- start start 2024-06-06T08:08:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.207+0000 7f4df0d7f740 1 Processor -- start 2024-06-06T08:08:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 0x563ecead8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81af800 con 0x563ecead5350 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81b0c70 con 0x563eceacaa90 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81afed0 con 0x563eceaca170 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 10 client.0 _ll_register_callbacks cb 0x7f4dd800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35270/0 (socket says 172.21.15.87:35270) 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 -- 172.21.15.87:0/1302863616 learned_addr learned my addr 172.21.15.87:0/1302863616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de77fe640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 0x563ecead8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 -- 172.21.15.87:0/1302863616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 msgr2=0x563ecead8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 0x563ecead8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 -- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 msgr2=0x563ecead9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 -- 172.21.15.87:0/1302863616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eceb48500 con 0x563ecead5350 2024-06-06T08:08:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de77fe640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 0x563ecead8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4de7fff640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4de00bca50 tx=0x7f4de00bc770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ca99f3395b244 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de00bcd50 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f4de00b93c0 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd80115c0 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4de00c1100 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 10 client.18336 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.208+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563eceb4f950 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80115c0 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f4de00c54a0 con 0x563ecead5350 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 10 client.18336 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 20 client.18336 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceb500c0 0x563eceb524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 4 client.18336 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ecea3bc20 con 0x563eceb500c0 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4de77fe640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceb500c0 0x563eceb524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4de77fe640 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceb500c0 0x563eceb524e0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6907bde5d26b08d1 server_cookie=23518f02f1c245bf in_seq=0 out_seq=0 2024-06-06T08:08:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.209+0000 7f4dc6ffd640 10 client.18336 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4dc6ffd640 1 -- 172.21.15.87:0/1302863616 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ecea3bc20 con 0x563eceb500c0 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4dc6ffd640 10 client.18336 handle_command_reply: tid=0 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 client.18336 shutdown 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 msgr2=0x563eceb47e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4de00bca50 tx=0x7f4de00bc770 comp rx=0 tx=0).stop 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 shutdown_connections 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x563eceb500c0 0x563eceb524e0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4dc6ffd640 0 client.18336 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563ecead5350 0x563eceb47e90 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563eceacaa90 0x563ecead9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 --2- 172.21.15.87:0/1302863616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563eceaca170 0x563ecead8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 >> 172.21.15.87:0/1302863616 conn(0x563eceac6680 msgr2=0x563eceacbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 shutdown_connections 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.456+0000 7f4df0d7f740 1 -- 172.21.15.87:0/1302863616 wait complete. 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4df0d7f740 20 client.18336 trim_cache size 0 max 16384 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:08:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:08:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 msgr2=0x7f4dd00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 0x7f4dd00a5260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4de81026e0 tx=0x7f4ddc00b700 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 msgr2=0x7f4de81016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4de0031a20 tx=0x7f4de0031a50 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 shutdown_connections 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4dd00a2da0 0x7f4dd00a5260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4de81079c0 0x7f4de81016d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4de806abb0 0x7f4de8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 --2- 172.21.15.87:0/2755524913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4de806a210 0x7f4de8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 >> 172.21.15.87:0/2755524913 conn(0x7f4de80775f0 msgr2=0x7f4de80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 shutdown_connections 2024-06-06T08:08:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:39.457+0000 7f4dedf90640 1 -- 172.21.15.87:0/2755524913 wait complete. 2024-06-06T08:08:40.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.48 MB/sec execute 282 sec latency 11546.894 ms 2024-06-06T08:08:40.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:40 smithi181 ceph-mon[36790]: pgmap v655: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-06-06T08:08:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:40 smithi087 ceph-mon[32276]: pgmap v655: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-06-06T08:08:40.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:40 smithi195 ceph-mon[37040]: pgmap v655: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-06-06T08:08:41.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.44 MB/sec execute 283 sec latency 12546.996 ms 2024-06-06T08:08:42.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.40 MB/sec execute 284 sec latency 13547.046 ms 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: pgmap v656: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:42 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: pgmap v656: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:42 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: pgmap v656: 129 pgs: 129 active+clean; 550 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-06-06T08:08:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:42.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:42 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:43.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.36 MB/sec execute 285 sec latency 14547.075 ms 2024-06-06T08:08:44.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.32 MB/sec execute 286 sec latency 15547.163 ms 2024-06-06T08:08:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:44 smithi181 ceph-mon[36790]: pgmap v657: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-06-06T08:08:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:44 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:44 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:44 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:44 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:44 smithi087 ceph-mon[32276]: pgmap v657: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-06-06T08:08:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:44 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:44 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:44 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:44 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:44 smithi195 ceph-mon[37040]: pgmap v657: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-06-06T08:08:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:44 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:44 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:44 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:44.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:44 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.28 MB/sec execute 287 sec latency 16547.243 ms 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T08:08:45.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:08:45.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:08:45.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:08:45.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi087 to 176.1M 2024-06-06T08:08:45.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi087 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi195 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.557 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:08:45.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:08:45.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:08:45.558 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:08:46.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.24 MB/sec execute 288 sec latency 17547.344 ms 2024-06-06T08:08:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:46 smithi181 ceph-mon[36790]: pgmap v658: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 958 KiB/s wr, 90 op/s 2024-06-06T08:08:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:46 smithi181 ceph-mon[36790]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T08:08:46.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:46 smithi181 ceph-mon[36790]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T08:08:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:46 smithi087 ceph-mon[32276]: pgmap v658: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 958 KiB/s wr, 90 op/s 2024-06-06T08:08:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:46 smithi087 ceph-mon[32276]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T08:08:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:46 smithi087 ceph-mon[32276]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T08:08:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:46 smithi195 ceph-mon[37040]: pgmap v658: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 958 KiB/s wr, 90 op/s 2024-06-06T08:08:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:46 smithi195 ceph-mon[37040]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-06-06T08:08:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:46 smithi195 ceph-mon[37040]: Unable to set osd_memory_target on smithi181 to 184755200: error parsing value: Value '184755200' is below minimum 939524096 2024-06-06T08:08:47.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.20 MB/sec execute 289 sec latency 18547.444 ms 2024-06-06T08:08:48.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.16 MB/sec execute 290 sec latency 19547.557 ms 2024-06-06T08:08:48.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:48 smithi181 ceph-mon[36790]: pgmap v659: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 962 KiB/s wr, 93 op/s 2024-06-06T08:08:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:48 smithi087 ceph-mon[32276]: pgmap v659: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 962 KiB/s wr, 93 op/s 2024-06-06T08:08:48.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:48 smithi195 ceph-mon[37040]: pgmap v659: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 962 KiB/s wr, 93 op/s 2024-06-06T08:08:49.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.13 MB/sec execute 291 sec latency 20547.649 ms 2024-06-06T08:08:50.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.09 MB/sec execute 292 sec latency 21547.761 ms 2024-06-06T08:08:50.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:50 smithi181 ceph-mon[36790]: pgmap v660: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-06-06T08:08:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:50 smithi087 ceph-mon[32276]: pgmap v660: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-06-06T08:08:50.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:50 smithi195 ceph-mon[37040]: pgmap v660: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-06-06T08:08:51.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.05 MB/sec execute 293 sec latency 22547.813 ms 2024-06-06T08:08:51.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:08:52.180 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:52.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 11.01 MB/sec execute 294 sec latency 23547.854 ms 2024-06-06T08:08:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 -- 172.21.15.87:0/1545202305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 msgr2=0x7fd2e4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 --2- 172.21.15.87:0/1545202305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4102e90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009900 tx=0x7fd2e002a480 comp rx=0 tx=0).stop 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 -- 172.21.15.87:0/1545202305 shutdown_connections 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 --2- 172.21.15.87:0/1545202305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 --2- 172.21.15.87:0/1545202305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 0x7fd2e410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 --2- 172.21.15.87:0/1545202305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4102e90 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 -- 172.21.15.87:0/1545202305 >> 172.21.15.87:0/1545202305 conn(0x7fd2e4077a20 msgr2=0x7fd2e4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.549+0000 7fd2ec128640 1 -- 172.21.15.87:0/1545202305 shutdown_connections 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 -- 172.21.15.87:0/1545202305 wait complete. 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 Processor -- start 2024-06-06T08:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 -- start start 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 0x7fd2e4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111d40 con 0x7fd2e4102ab0 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111bc0 con 0x7fd2e410bfd0 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2ec128640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111ec0 con 0x7fd2e41033d0 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39884/0 (socket says 172.21.15.87:39884) 2024-06-06T08:08:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.550+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/2607849292 learned_addr learned my addr 172.21.15.87:0/2607849292 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 0x7fd2e4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/2607849292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 msgr2=0x7fd2e4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 0x7fd2e4198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/2607849292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 msgr2=0x7fd2e419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/2607849292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e419d230 con 0x7fd2e4102ab0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006010 tx=0x7fd2e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2daffd640 1 -- 172.21.15.87:0/2607849292 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e003f030 con 0x7fd2e4102ab0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2daffd640 1 -- 172.21.15.87:0/2607849292 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e0003ce0 con 0x7fd2e4102ab0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e419d4c0 con 0x7fd2e4102ab0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2daffd640 1 -- 172.21.15.87:0/2607849292 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0035070 con 0x7fd2e4102ab0 2024-06-06T08:08:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.551+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a4db0 con 0x7fd2e4102ab0 2024-06-06T08:08:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.553+0000 7fd2daffd640 1 -- 172.21.15.87:0/2607849292 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd2e0042030 con 0x7fd2e4102ab0 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 Processor -- start 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 -- start start 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 0x5577a0382450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0391760 0x5577a0393b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5577a02e64f0 con 0x5577a0382990 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5577a0284870 con 0x5577a0382070 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2eee31740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5577a02e24b0 con 0x5577a0391760 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.554+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39888/0 (socket says 172.21.15.87:39888) 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2daffd640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 0x7fd2d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/567022249 learned_addr learned my addr 172.21.15.87:0/567022249 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 0x5577a0382450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0391760 0x5577a0393b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 0x7fd2d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2daffd640 1 -- 172.21.15.87:0/2607849292 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fd2e00efe90 con 0x7fd2e4102ab0 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 -- 172.21.15.87:0/567022249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0391760 msgr2=0x5577a0393b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0391760 0x5577a0393b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 -- 172.21.15.87:0/567022249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 msgr2=0x5577a0391220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 -- 172.21.15.87:0/567022249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a02a09b0 con 0x5577a0382070 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 0x7fd2d00a5260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0092f0 tx=0x7fd2dc01c000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.555+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 0x5577a0382450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005c40 tx=0x7fd2dc011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eccc35a4f29818f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2cffff640 1 -- 172.21.15.87:0/567022249 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc013720 con 0x5577a0382070 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2cffff640 1 -- 172.21.15.87:0/567022249 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc013050 con 0x5577a0382070 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2cffff640 1 -- 172.21.15.87:0/567022249 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00ce20 con 0x5577a0382070 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 -- 172.21.15.87:0/567022249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 msgr2=0x5577a0382450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 0x5577a0382450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005c40 tx=0x7fd2dc011750 comp rx=0 tx=0).stop 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 -- 172.21.15.87:0/567022249 shutdown_connections 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0391760 0x5577a0393b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a0391220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 --2- 172.21.15.87:0/567022249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0382070 0x5577a0382450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 -- 172.21.15.87:0/567022249 >> 172.21.15.87:0/567022249 conn(0x5577a037e680 msgr2=0x5577a037ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 -- 172.21.15.87:0/567022249 shutdown_connections 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.556+0000 7fd2eee31740 1 -- 172.21.15.87:0/567022249 wait complete. 2024-06-06T08:08:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.557+0000 7fd2eee31740 1 -- start start 2024-06-06T08:08:52.560 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:52 smithi087 ceph-mon[32276]: pgmap v661: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 323 KiB/s wr, 85 op/s 2024-06-06T08:08:52.562 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:52 smithi181 ceph-mon[36790]: pgmap v661: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 323 KiB/s wr, 85 op/s 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.557+0000 7fd2eee31740 1 Processor -- start 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.557+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a03910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5577a0394330 con 0x5577a0382990 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5577a026fae0 con 0x5577a0391760 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5577a03944b0 con 0x5577a0382070 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2eee31740 10 client.0 _ll_register_callbacks cb 0x5577a02a89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e96c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a03910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e96c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39904/0 (socket says 172.21.15.87:39904) 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e96c3640 1 -- 172.21.15.87:0/3417385522 learned_addr learned my addr 172.21.15.87:0/3417385522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 msgr2=0x5577a038eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/3417385522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 msgr2=0x5577a038e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/3417385522 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779fffa210 con 0x5577a0382070 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a03910f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00bbf80 tx=0x7fd2e0003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66b50bce5597a10 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.558+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0049070 con 0x5577a0382070 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc013050 con 0x5577a0382070 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fd2e00ed3e0 con 0x5577a0382070 2024-06-06T08:08:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd2e003bcb0 con 0x5577a0382070 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 10 client.27436 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5577a00c6620 con 0x5577a0382070 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc013050 con 0x5577a0382070 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd2e00dab70 con 0x5577a0382070 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 10 client.27436 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 10 client.27436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 20 client.27436 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a03a5ce0 0x5577a03a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 4 client.27436 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.559+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577a02f9f10 con 0x5577a03a5ce0 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.560+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a03a5ce0 0x5577a03a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.560+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a03a5ce0 0x5577a03a80d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a7731a904f992a server_cookie=2268ed8523e05e0a in_seq=0 out_seq=0 2024-06-06T08:08:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.560+0000 7fd2cd7fa640 10 client.27436 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/3417385522 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5577a02f9f10 con 0x5577a03a5ce0 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2cd7fa640 10 client.27436 handle_command_reply: tid=0 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 client.27436 shutdown 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 msgr2=0x5577a03910f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a03910f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00bbf80 tx=0x7fd2e0003f50 comp rx=0 tx=0).stop 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 shutdown_connections 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a03a5ce0 0x5577a03a80d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2cd7fa640 0 client.27436 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a038e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 --2- 172.21.15.87:0/3417385522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a03910f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 >> 172.21.15.87:0/3417385522 conn(0x5577a037e680 msgr2=0x5577a03844f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 shutdown_connections 2024-06-06T08:08:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 1 -- 172.21.15.87:0/3417385522 wait complete. 2024-06-06T08:08:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.561+0000 7fd2eee31740 20 client.27436 trim_cache size 0 max 16384 2024-06-06T08:08:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 Processor -- start 2024-06-06T08:08:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 -- start start 2024-06-06T08:08:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382990 0x5577a03adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41af6c0 con 0x5577a0382070 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41acf90 con 0x5577a0391760 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2eee31740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41aae00 con 0x5577a0382990 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39914/0 (socket says 172.21.15.87:39914) 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/1582212868 learned_addr learned my addr 172.21.15.87:0/1582212868 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382990 0x5577a03adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/1582212868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382990 msgr2=0x5577a03adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382990 0x5577a03adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/1582212868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 msgr2=0x5577a038b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.577+0000 7fd2e9ec4640 1 -- 172.21.15.87:0/1582212868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a03a8440 con 0x5577a0382070 2024-06-06T08:08:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.578+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005ab0 tx=0x7fd2e002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b40beee032ae8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.578+0000 7fd2cf7fe640 1 -- 172.21.15.87:0/1582212868 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0049030 con 0x5577a0382070 2024-06-06T08:08:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.578+0000 7fd2cf7fe640 1 -- 172.21.15.87:0/1582212868 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e00d7060 con 0x5577a0382070 2024-06-06T08:08:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.578+0000 7fd2cf7fe640 1 -- 172.21.15.87:0/1582212868 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0086080 con 0x5577a0382070 2024-06-06T08:08:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- 172.21.15.87:0/1582212868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 msgr2=0x5577a03ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005ab0 tx=0x7fd2e002a480 comp rx=0 tx=0).stop 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- 172.21.15.87:0/1582212868 shutdown_connections 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a038b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382990 0x5577a03adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1582212868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382070 0x5577a03ad9a0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- 172.21.15.87:0/1582212868 >> 172.21.15.87:0/1582212868 conn(0x5577a037e680 msgr2=0x5577a038cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- 172.21.15.87:0/1582212868 shutdown_connections 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- 172.21.15.87:0/1582212868 wait complete. 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 -- start start 2024-06-06T08:08:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.579+0000 7fd2eee31740 1 Processor -- start 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a039f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a0408940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41ab450 con 0x5577a0382990 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41ac8c0 con 0x5577a0391760 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41abb20 con 0x5577a0382070 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2eee31740 10 client.0 _ll_register_callbacks cb 0x7fd2e00ed100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39920/0 (socket says 172.21.15.87:39920) 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 -- 172.21.15.87:0/1373211884 learned_addr learned my addr 172.21.15.87:0/1373211884 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a039f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2ea6c5640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a0408940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 -- 172.21.15.87:0/1373211884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 msgr2=0x5577a039f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a039f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 -- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 msgr2=0x5577a0408940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a0408940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.580+0000 7fd2e96c3640 1 -- 172.21.15.87:0/1373211884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a0408f30 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2e96c3640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005fd0 tx=0x7fd2d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f88f7e3d4730e8f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4005220 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fd2d40053c0 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e00d7060 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd2d40055c0 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2eee31740 10 client.18354 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5577a0410370 con 0x5577a0382990 2024-06-06T08:08:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e00d7060 con 0x5577a0382990 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd2d400f040 con 0x5577a0382990 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2eee31740 10 client.18354 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.581+0000 7fd2eee31740 10 client.18354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2eee31740 20 client.18354 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a0410b40 0x5577a0412f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2eee31740 4 client.18354 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577a02f3c20 con 0x5577a0410b40 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a0410b40 0x5577a0412f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2cd7fa640 10 client.18354 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.582+0000 7fd2e9ec4640 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a0410b40 0x5577a0412f60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa14f6a504498d5a server_cookie=8f5171311a7929b7 in_seq=0 out_seq=0 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2cd7fa640 1 -- 172.21.15.87:0/1373211884 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5577a02f3c20 con 0x5577a0410b40 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2cd7fa640 10 client.18354 handle_command_reply: tid=0 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 client.18354 shutdown 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 msgr2=0x5577a039fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005fd0 tx=0x7fd2d4009f90 comp rx=0 tx=0).stop 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 shutdown_connections 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5577a0410b40 0x5577a0412f60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2cd7fa640 0 client.18354 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5577a0391760 0x5577a0408940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5577a0382990 0x5577a039fcf0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 --2- 172.21.15.87:0/1373211884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5577a0382070 0x5577a039f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 >> 172.21.15.87:0/1373211884 conn(0x5577a037e680 msgr2=0x5577a037cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 shutdown_connections 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 1 -- 172.21.15.87:0/1373211884 wait complete. 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.644+0000 7fd2eee31740 20 client.18354 trim_cache size 0 max 16384 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:08:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 msgr2=0x7fd2d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 0x7fd2d00a5260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0092f0 tx=0x7fd2dc01c000 comp rx=0 tx=0).stop 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 msgr2=0x7fd2e4198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006010 tx=0x7fd2e00042f0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 shutdown_connections 2024-06-06T08:08:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd2d00a2da0 0x7fd2d00a5260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e410bfd0 0x7fd2e419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2e41033d0 0x7fd2e4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 --2- 172.21.15.87:0/2607849292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2e4102ab0 0x7fd2e4198240 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 >> 172.21.15.87:0/2607849292 conn(0x7fd2e4077a20 msgr2=0x7fd2e40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 shutdown_connections 2024-06-06T08:08:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:08:52.645+0000 7fd2ec128640 1 -- 172.21.15.87:0/2607849292 wait complete. 2024-06-06T08:08:52.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:52 smithi195 ceph-mon[37040]: pgmap v661: 129 pgs: 129 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 323 KiB/s wr, 85 op/s 2024-06-06T08:08:53.184 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 10.98 MB/sec execute 295 sec latency 24547.914 ms 2024-06-06T08:08:54.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 10.94 MB/sec execute 296 sec latency 25547.944 ms 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 -- 172.21.15.87:0/360653344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec1066b0 msgr2=0x7ff0ec10e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 --2- 172.21.15.87:0/360653344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec10e280 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400b0d0 tx=0x7ff0e400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 -- 172.21.15.87:0/360653344 shutdown_connections 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 --2- 172.21.15.87:0/360653344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec10e280 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 --2- 172.21.15.87:0/360653344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101620 0x7ff0ec101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 --2- 172.21.15.87:0/360653344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec101fd0 0x7ff0ec105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 -- 172.21.15.87:0/360653344 >> 172.21.15.87:0/360653344 conn(0x7ff0ec078340 msgr2=0x7ff0ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:54.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 -- 172.21.15.87:0/360653344 shutdown_connections 2024-06-06T08:08:54.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.185+0000 7ff0f4728700 1 -- 172.21.15.87:0/360653344 wait complete. 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 Processor -- start 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 -- start start 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101fd0 0x7ff0ec19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec111710 con 0x7ff0ec101620 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec111590 con 0x7ff0ec1066b0 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f4728700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec111890 con 0x7ff0ec101fd0 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f24c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f24c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39940/0 (socket says 172.21.15.87:39940) 2024-06-06T08:08:54.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f24c4700 1 -- 172.21.15.87:0/4122584416 learned_addr learned my addr 172.21.15.87:0/4122584416 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f2cc5700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.186+0000 7ff0f1cc3700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101fd0 0x7ff0ec19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 -- 172.21.15.87:0/4122584416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101fd0 msgr2=0x7ff0ec19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101fd0 0x7ff0ec19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 -- 172.21.15.87:0/4122584416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec1066b0 msgr2=0x7ff0ec198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 -- 172.21.15.87:0/4122584416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1992b0 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f24c4700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800b530 tx=0x7ff0e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8015020 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e8011d10 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1995a0 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800f2e0 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.187+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1a5810 con 0x7ff0ec101620 2024-06-06T08:08:54.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.188+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec069180 con 0x7ff0ec101620 2024-06-06T08:08:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.189+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff0e8018020 con 0x7ff0ec101620 2024-06-06T08:08:54.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.190+0000 7ff0e37fe700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 0x7ff0d80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:54.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.190+0000 7ff0f1cc3700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 0x7ff0d80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:54.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.190+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7ff0e80cedb0 con 0x7ff0ec101620 2024-06-06T08:08:54.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.190+0000 7ff0f1cc3700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 0x7ff0d80a52d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec06bc30 tx=0x7ff0dc00a300 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:54.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.195+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0e801f020 con 0x7ff0ec101620 2024-06-06T08:08:54.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.437+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0ec199ff0 con 0x7ff0ec101620 2024-06-06T08:08:54.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.438+0000 7ff0e37fe700 1 -- 172.21.15.87:0/4122584416 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7ff0e809b6c0 con 0x7ff0ec101620 2024-06-06T08:08:54.438 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:08:54.438 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 msgr2=0x7ff0d80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 0x7ff0d80a52d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec06bc30 tx=0x7ff0dc00a300 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 msgr2=0x7ff0ec19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800b530 tx=0x7ff0e800b840 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 shutdown_connections 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec1066b0 0x7ff0ec198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff0d80a2e10 0x7ff0d80a52d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0ec101620 0x7ff0ec19e940 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 --2- 172.21.15.87:0/4122584416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec101fd0 0x7ff0ec19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 >> 172.21.15.87:0/4122584416 conn(0x7ff0ec078340 msgr2=0x7ff0ec105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:54.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 shutdown_connections 2024-06-06T08:08:54.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:54.440+0000 7ff0f4728700 1 -- 172.21.15.87:0/4122584416 wait complete. 2024-06-06T08:08:54.441 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:08:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:54 smithi181 ceph-mon[36790]: pgmap v662: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 402 KiB/s wr, 91 op/s 2024-06-06T08:08:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:54 smithi087 ceph-mon[32276]: pgmap v662: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 402 KiB/s wr, 91 op/s 2024-06-06T08:08:54.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:54 smithi195 ceph-mon[37040]: pgmap v662: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 402 KiB/s wr, 91 op/s 2024-06-06T08:08:55.090 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 79878835 --release --await' 2024-06-06T08:08:55.091 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 79878835 --release --await 2024-06-06T08:08:55.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 10.90 MB/sec execute 297 sec latency 26547.999 ms 2024-06-06T08:08:55.330 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:08:55.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:55 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4122584416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:55.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:55 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4122584416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:55.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:55 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4122584416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:08:56.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 10.87 MB/sec execute 298 sec latency 27548.082 ms 2024-06-06T08:08:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:56 smithi181 ceph-mon[36790]: pgmap v663: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 11 op/s 2024-06-06T08:08:56.587 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:56 smithi087 ceph-mon[32276]: pgmap v663: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 11 op/s 2024-06-06T08:08:56.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:56 smithi195 ceph-mon[37040]: pgmap v663: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 11 op/s 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 -- 172.21.15.87:0/1217146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 msgr2=0x7f01e4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 --2- 172.21.15.87:0/1217146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4101810 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009990 tx=0x7f01d0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 -- 172.21.15.87:0/1217146647 shutdown_connections 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 --2- 172.21.15.87:0/1217146647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4101d50 0x7f01e410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 --2- 172.21.15.87:0/1217146647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4107400 0x7f01e41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 --2- 172.21.15.87:0/1217146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4101810 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 -- 172.21.15.87:0/1217146647 >> 172.21.15.87:0/1217146647 conn(0x7f01e4078340 msgr2=0x7f01e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 -- 172.21.15.87:0/1217146647 shutdown_connections 2024-06-06T08:08:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.152+0000 7f01eacbf700 1 -- 172.21.15.87:0/1217146647 wait complete. 2024-06-06T08:08:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 Processor -- start 2024-06-06T08:08:57.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 -- start start 2024-06-06T08:08:57.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4101d50 0x7f01e419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4107400 0x7f01e41a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4111670 con 0x7f01e4101390 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41114f0 con 0x7f01e4107400 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01eacbf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41117f0 con 0x7f01e4101d50 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40996/0 (socket says 172.21.15.87:40996) 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3359934616 learned_addr learned my addr 172.21.15.87:0/3359934616 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4101d50 0x7f01e419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e925c700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4107400 0x7f01e41a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3359934616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4101d50 msgr2=0x7f01e419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4101d50 0x7f01e419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3359934616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4107400 msgr2=0x7f01e41a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4107400 0x7f01e41a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.153+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3359934616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41a50b0 con 0x7f01e4101390 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d950 tx=0x7f01d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01e1ffb700 1 -- 172.21.15.87:0/3359934616 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8016070 con 0x7f01e4101390 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41a52e0 con 0x7f01e4101390 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01e1ffb700 1 -- 172.21.15.87:0/3359934616 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d800fd70 con 0x7f01e4101390 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01e1ffb700 1 -- 172.21.15.87:0/3359934616 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d800e610 con 0x7f01e4101390 2024-06-06T08:08:57.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.154+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41a5840 con 0x7f01e4101390 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.156+0000 7f01e1ffb700 1 -- 172.21.15.87:0/3359934616 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f01d8019070 con 0x7f01e4101390 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.157+0000 7f01e1ffb700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 0x7f01d40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.157+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 0x7f01d40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.157+0000 7f01e1ffb700 1 -- 172.21.15.87:0/3359934616 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f01d80ced10 con 0x7f01e4101390 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.157+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 0x7f01d40a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b5c0 tx=0x7f01d0005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 Processor -- start 2024-06-06T08:08:57.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 -- start start 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b54ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b54d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56214b49e910 con 0x56214b552c40 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56214b446820 con 0x56214b553560 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01f5542b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56214b49a8d0 con 0x56214b54d3d0 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41004/0 (socket says 172.21.15.87:41004) 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 -- 172.21.15.87:0/2895158179 learned_addr learned my addr 172.21.15.87:0/2895158179 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e925c700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b54d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e3fff700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b54ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 -- 172.21.15.87:0/2895158179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 msgr2=0x56214b54d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b54d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 -- 172.21.15.87:0/2895158179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 msgr2=0x56214b54ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b54ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 -- 172.21.15.87:0/2895158179 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214b43d050 con 0x56214b552c40 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e925c700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b54d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f01d80cc640 tx=0x7f01d809ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43b325ae3e4ebf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.158+0000 7f01ceffd700 1 -- 172.21.15.87:0/2895158179 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8020040 con 0x56214b552c40 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01ceffd700 1 -- 172.21.15.87:0/2895158179 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d80a3a80 con 0x56214b552c40 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01ceffd700 1 -- 172.21.15.87:0/2895158179 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80a35f0 con 0x56214b552c40 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 -- 172.21.15.87:0/2895158179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 msgr2=0x56214b553020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f01d80cc640 tx=0x7f01d809ad10 comp rx=0 tx=0).stop 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 -- 172.21.15.87:0/2895158179 shutdown_connections 2024-06-06T08:08:57.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b54d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b553020 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2895158179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b54ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 -- 172.21.15.87:0/2895158179 >> 172.21.15.87:0/2895158179 conn(0x56214b477dc0 msgr2=0x56214b548120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 -- 172.21.15.87:0/2895158179 shutdown_connections 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.159+0000 7f01f5542b80 1 -- 172.21.15.87:0/2895158179 wait complete. 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 -- start start 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 Processor -- start 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b559390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b556c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56214b559fd0 con 0x56214b552c40 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56214b43ecf0 con 0x56214b553560 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56214b55a150 con 0x56214b54d3d0 2024-06-06T08:08:57.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01f5542b80 10 client.0 _ll_register_callbacks cb 0x56214b438a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41006/0 (socket says 172.21.15.87:41006) 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.160+0000 7f01e3fff700 1 -- 172.21.15.87:0/2919812037 learned_addr learned my addr 172.21.15.87:0/2919812037 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b559390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e925c700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b556c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 -- 172.21.15.87:0/2919812037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 msgr2=0x56214b559390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b559390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 -- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 msgr2=0x56214b556c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b556c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 -- 172.21.15.87:0/2919812037 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214b3ed9d0 con 0x56214b552c40 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e925c700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b556c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01e3fff700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009990 tx=0x7f01d0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ab23ebf4dcf0d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:57.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034060 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f01d002f640 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d80a3a80 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f01d003b050 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01f5542b80 10 client.18372 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56214b564940 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.161+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d80a3a80 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f01d0024aa0 con 0x56214b552c40 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 10 client.18372 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 10 client.18372 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 20 client.18372 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b566f80 0x56214b569370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 4 client.18372 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56214b4abe10 con 0x56214b566f80 2024-06-06T08:08:57.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b566f80 0x56214b569370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01c3fff700 10 client.18372 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.162+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b566f80 0x56214b569370 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78ce600286f871d server_cookie=bcea785b17606bec in_seq=0 out_seq=0 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01c3fff700 1 -- 172.21.15.87:0/2919812037 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56214b4abe10 con 0x56214b566f80 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01c3fff700 10 client.18372 handle_command_reply: tid=0 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 client.18372 shutdown 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 msgr2=0x56214b5566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009990 tx=0x7f01d0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 shutdown_connections 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b566f80 0x56214b569370 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b556c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b559390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 --2- 172.21.15.87:0/2919812037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b5566e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01c3fff700 0 client.18372 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:57.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.163+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 >> 172.21.15.87:0/2919812037 conn(0x56214b477dc0 msgr2=0x56214b54be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.164+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 shutdown_connections 2024-06-06T08:08:57.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.164+0000 7f01f5542b80 1 -- 172.21.15.87:0/2919812037 wait complete. 2024-06-06T08:08:57.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.164+0000 7f01f5542b80 20 client.18372 trim_cache size 0 max 16384 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.183+0000 7f01f5542b80 1 Processor -- start 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.183+0000 7f01f5542b80 1 -- start start 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b552c40 0x56214b554110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b55f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41b1120 con 0x56214b54d3d0 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41ae9b0 con 0x56214b553560 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01f5542b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41ac820 con 0x56214b552c40 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41020/0 (socket says 172.21.15.87:41020) 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3505211828 learned_addr learned my addr 172.21.15.87:0/3505211828 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3505211828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b552c40 msgr2=0x56214b554110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b552c40 0x56214b554110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3505211828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 msgr2=0x56214b55f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b55f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 -- 172.21.15.87:0/3505211828 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214b554c50 con 0x56214b54d3d0 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f01d809ab10 tx=0x7f01d8010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3f2f299f668f2e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01ce7fc700 1 -- 172.21.15.87:0/3505211828 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8020070 con 0x56214b54d3d0 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01ce7fc700 1 -- 172.21.15.87:0/3505211828 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d80991d0 con 0x56214b54d3d0 2024-06-06T08:08:57.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.184+0000 7f01ce7fc700 1 -- 172.21.15.87:0/3505211828 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80a3850 con 0x56214b54d3d0 2024-06-06T08:08:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- 172.21.15.87:0/3505211828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 msgr2=0x56214b5632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f01d809ab10 tx=0x7f01d8010b10 comp rx=0 tx=0).stop 2024-06-06T08:08:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- 172.21.15.87:0/3505211828 shutdown_connections 2024-06-06T08:08:57.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b55f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b54d3d0 0x56214b5632a0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3505211828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b552c40 0x56214b554110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- 172.21.15.87:0/3505211828 >> 172.21.15.87:0/3505211828 conn(0x56214b477dc0 msgr2=0x56214b553a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- 172.21.15.87:0/3505211828 shutdown_connections 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- 172.21.15.87:0/3505211828 wait complete. 2024-06-06T08:08:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 -- start start 2024-06-06T08:08:57.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 85909 10.83 MB/sec execute 299 sec latency 28548.127 ms 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.185+0000 7f01f5542b80 1 Processor -- start 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b56dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b56e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41ace70 con 0x56214b552c40 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41ae2e0 con 0x56214b553560 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41ad540 con 0x56214b54d3d0 2024-06-06T08:08:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01f5542b80 10 client.0 _ll_register_callbacks cb 0x7f01d000bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41036/0 (socket says 172.21.15.87:41036) 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 -- 172.21.15.87:0/3367461030 learned_addr learned my addr 172.21.15.87:0/3367461030 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b56dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 -- 172.21.15.87:0/3367461030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 msgr2=0x56214b56dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b56dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 -- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 msgr2=0x56214b56e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:08:57.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b56e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 -- 172.21.15.87:0/3367461030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214b5d5e80 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b56dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01e3fff700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f01d0017490 tx=0x7f01d00177c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825dc7022bf3ddd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.186+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034070 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7f01d00165b0 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56214b4a6390 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f01d00167d0 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 10 client.18378 fetch_fsmap learned FSMap version 55 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56214b5d71a0 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56214b4a6390 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f01d00243f0 con 0x56214b552c40 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 10 client.18378 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:08:57.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 10 client.18378 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 20 client.18378 populate_metadata read hostname 'smithi087' 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b5d7900 0x56214b5d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 4 client.18378 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "79878835", "release": true, "await": true}] 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "79878835", "release": true, "await": true}) -- 0x56214b320350 con 0x56214b5d7900 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.187+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b5d7900 0x56214b5d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.188+0000 7f01e8a5b700 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b5d7900 0x56214b5d9d20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0fea78159739e3 server_cookie=d6a3a388c7ffcd42 in_seq=0 out_seq=0 2024-06-06T08:08:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.188+0000 7f01c3fff700 10 client.18378 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:57.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01c3fff700 1 -- 172.21.15.87:0/3367461030 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56214b320350 con 0x56214b5d7900 2024-06-06T08:08:57.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01c3fff700 10 client.18378 handle_command_reply: tid=0 2024-06-06T08:08:57.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 client.18378 shutdown 2024-06-06T08:08:57.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 msgr2=0x56214b56e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f01d0017490 tx=0x7f01d00177c0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 shutdown_connections 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56214b5d7900 0x56214b5d9d20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56214b553560 0x56214b56e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56214b54d3d0 0x56214b56dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 --2- 172.21.15.87:0/3367461030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56214b552c40 0x56214b56e1e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 >> 172.21.15.87:0/3367461030 conn(0x56214b477dc0 msgr2=0x56214b54cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.280+0000 7f01c3fff700 0 client.18378 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.281+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 shutdown_connections 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.281+0000 7f01f5542b80 1 -- 172.21.15.87:0/3367461030 wait complete. 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.281+0000 7f01f5542b80 20 client.18378 trim_cache size 0 max 16384 2024-06-06T08:08:57.284 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 68, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "79878835": { 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 68, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:08:57.285 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 150.0, 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:08:57.286 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 msgr2=0x7f01d40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 0x7f01d40a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b5c0 tx=0x7f01d0005960 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 msgr2=0x7f01e4199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d950 tx=0x7f01d800dc60 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 shutdown_connections 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e4107400 0x7f01e41a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01d40a2d90 0x7f01d40a5250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e4101390 0x7f01e4199af0 secure :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d950 tx=0x7f01d800dc60 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 --2- 172.21.15.87:0/3359934616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e4101d50 0x7f01e419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:08:57.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 >> 172.21.15.87:0/3359934616 conn(0x7f01e4078340 msgr2=0x7f01e40fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:08:57.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.282+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 shutdown_connections 2024-06-06T08:08:57.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:08:57.283+0000 7f01eacbf700 1 -- 172.21.15.87:0/3359934616 wait complete. 2024-06-06T08:08:58.068 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '79878835', total seconds elapsed: 36.2 2024-06-06T08:08:58.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 86160 10.82 MB/sec execute 300 sec latency 28740.585 ms 2024-06-06T08:08:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:08:58 smithi087 ceph-mon[32276]: pgmap v664: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 237 KiB/s wr, 22 op/s 2024-06-06T08:08:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:08:58 smithi181 ceph-mon[36790]: pgmap v664: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 237 KiB/s wr, 22 op/s 2024-06-06T08:08:58.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:08:58 smithi195 ceph-mon[37040]: pgmap v664: 129 pgs: 129 active+clean; 542 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 237 KiB/s wr, 22 op/s 2024-06-06T08:08:59.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 86430 10.85 MB/sec execute 301 sec latency 286.062 ms 2024-06-06T08:09:00.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 86782 10.88 MB/sec execute 302 sec latency 182.909 ms 2024-06-06T08:09:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:00 smithi181 ceph-mon[36790]: pgmap v665: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 605 KiB/s wr, 41 op/s 2024-06-06T08:09:00.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:00 smithi087 ceph-mon[32276]: pgmap v665: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 605 KiB/s wr, 41 op/s 2024-06-06T08:09:00.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:00 smithi195 ceph-mon[37040]: pgmap v665: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 605 KiB/s wr, 41 op/s 2024-06-06T08:09:01.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 87053 10.88 MB/sec execute 303 sec latency 193.855 ms 2024-06-06T08:09:02.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 87308 10.88 MB/sec execute 304 sec latency 192.317 ms 2024-06-06T08:09:02.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:02 smithi181 ceph-mon[36790]: pgmap v666: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 588 KiB/s wr, 38 op/s 2024-06-06T08:09:02.561 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:02 smithi195 ceph-mon[37040]: pgmap v666: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 588 KiB/s wr, 38 op/s 2024-06-06T08:09:02.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:02 smithi087 ceph-mon[32276]: pgmap v666: 129 pgs: 129 active+clean; 545 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 588 KiB/s wr, 38 op/s 2024-06-06T08:09:03.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 87464 10.87 MB/sec execute 305 sec latency 305.581 ms 2024-06-06T08:09:04.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 87687 10.86 MB/sec execute 306 sec latency 354.646 ms 2024-06-06T08:09:04.588 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:04 smithi087 ceph-mon[32276]: pgmap v667: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-06T08:09:04.661 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:04 smithi181 ceph-mon[36790]: pgmap v667: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-06T08:09:04.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:04 smithi195 ceph-mon[37040]: pgmap v667: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-06T08:09:05.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 87868 10.85 MB/sec execute 307 sec latency 345.385 ms 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3890186465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb18108bb0 msgr2=0x7fcb1810cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3890186465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb18108bb0 0x7fcb1810cbf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802a3f0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3890186465 shutdown_connections 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3890186465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb1810d2d0 0x7fcb18114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3890186465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb18108bb0 0x7fcb1810cbf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3890186465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb18108200 0x7fcb181085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3890186465 >> 172.21.15.87:0/3890186465 conn(0x7fcb18075f30 msgr2=0x7fcb18078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3890186465 shutdown_connections 2024-06-06T08:09:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.773+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3890186465 wait complete. 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 Processor -- start 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 -- start start 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 0x7fcb181a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb1819e5f0 0x7fcb1819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18118390 con 0x7fcb18108200 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18118210 con 0x7fcb1810d2d0 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1ffe3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18118510 con 0x7fcb1819e5f0 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1dd7f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1dd7f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41064/0 (socket says 172.21.15.87:41064) 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.774+0000 7fcb1dd7f640 1 -- 172.21.15.87:0/3399585766 learned_addr learned my addr 172.21.15.87:0/3399585766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb1819e5f0 0x7fcb1819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 0x7fcb181a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 -- 172.21.15.87:0/3399585766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb1819e5f0 msgr2=0x7fcb1819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb1819e5f0 0x7fcb1819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 -- 172.21.15.87:0/3399585766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 msgr2=0x7fcb181a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 0x7fcb181a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 -- 172.21.15.87:0/3399585766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1819f110 con 0x7fcb18108200 2024-06-06T08:09:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 0x7fcb181a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9f0 tx=0x7fcb1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb0effd640 1 -- 172.21.15.87:0/3399585766 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400cd80 con 0x7fcb18108200 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1819f400 con 0x7fcb18108200 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb0effd640 1 -- 172.21.15.87:0/3399585766 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14010910 con 0x7fcb18108200 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb0effd640 1 -- 172.21.15.87:0/3399585766 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018990 con 0x7fcb18108200 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.775+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181ab650 con 0x7fcb18108200 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.779+0000 7fcb22dcd740 1 Processor -- start 2024-06-06T08:09:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.779+0000 7fcb22dcd740 1 -- start start 2024-06-06T08:09:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.779+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d879550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.779+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d883e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b39d7dd4f0 con 0x55b39d879a90 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b39d77b870 con 0x55b39d879170 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b39d7d94b0 con 0x55b39d884350 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb0effd640 1 -- 172.21.15.87:0/3399585766 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcb14010ac0 con 0x7fcb18108200 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:56026/0 (socket says 172.21.15.87:56026) 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 -- 172.21.15.87:0/3335270584 learned_addr learned my addr 172.21.15.87:0/3335270584 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d879550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d883e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 -- 172.21.15.87:0/3335270584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 msgr2=0x55b39d879550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d879550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.780+0000 7fcb1d57e640 1 -- 172.21.15.87:0/3335270584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 msgr2=0x55b39d883e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb0effd640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 0x7fcb040a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d883e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb1d57e640 1 -- 172.21.15.87:0/3335270584 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39d7979b0 con 0x55b39d884350 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d883e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb0effd640 1 -- 172.21.15.87:0/3399585766 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fcb140cfc90 con 0x7fcb18108200 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.781+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009850 tx=0x7fcb080314d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589b292602e9b878 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.782+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 0x7fcb040a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.782+0000 7fcb03fff640 1 -- 172.21.15.87:0/3335270584 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0803f070 con 0x55b39d884350 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.782+0000 7fcb03fff640 1 -- 172.21.15.87:0/3335270584 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb080043c0 con 0x55b39d884350 2024-06-06T08:09:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.782+0000 7fcb03fff640 1 -- 172.21.15.87:0/3335270584 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08035030 con 0x55b39d884350 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3335270584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 msgr2=0x55b39d886740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 0x7fcb040a5330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005fd0 tx=0x7fcb10009500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009850 tx=0x7fcb080314d0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3335270584 shutdown_connections 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d886740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d883e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3335270584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d879550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.783+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3335270584 >> 172.21.15.87:0/3335270584 conn(0x55b39d875680 msgr2=0x55b39d875a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.784+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3335270584 shutdown_connections 2024-06-06T08:09:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.784+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3335270584 wait complete. 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.784+0000 7fcb22dcd740 1 -- start start 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.784+0000 7fcb22dcd740 1 Processor -- start 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d881290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d881d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b39d886f20 con 0x55b39d879a90 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b39d766ae0 con 0x55b39d879170 2024-06-06T08:09:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b39d8870a0 con 0x55b39d884350 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb22dcd740 10 client.0 _ll_register_callbacks cb 0x55b39d79f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41088/0 (socket says 172.21.15.87:41088) 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 -- 172.21.15.87:0/1401151749 learned_addr learned my addr 172.21.15.87:0/1401151749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1e580640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d881d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d881290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 -- 172.21.15.87:0/1401151749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 msgr2=0x55b39d881d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d881d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 -- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 msgr2=0x55b39d881290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d881290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1d57e640 1 -- 172.21.15.87:0/1401151749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39d4f1210 con 0x55b39d879a90 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1e580640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d881d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.785+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d881290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:09:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcb0802a430 tx=0x7fcb08004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449ae6f718f8755e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0804c020 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fcb08003a00 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb080043c0 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcb08057050 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb22dcd740 10 client.18387 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b39d5bd620 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.786+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb080043c0 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcb08004c30 con 0x55b39d879a90 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 10 client.18387 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 10 client.18387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 20 client.18387 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d894220 0x55b39d896610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 4 client.18387 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:09:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39d7f0f10 con 0x55b39d894220 2024-06-06T08:09:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d894220 0x55b39d896610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb017fa640 10 client.18387 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.787+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d894220 0x55b39d896610 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f961470adfdaf4 server_cookie=6f49c5d1eb8f5f1a in_seq=0 out_seq=0 2024-06-06T08:09:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.791+0000 7fcb017fa640 1 -- 172.21.15.87:0/1401151749 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b39d7f0f10 con 0x55b39d894220 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.791+0000 7fcb017fa640 10 client.18387 handle_command_reply: tid=0 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.791+0000 7fcb22dcd740 1 client.18387 shutdown 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.791+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 msgr2=0x55b39d8817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.791+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcb0802a430 tx=0x7fcb08004970 comp rx=0 tx=0).stop 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 shutdown_connections 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d894220 0x55b39d896610 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d884350 0x55b39d881d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d879a90 0x55b39d8817d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/1401151749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d881290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 >> 172.21.15.87:0/1401151749 conn(0x55b39d875680 msgr2=0x55b39d87b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb017fa640 0 client.18387 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 shutdown_connections 2024-06-06T08:09:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 1 -- 172.21.15.87:0/1401151749 wait complete. 2024-06-06T08:09:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.792+0000 7fcb22dcd740 20 client.18387 trim_cache size 0 max 16384 2024-06-06T08:09:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.814+0000 7fcb22dcd740 1 Processor -- start 2024-06-06T08:09:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.814+0000 7fcb22dcd740 1 -- start start 2024-06-06T08:09:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d8839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879a90 0x55b39d87bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b7ec0 con 0x55b39d884350 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b5790 con 0x55b39d879170 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b3600 con 0x55b39d879a90 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1d57e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879a90 0x55b39d87bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41102/0 (socket says 172.21.15.87:41102) 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 -- 172.21.15.87:0/2558138580 learned_addr learned my addr 172.21.15.87:0/2558138580 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d8839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 -- 172.21.15.87:0/2558138580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879a90 msgr2=0x55b39d87bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879a90 0x55b39d87bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 -- 172.21.15.87:0/2558138580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 msgr2=0x55b39d8839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d8839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 -- 172.21.15.87:0/2558138580 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39d890c90 con 0x55b39d884350 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.815+0000 7fcb1e580640 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fcb10003d60 tx=0x7fcb10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68e73892a4f8ab2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.816+0000 7fcb037fe640 1 -- 172.21.15.87:0/2558138580 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1001e460 con 0x55b39d884350 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.816+0000 7fcb037fe640 1 -- 172.21.15.87:0/2558138580 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1001eae0 con 0x55b39d884350 2024-06-06T08:09:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.816+0000 7fcb037fe640 1 -- 172.21.15.87:0/2558138580 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1001f970 con 0x55b39d884350 2024-06-06T08:09:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- 172.21.15.87:0/2558138580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 msgr2=0x55b39d89bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fcb10003d60 tx=0x7fcb10017000 comp rx=0 tx=0).stop 2024-06-06T08:09:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- 172.21.15.87:0/2558138580 shutdown_connections 2024-06-06T08:09:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d89bd00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879a90 0x55b39d87bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/2558138580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879170 0x55b39d8839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- 172.21.15.87:0/2558138580 >> 172.21.15.87:0/2558138580 conn(0x55b39d875680 msgr2=0x55b39d879fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- 172.21.15.87:0/2558138580 shutdown_connections 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- 172.21.15.87:0/2558138580 wait complete. 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.817+0000 7fcb22dcd740 1 -- start start 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 Processor -- start 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879170 0x55b39d8872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879a90 0x55b39d8877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b3c50 con 0x55b39d884350 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b50c0 con 0x55b39d879a90 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181b4320 con 0x55b39d879170 2024-06-06T08:09:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb22dcd740 10 client.0 _ll_register_callbacks cb 0x7fcb14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41108/0 (socket says 172.21.15.87:41108) 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 -- 172.21.15.87:0/3389666929 learned_addr learned my addr 172.21.15.87:0/3389666929 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1d57e640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879a90 0x55b39d8877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 -- 172.21.15.87:0/3389666929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879170 msgr2=0x55b39d8872b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879170 0x55b39d8872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 -- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879a90 msgr2=0x55b39d8877f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.818+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879a90 0x55b39d8877f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb1e580640 1 -- 172.21.15.87:0/3389666929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39d8fcdc0 con 0x55b39d884350 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb1e580640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcb1001e690 tx=0x7fcb1001e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fab5843b5d581a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1001e8e0 con 0x55b39d884350 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9583+0+0 (secure 0 0 0) 0x7fcb10026040 con 0x55b39d884350 2024-06-06T08:09:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb1001eae0 con 0x55b39d884350 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcb1003f050 con 0x55b39d884350 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb22dcd740 10 client.18393 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.819+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b39d8fe0d0 con 0x55b39d884350 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1001eae0 con 0x55b39d884350 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcb1003a030 con 0x55b39d884350 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 10 client.18393 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 10 client.18393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 20 client.18393 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d8fe7f0 0x55b39d900c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 4 client.18393 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.820+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b39d7eac20 con 0x55b39d8fe7f0 2024-06-06T08:09:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.821+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d8fe7f0 0x55b39d900c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.822+0000 7fcb1dd7f640 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d8fe7f0 0x55b39d900c10 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed822dc6453474e server_cookie=d571fd3a878b8dd7 in_seq=0 out_seq=0 2024-06-06T08:09:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:05.822+0000 7fcb017fa640 10 client.18393 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:06.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88010 10.85 MB/sec execute 308 sec latency 500.015 ms 2024-06-06T08:09:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.531+0000 7fcb017fa640 1 -- 172.21.15.87:0/3389666929 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b39d7eac20 con 0x55b39d8fe7f0 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.531+0000 7fcb017fa640 10 client.18393 handle_command_reply: tid=0 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.531+0000 7fcb22dcd740 1 client.18393 shutdown 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.531+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 msgr2=0x55b39d887d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.531+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcb1001e690 tx=0x7fcb1001e6c0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 shutdown_connections 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55b39d8fe7f0 0x55b39d900c10 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b39d884350 0x55b39d887d30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb017fa640 0 client.18393 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b39d879a90 0x55b39d8877f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 --2- 172.21.15.87:0/3389666929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b39d879170 0x55b39d8872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 >> 172.21.15.87:0/3389666929 conn(0x55b39d875680 msgr2=0x55b39d881080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 shutdown_connections 2024-06-06T08:09:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 1 -- 172.21.15.87:0/3389666929 wait complete. 2024-06-06T08:09:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.532+0000 7fcb22dcd740 20 client.18393 trim_cache size 0 max 16384 2024-06-06T08:09:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:09:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:09:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 msgr2=0x7fcb040a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 0x7fcb040a5330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005fd0 tx=0x7fcb10009500 comp rx=0 tx=0).stop 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 msgr2=0x7fcb181a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9f0 tx=0x7fcb1400eeb0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 shutdown_connections 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcb040a2e70 0x7fcb040a5330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb1819e5f0 0x7fcb1819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb1810d2d0 0x7fcb181a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 --2- 172.21.15.87:0/3399585766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb18108200 0x7fcb181a4440 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 >> 172.21.15.87:0/3399585766 conn(0x7fcb18075f30 msgr2=0x7fcb18076a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 shutdown_connections 2024-06-06T08:09:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:06.533+0000 7fcb1ffe3640 1 -- 172.21.15.87:0/3399585766 wait complete. 2024-06-06T08:09:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:06 smithi087 ceph-mon[32276]: pgmap v668: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-06-06T08:09:06.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:06 smithi195 ceph-mon[37040]: pgmap v668: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-06-06T08:09:06.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:06 smithi181 ceph-mon[36790]: pgmap v668: 129 pgs: 129 active+clean; 554 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-06-06T08:09:07.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88157 10.84 MB/sec execute 309 sec latency 959.204 ms 2024-06-06T08:09:07.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:07 smithi087 ceph-mon[32276]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T08:09:07.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:07 smithi195 ceph-mon[37040]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T08:09:07.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:07 smithi181 ceph-mon[36790]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T08:09:08.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88259 10.82 MB/sec execute 310 sec latency 1254.602 ms 2024-06-06T08:09:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:08 smithi087 ceph-mon[32276]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-06-06T08:09:08.766 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:08 smithi195 ceph-mon[37040]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-06-06T08:09:08.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:08 smithi181 ceph-mon[36790]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-06-06T08:09:09.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88407 10.83 MB/sec execute 311 sec latency 1043.192 ms 2024-06-06T08:09:10.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88511 10.81 MB/sec execute 312 sec latency 1000.151 ms 2024-06-06T08:09:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:10 smithi087 ceph-mon[32276]: pgmap v671: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:10.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:10 smithi195 ceph-mon[37040]: pgmap v671: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:10.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:10 smithi181 ceph-mon[36790]: pgmap v671: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:11.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88632 10.80 MB/sec execute 313 sec latency 1111.213 ms 2024-06-06T08:09:12.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 88790 10.79 MB/sec execute 314 sec latency 1032.811 ms 2024-06-06T08:09:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:12 smithi087 ceph-mon[32276]: pgmap v672: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:12 smithi195 ceph-mon[37040]: pgmap v672: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:12.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:12 smithi181 ceph-mon[36790]: pgmap v672: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 549 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 562 op/s 2024-06-06T08:09:13.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 89032 10.80 MB/sec execute 315 sec latency 500.234 ms 2024-06-06T08:09:14.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 89275 10.81 MB/sec execute 316 sec latency 224.973 ms 2024-06-06T08:09:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:14 smithi087 ceph-mon[32276]: pgmap v673: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:14 smithi195 ceph-mon[37040]: pgmap v673: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:14 smithi181 ceph-mon[36790]: pgmap v673: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:15.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 89575 10.84 MB/sec execute 317 sec latency 233.563 ms 2024-06-06T08:09:16.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 89836 10.86 MB/sec execute 318 sec latency 505.196 ms 2024-06-06T08:09:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:16 smithi087 ceph-mon[32276]: pgmap v674: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:16.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:16 smithi195 ceph-mon[37040]: pgmap v674: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:16.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:16 smithi181 ceph-mon[36790]: pgmap v674: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 546 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 463 op/s 2024-06-06T08:09:17.185 INFO:tasks.workunit.client.0.smithi087.stdout: 10 90095 10.88 MB/sec execute 319 sec latency 527.712 ms 2024-06-06T08:09:18.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 90345 10.88 MB/sec execute 320 sec latency 454.161 ms 2024-06-06T08:09:18.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:18 smithi195 ceph-mon[37040]: pgmap v675: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 548 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-06-06T08:09:18.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:18 smithi181 ceph-mon[36790]: pgmap v675: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 548 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-06-06T08:09:18.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:18 smithi087 ceph-mon[32276]: pgmap v675: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 548 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-06-06T08:09:19.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 90566 10.86 MB/sec execute 321 sec latency 232.434 ms 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 -- 172.21.15.87:0/3404141691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701008a0 msgr2=0x7fd170100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 --2- 172.21.15.87:0/3404141691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701008a0 0x7fd170100d20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd164009930 tx=0x7fd16402a420 comp rx=0 tx=0).stop 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 -- 172.21.15.87:0/3404141691 shutdown_connections 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 --2- 172.21.15.87:0/3404141691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd17010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 --2- 172.21.15.87:0/3404141691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701008a0 0x7fd170100d20 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 --2- 172.21.15.87:0/3404141691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701068b0 0x7fd170106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 -- 172.21.15.87:0/3404141691 >> 172.21.15.87:0/3404141691 conn(0x7fd1700779b0 msgr2=0x7fd170077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 -- 172.21.15.87:0/3404141691 shutdown_connections 2024-06-06T08:09:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 -- 172.21.15.87:0/3404141691 wait complete. 2024-06-06T08:09:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.649+0000 7fd1767a9640 1 Processor -- start 2024-06-06T08:09:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 -- start start 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 0x7fd1701981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110cc0 con 0x7fd1701068b0 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110b40 con 0x7fd1701008a0 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd1767a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110e40 con 0x7fd170101260 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd174d46640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 0x7fd1701981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47950/0 (socket says 172.21.15.87:47950) 2024-06-06T08:09:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd174d46640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55586/0 (socket says 172.21.15.87:55586) 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.650+0000 7fd16f7fe640 1 -- 172.21.15.87:0/4158307860 learned_addr learned my addr 172.21.15.87:0/4158307860 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 -- 172.21.15.87:0/4158307860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 msgr2=0x7fd170198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 -- 172.21.15.87:0/4158307860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 msgr2=0x7fd1701981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 0x7fd1701981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 -- 172.21.15.87:0/4158307860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17019d1c0 con 0x7fd1701068b0 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd16ffff640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 0x7fd1701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd174d46640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd168009fd0 tx=0x7fd16800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd16d7fa640 1 -- 172.21.15.87:0/4158307860 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16800ce20 con 0x7fd1701068b0 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd16d7fa640 1 -- 172.21.15.87:0/4158307860 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd168005020 con 0x7fd1701068b0 2024-06-06T08:09:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd16d7fa640 1 -- 172.21.15.87:0/4158307860 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168010720 con 0x7fd1701068b0 2024-06-06T08:09:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17019d4b0 con 0x7fd1701068b0 2024-06-06T08:09:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.651+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd170107740 con 0x7fd1701068b0 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.654+0000 7fd16d7fa640 1 -- 172.21.15.87:0/4158307860 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd168005540 con 0x7fd1701068b0 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd16d7fa640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 0x7fd1580a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd16d7fa640 1 -- 172.21.15.87:0/4158307860 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9671+0+0 (secure 0 0 0) 0x7fd1680cfe40 con 0x7fd1701068b0 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 Processor -- start 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 -- start start 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 0x556e02da8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 0x556e02db5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e02d0c4f0 con 0x556e02da8170 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e02caa870 con 0x556e02db3350 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.655+0000 7fd179599740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e02d084b0 con 0x556e02da8a90 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd174d46640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 0x556e02db5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 0x556e02da8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd174d46640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 0x556e02db5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54318/0 (socket says 172.21.15.87:54318) 2024-06-06T08:09:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd174d46640 1 -- 172.21.15.87:0/946289746 learned_addr learned my addr 172.21.15.87:0/946289746 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 -- 172.21.15.87:0/946289746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 msgr2=0x556e02db2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 -- 172.21.15.87:0/946289746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 msgr2=0x556e02db5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 0x556e02db5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.656+0000 7fd16ffff640 1 -- 172.21.15.87:0/946289746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e02cc69b0 con 0x556e02da8170 2024-06-06T08:09:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.657+0000 7fd16ffff640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 0x7fd1580a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.657+0000 7fd16ffff640 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 0x556e02da8550 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd1600114c0 tx=0x7fd160011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc374794f19f6272 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.657+0000 7fd1567fc640 1 -- 172.21.15.87:0/946289746 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000f7e0 con 0x556e02da8170 2024-06-06T08:09:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.657+0000 7fd1567fc640 1 -- 172.21.15.87:0/946289746 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16001cdc0 con 0x556e02da8170 2024-06-06T08:09:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.657+0000 7fd1567fc640 1 -- 172.21.15.87:0/946289746 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16001ae20 con 0x556e02da8170 2024-06-06T08:09:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 -- 172.21.15.87:0/946289746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 msgr2=0x556e02da8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 0x556e02da8550 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd1600114c0 tx=0x7fd160011980 comp rx=0 tx=0).stop 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd16ffff640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 0x7fd1580a53c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd160021000 tx=0x7fd16001e000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 -- 172.21.15.87:0/946289746 shutdown_connections 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02db3350 0x556e02db5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 --2- 172.21.15.87:0/946289746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8170 0x556e02da8550 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 -- 172.21.15.87:0/946289746 >> 172.21.15.87:0/946289746 conn(0x556e02da4680 msgr2=0x556e02da4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 -- 172.21.15.87:0/946289746 shutdown_connections 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.658+0000 7fd179599740 1 -- 172.21.15.87:0/946289746 wait complete. 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 -- start start 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 Processor -- start 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02db0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556e02db5f80 con 0x556e02db3350 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e02c95ae0 con 0x556e02da8170 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.659+0000 7fd179599740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e02db6100 con 0x556e02da8a90 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd179599740 10 client.0 _ll_register_callbacks cb 0x556e02cce9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54328/0 (socket says 172.21.15.87:54328) 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 -- 172.21.15.87:0/1895270766 learned_addr learned my addr 172.21.15.87:0/1895270766 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd174d46640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02db0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 -- 172.21.15.87:0/1895270766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 msgr2=0x556e02db0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 -- 172.21.15.87:0/1895270766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 msgr2=0x556e02db0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02db0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 -- 172.21.15.87:0/1895270766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e02a20210 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd174d46640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02db0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd16ffff640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd16001a9e0 tx=0x7fd160009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fcf6f0b9801e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd14f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16002a050 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16001cdc0 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.660+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9671+0+0 (secure 0 0 0) 0x7fd16001a400 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd16003d050 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 10 client.27302 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556e02aec620 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16001cdc0 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd160004660 con 0x556e02da8170 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 10 client.27302 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 10 client.27302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 20 client.27302 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02dc3480 0x556e02dc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 4 client.27302 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e02d1ff10 con 0x556e02dc3480 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.661+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02dc3480 0x556e02dc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.662+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02dc3480 0x556e02dc5870 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20a5b2063a30ec6 server_cookie=3d729bf5e8f0b970 in_seq=0 out_seq=0 2024-06-06T08:09:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.662+0000 7fd14f7fe640 10 client.27302 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd14f7fe640 1 -- 172.21.15.87:0/1895270766 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e02d1ff10 con 0x556e02dc3480 2024-06-06T08:09:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd14f7fe640 10 client.27302 handle_command_reply: tid=0 2024-06-06T08:09:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 client.27302 shutdown 2024-06-06T08:09:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 msgr2=0x556e02db0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd16001a9e0 tx=0x7fd160009310 comp rx=0 tx=0).stop 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 shutdown_connections 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02dc3480 0x556e02dc5870 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02db0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02db0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd14f7fe640 0 client.27302 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 --2- 172.21.15.87:0/1895270766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02db0350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 >> 172.21.15.87:0/1895270766 conn(0x556e02da4680 msgr2=0x556e02daa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 shutdown_connections 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 1 -- 172.21.15.87:0/1895270766 wait complete. 2024-06-06T08:09:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.665+0000 7fd179599740 20 client.27302 trim_cache size 0 max 16384 2024-06-06T08:09:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 Processor -- start 2024-06-06T08:09:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 -- start start 2024-06-06T08:09:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02da3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02dcae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701af7d0 con 0x556e02db3350 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ad0a0 con 0x556e02da8170 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.681+0000 7fd179599740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701aaf10 con 0x556e02da8a90 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55604/0 (socket says 172.21.15.87:55604) 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 -- 172.21.15.87:0/1075932625 learned_addr learned my addr 172.21.15.87:0/1075932625 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02dcae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd16ffff640 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02da3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 -- 172.21.15.87:0/1075932625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 msgr2=0x556e02dcae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02dcae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 -- 172.21.15.87:0/1075932625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 msgr2=0x556e02da3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02da3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 -- 172.21.15.87:0/1075932625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e02dabd40 con 0x556e02db3350 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd174d46640 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd168098850 tx=0x7fd1680bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5b6b1b303f79f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd155ffb640 1 -- 172.21.15.87:0/1075932625 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680984f0 con 0x556e02db3350 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd155ffb640 1 -- 172.21.15.87:0/1075932625 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1680b8070 con 0x556e02db3350 2024-06-06T08:09:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.682+0000 7fd155ffb640 1 -- 172.21.15.87:0/1075932625 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680a4660 con 0x556e02db3350 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 -- 172.21.15.87:0/1075932625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 msgr2=0x556e02dcb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd168098850 tx=0x7fd1680bc040 comp rx=0 tx=0).stop 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 -- 172.21.15.87:0/1075932625 shutdown_connections 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02db3350 0x556e02dcb3b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02da8a90 0x556e02dcae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 --2- 172.21.15.87:0/1075932625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02da3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 -- 172.21.15.87:0/1075932625 >> 172.21.15.87:0/1075932625 conn(0x556e02da4680 msgr2=0x556e02dab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 -- 172.21.15.87:0/1075932625 shutdown_connections 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.683+0000 7fd179599740 1 -- 172.21.15.87:0/1075932625 wait complete. 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 -- start start 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 Processor -- start 2024-06-06T08:09:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02db3350 0x556e02db7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ab560 con 0x556e02da8a90 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ac9d0 con 0x556e02da8170 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701abc30 con 0x556e02db3350 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd179599740 10 client.0 _ll_register_callbacks cb 0x7fd168098120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:54354/0 (socket says 172.21.15.87:54354) 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55606/0 (socket says 172.21.15.87:55606) 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd16ffff640 1 -- 172.21.15.87:0/2710257982 learned_addr learned my addr 172.21.15.87:0/2710257982 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.684+0000 7fd174d46640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02db3350 0x556e02db7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 -- 172.21.15.87:0/2710257982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02db3350 msgr2=0x556e02db7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02db3350 0x556e02db7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 -- 172.21.15.87:0/2710257982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 msgr2=0x556e02db6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 -- 172.21.15.87:0/2710257982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e02db7b40 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd16ffff640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160005860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c0e82d06923d662 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd14f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16002a040 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1680b8070 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9671+0+0 (secure 0 0 0) 0x7fd16001cb70 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd16004e050 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd179599740 10 client.27308 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556e02e2de20 con 0x556e02da8170 2024-06-06T08:09:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.685+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1680b8070 con 0x556e02da8170 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd160049030 con 0x556e02da8170 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 10 client.27308 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 10 client.27308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 20 client.27308 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02e2e530 0x556e02e30920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 4 client.27308 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e02d19c20 con 0x556e02e2e530 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02e2e530 0x556e02e30920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd14f7fe640 10 client.27308 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:19.686+0000 7fd16f7fe640 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02e2e530 0x556e02e30920 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56042360fb77c6f server_cookie=e2ef4a268c20dd8b in_seq=0 out_seq=0 2024-06-06T08:09:20.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 90742 10.85 MB/sec execute 322 sec latency 477.264 ms 2024-06-06T08:09:20.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:20 smithi195 ceph-mon[37040]: pgmap v676: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-06-06T08:09:20.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:20 smithi195 ceph-mon[37040]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T08:09:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:20 smithi181 ceph-mon[36790]: pgmap v676: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-06-06T08:09:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:20 smithi181 ceph-mon[36790]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T08:09:20.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:20 smithi087 ceph-mon[32276]: pgmap v676: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-06-06T08:09:20.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:20 smithi087 ceph-mon[32276]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T08:09:21.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 91120 10.88 MB/sec execute 323 sec latency 480.711 ms 2024-06-06T08:09:22.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 91496 10.92 MB/sec execute 324 sec latency 1393.733 ms 2024-06-06T08:09:22.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:22 smithi087 ceph-mon[32276]: pgmap v678: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-06-06T08:09:22.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:22 smithi195 ceph-mon[37040]: pgmap v678: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-06-06T08:09:22.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:22 smithi181 ceph-mon[36790]: pgmap v678: 129 pgs: 129 active+clean; 559 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.5 MiB/s wr, 250 op/s 2024-06-06T08:09:23.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 91776 10.94 MB/sec execute 325 sec latency 2393.763 ms 2024-06-06T08:09:24.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 92124 10.96 MB/sec execute 326 sec latency 3393.796 ms 2024-06-06T08:09:24.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:24 smithi195 ceph-mon[37040]: pgmap v679: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:24.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:24 smithi181 ceph-mon[36790]: pgmap v679: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:24.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:24 smithi087 ceph-mon[32276]: pgmap v679: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:25.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 92492 10.98 MB/sec execute 327 sec latency 4393.830 ms 2024-06-06T08:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd14f7fe640 1 -- 172.21.15.87:0/2710257982 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e02d19c20 con 0x556e02e2e530 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd14f7fe640 10 client.27308 handle_command_reply: tid=0 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 client.27308 shutdown 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 msgr2=0x556e02dbba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160005860 comp rx=0 tx=0).stop 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 shutdown_connections 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556e02e2e530 0x556e02e30920 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e02db3350 0x556e02db7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556e02da8a90 0x556e02db6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 --2- 172.21.15.87:0/2710257982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e02da8170 0x556e02dbba80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 >> 172.21.15.87:0/2710257982 conn(0x556e02da4680 msgr2=0x556e02db02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd14f7fe640 0 client.27308 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 shutdown_connections 2024-06-06T08:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 1 -- 172.21.15.87:0/2710257982 wait complete. 2024-06-06T08:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.912+0000 7fd179599740 20 client.27308 trim_cache size 0 max 16384 2024-06-06T08:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 msgr2=0x7fd1580a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 0x7fd1580a53c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd160021000 tx=0x7fd16001e000 comp rx=0 tx=0).stop 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 msgr2=0x7fd17019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd168009fd0 tx=0x7fd16800ef10 comp rx=0 tx=0).stop 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 shutdown_connections 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd1580a2f00 0x7fd1580a53c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1701068b0 0x7fd17019ca90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170101260 0x7fd170198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 --2- 172.21.15.87:0/4158307860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1701008a0 0x7fd1701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.913+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 >> 172.21.15.87:0/4158307860 conn(0x7fd1700779b0 msgr2=0x7fd1701023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.914+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 shutdown_connections 2024-06-06T08:09:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:25.914+0000 7fd1767a9640 1 -- 172.21.15.87:0/4158307860 wait complete. 2024-06-06T08:09:26.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 92845 11.01 MB/sec execute 328 sec latency 5272.428 ms 2024-06-06T08:09:26.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:26 smithi195 ceph-mon[37040]: pgmap v680: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:26.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:26 smithi181 ceph-mon[36790]: pgmap v680: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:26 smithi087 ceph-mon[32276]: pgmap v680: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-06-06T08:09:27.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 93224 11.04 MB/sec execute 329 sec latency 204.677 ms 2024-06-06T08:09:28.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 93514 11.06 MB/sec execute 330 sec latency 200.710 ms 2024-06-06T08:09:28.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:28 smithi195 ceph-mon[37040]: pgmap v681: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-06T08:09:28.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:28 smithi181 ceph-mon[36790]: pgmap v681: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-06T08:09:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:28 smithi087 ceph-mon[32276]: pgmap v681: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-06T08:09:29.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 93820 11.07 MB/sec execute 331 sec latency 183.249 ms 2024-06-06T08:09:30.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94119 11.09 MB/sec execute 332 sec latency 214.980 ms 2024-06-06T08:09:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:30 smithi195 ceph-mon[37040]: pgmap v682: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-06T08:09:30.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:30 smithi181 ceph-mon[36790]: pgmap v682: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-06T08:09:30.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:30 smithi087 ceph-mon[32276]: pgmap v682: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-06T08:09:31.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94383 11.11 MB/sec execute 333 sec latency 269.629 ms 2024-06-06T08:09:32.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94600 11.09 MB/sec execute 334 sec latency 294.435 ms 2024-06-06T08:09:32.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:32 smithi195 ceph-mon[37040]: pgmap v683: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-06-06T08:09:32.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:32 smithi181 ceph-mon[36790]: pgmap v683: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-06-06T08:09:32.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:32 smithi087 ceph-mon[32276]: pgmap v683: 129 pgs: 129 active+clean; 574 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-06-06T08:09:33.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94858 11.10 MB/sec execute 335 sec latency 302.277 ms 2024-06-06T08:09:34.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94870 11.07 MB/sec execute 336 sec latency 1148.265 ms 2024-06-06T08:09:34.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:34 smithi195 ceph-mon[37040]: pgmap v684: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-06-06T08:09:34.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:34 smithi181 ceph-mon[36790]: pgmap v684: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-06-06T08:09:34.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:34 smithi087 ceph-mon[32276]: pgmap v684: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-06-06T08:09:35.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 94907 11.04 MB/sec execute 337 sec latency 2148.308 ms 2024-06-06T08:09:36.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 95106 11.03 MB/sec execute 338 sec latency 2219.073 ms 2024-06-06T08:09:36.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:36 smithi195 ceph-mon[37040]: pgmap v685: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-06-06T08:09:36.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:36 smithi181 ceph-mon[36790]: pgmap v685: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-06-06T08:09:36.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:36 smithi087 ceph-mon[32276]: pgmap v685: 129 pgs: 129 active+clean; 589 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-06-06T08:09:37.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 95330 11.02 MB/sec execute 339 sec latency 431.891 ms 2024-06-06T08:09:38.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 95705 11.07 MB/sec execute 340 sec latency 357.643 ms 2024-06-06T08:09:38.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:38 smithi195 ceph-mon[37040]: pgmap v686: 129 pgs: 129 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T08:09:38.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:38 smithi181 ceph-mon[36790]: pgmap v686: 129 pgs: 129 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T08:09:38.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:38 smithi087 ceph-mon[32276]: pgmap v686: 129 pgs: 129 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T08:09:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 -- 172.21.15.87:0/840284547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce30100bf0 msgr2=0x7fce30101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 --2- 172.21.15.87:0/840284547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce30100bf0 0x7fce30101070 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fce20009900 tx=0x7fce2002a410 comp rx=0 tx=0).stop 2024-06-06T08:09:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 -- 172.21.15.87:0/840284547 shutdown_connections 2024-06-06T08:09:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 --2- 172.21.15.87:0/840284547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce301015b0 0x7fce3010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 --2- 172.21.15.87:0/840284547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce30100bf0 0x7fce30101070 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 --2- 172.21.15.87:0/840284547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30105840 0x7fce30105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 -- 172.21.15.87:0/840284547 >> 172.21.15.87:0/840284547 conn(0x7fce300766a0 msgr2=0x7fce30076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 -- 172.21.15.87:0/840284547 shutdown_connections 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.013+0000 7fce373a5640 1 -- 172.21.15.87:0/840284547 wait complete. 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 Processor -- start 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 -- start start 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce30105840 0x7fce3019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce30110c30 con 0x7fce301015b0 2024-06-06T08:09:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce30110ab0 con 0x7fce30100bf0 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce373a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce30110db0 con 0x7fce30105840 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce34940640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce35141640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce34940640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56934/0 (socket says 172.21.15.87:56934) 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce34940640 1 -- 172.21.15.87:0/2921075757 learned_addr learned my addr 172.21.15.87:0/2921075757 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce35141640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:55062/0 (socket says 172.21.15.87:55062) 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.014+0000 7fce35942640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce30105840 0x7fce3019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 -- 172.21.15.87:0/2921075757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce30105840 msgr2=0x7fce3019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce30105840 0x7fce3019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 -- 172.21.15.87:0/2921075757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 msgr2=0x7fce30198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 -- 172.21.15.87:0/2921075757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3019d0f0 con 0x7fce301015b0 2024-06-06T08:09:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce35141640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce34940640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fce20006010 tx=0x7fce20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce267fc640 1 -- 172.21.15.87:0/2921075757 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2003f030 con 0x7fce301015b0 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce267fc640 1 -- 172.21.15.87:0/2921075757 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20004440 con 0x7fce301015b0 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3019d380 con 0x7fce301015b0 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce267fc640 1 -- 172.21.15.87:0/2921075757 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20035070 con 0x7fce301015b0 2024-06-06T08:09:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.015+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301a4c70 con 0x7fce301015b0 2024-06-06T08:09:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce267fc640 1 -- 172.21.15.87:0/2921075757 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fce20042030 con 0x7fce301015b0 2024-06-06T08:09:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 Processor -- start 2024-06-06T08:09:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 -- start start 2024-06-06T08:09:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f475512220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f475514b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce267fc640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 0x7fce1c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f4754674f0 con 0x55f475503070 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f475405870 con 0x55f475503990 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.018+0000 7fce3a18e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4754634b0 con 0x55f475512760 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce34940640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 0x7fce1c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56946/0 (socket says 172.21.15.87:56946) 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 -- 172.21.15.87:0/444620263 learned_addr learned my addr 172.21.15.87:0/444620263 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35141640 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f475512220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce267fc640 1 -- 172.21.15.87:0/2921075757 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7fce200efbb0 con 0x7fce301015b0 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f475514b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 -- 172.21.15.87:0/444620263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 msgr2=0x55f475514b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f475514b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 -- 172.21.15.87:0/444620263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 msgr2=0x55f475512220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f475512220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 -- 172.21.15.87:0/444620263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4754219b0 con 0x55f475503070 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce34940640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 0x7fce1c0a5260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce200b81c0 tx=0x7fce2003ae50 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce35942640 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fce28013b60 tx=0x7fce28009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393f2650dcdd446f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce1b7fe640 1 -- 172.21.15.87:0/444620263 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28013130 con 0x55f475503070 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce1b7fe640 1 -- 172.21.15.87:0/444620263 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce28013c50 con 0x55f475503070 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.019+0000 7fce1b7fe640 1 -- 172.21.15.87:0/444620263 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280053a0 con 0x55f475503070 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- 172.21.15.87:0/444620263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 msgr2=0x55f475503450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fce28013b60 tx=0x7fce28009b30 comp rx=0 tx=0).stop 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- 172.21.15.87:0/444620263 shutdown_connections 2024-06-06T08:09:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f475514b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f475512220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 --2- 172.21.15.87:0/444620263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f475503450 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- 172.21.15.87:0/444620263 >> 172.21.15.87:0/444620263 conn(0x55f4754ff680 msgr2=0x55f4754ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- 172.21.15.87:0/444620263 shutdown_connections 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- 172.21.15.87:0/444620263 wait complete. 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 -- start start 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.020+0000 7fce3a18e740 1 Processor -- start 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f4755100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f475515330 con 0x55f475512760 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f4753f0ae0 con 0x55f475503070 2024-06-06T08:09:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4755154b0 con 0x55f475503990 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce3a18e740 10 client.0 _ll_register_callbacks cb 0x55f4754299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f4755100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47416/0 (socket says 172.21.15.87:47416) 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 -- 172.21.15.87:0/2696565643 learned_addr learned my addr 172.21.15.87:0/2696565643 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35942640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 -- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 msgr2=0x55f47550f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 -- 172.21.15.87:0/2696565643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 msgr2=0x55f4755100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f4755100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.021+0000 7fce35141640 1 -- 172.21.15.87:0/2696565643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47517b210 con 0x55f475503990 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce35141640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fce2c012d20 tx=0x7fce2c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97debec6e20561d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:09:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c012110 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7fce2c004050 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce28013c50 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fce2c025050 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 10 client.27496 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f475247620 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce28013c50 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fce2c010430 con 0x55f475503990 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 10 client.27496 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 10 client.27496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 20 client.27496 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f4755228e0 0x55f475524cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 4 client.27496 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.022+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f47547af10 con 0x55f4755228e0 2024-06-06T08:09:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.023+0000 7fce35942640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f4755228e0 0x55f475524cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.023+0000 7fce35942640 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f4755228e0 0x55f475524cd0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3fb87283cf407c server_cookie=26dc62b20236489d in_seq=0 out_seq=0 2024-06-06T08:09:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.023+0000 7fce18ff9640 10 client.27496 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce18ff9640 1 -- 172.21.15.87:0/2696565643 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f47547af10 con 0x55f4755228e0 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce18ff9640 10 client.27496 handle_command_reply: tid=0 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 client.27496 shutdown 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 msgr2=0x55f47550fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fce2c012d20 tx=0x7fce2c00f790 comp rx=0 tx=0).stop 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 shutdown_connections 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f4755228e0 0x55f475524cd0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f4755100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47550fb60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce18ff9640 0 client.27496 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2696565643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 >> 172.21.15.87:0/2696565643 conn(0x55f4754ff680 msgr2=0x55f4755054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 shutdown_connections 2024-06-06T08:09:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 1 -- 172.21.15.87:0/2696565643 wait complete. 2024-06-06T08:09:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.029+0000 7fce3a18e740 20 client.27496 trim_cache size 0 max 16384 2024-06-06T08:09:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 Processor -- start 2024-06-06T08:09:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 -- start start 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47552a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce301af600 con 0x55f475512760 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aced0 con 0x55f475503070 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce3a18e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aad40 con 0x55f475503990 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56966/0 (socket says 172.21.15.87:56966) 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35141640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47552a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 -- 172.21.15.87:0/2501801526 learned_addr learned my addr 172.21.15.87:0/2501801526 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 -- 172.21.15.87:0/2501801526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 msgr2=0x55f47552a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47552a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 -- 172.21.15.87:0/2501801526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 msgr2=0x55f47550fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.046+0000 7fce35942640 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.047+0000 7fce35942640 1 -- 172.21.15.87:0/2501801526 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f475525040 con 0x55f475512760 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.047+0000 7fce35942640 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fce28013b60 tx=0x7fce28009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842132ef2ee58f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.047+0000 7fce1affd640 1 -- 172.21.15.87:0/2501801526 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28013c50 con 0x55f475512760 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.047+0000 7fce1affd640 1 -- 172.21.15.87:0/2501801526 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce28017020 con 0x55f475512760 2024-06-06T08:09:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.047+0000 7fce1affd640 1 -- 172.21.15.87:0/2501801526 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280053f0 con 0x55f475512760 2024-06-06T08:09:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- 172.21.15.87:0/2501801526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 msgr2=0x55f47552ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fce28013b60 tx=0x7fce28009f30 comp rx=0 tx=0).stop 2024-06-06T08:09:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- 172.21.15.87:0/2501801526 shutdown_connections 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475512760 0x55f47552ab20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475503990 0x55f47552a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- 172.21.15.87:0/2501801526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503070 0x55f47550fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- 172.21.15.87:0/2501801526 >> 172.21.15.87:0/2501801526 conn(0x55f4754ff680 msgr2=0x55f47550a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- 172.21.15.87:0/2501801526 shutdown_connections 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- 172.21.15.87:0/2501801526 wait complete. 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- start start 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 Processor -- start 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f47551ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f47550beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ab390 con 0x55f475503070 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ac800 con 0x55f475503990 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.048+0000 7fce3a18e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aba60 con 0x55f475512760 2024-06-06T08:09:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce3a18e740 10 client.0 _ll_register_callbacks cb 0x7fce2c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35141640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f47551ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56980/0 (socket says 172.21.15.87:56980) 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 -- 172.21.15.87:0/3837931173 learned_addr learned my addr 172.21.15.87:0/3837931173 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35141640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f47551ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:55084/0 (socket says 172.21.15.87:55084) 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f47550beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 -- 172.21.15.87:0/3837931173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 msgr2=0x55f47550beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f47550beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 -- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 msgr2=0x55f47551ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f47551ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 -- 172.21.15.87:0/3837931173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47550c5f0 con 0x55f475503070 2024-06-06T08:09:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce35942640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fce280052d0 tx=0x7fce28024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66cd2514b17fce8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280131f0 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7fce2801c040 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce28017020 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fce28003970 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce3a18e740 10 client.18429 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.049+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f47558d000 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce28017020 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fce28037020 con 0x55f475503070 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 10 client.18429 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 10 client.18429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 20 client.18429 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f47558d470 0x55f47558f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 4 client.18429 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f475474c20 con 0x55f47558d470 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce35141640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f47558d470 0x55f47558f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce18ff9640 10 client.18429 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.050+0000 7fce35141640 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f47558d470 0x55f47558f860 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fae9898e08fbf8f server_cookie=8b6f40653c1ea6c3 in_seq=0 out_seq=0 2024-06-06T08:09:39.186 INFO:tasks.workunit.client.0.smithi087.stdout: 10 96025 11.10 MB/sec execute 341 sec latency 319.098 ms 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce18ff9640 1 -- 172.21.15.87:0/3837931173 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f475474c20 con 0x55f47558d470 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce18ff9640 10 client.18429 handle_command_reply: tid=0 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 client.18429 shutdown 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 msgr2=0x55f47551a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fce280052d0 tx=0x7fce28024040 comp rx=0 tx=0).stop 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 shutdown_connections 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f47558d470 0x55f47558f860 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce18ff9640 0 client.18429 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f475512760 0x55f47550beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f475503990 0x55f47551ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 --2- 172.21.15.87:0/3837931173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f475503070 0x55f47551a930 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.663+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 >> 172.21.15.87:0/3837931173 conn(0x55f4754ff680 msgr2=0x55f4754fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.664+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 shutdown_connections 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.664+0000 7fce3a18e740 1 -- 172.21.15.87:0/3837931173 wait complete. 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.664+0000 7fce3a18e740 20 client.18429 trim_cache size 0 max 16384 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:09:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:09:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 msgr2=0x7fce1c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 0x7fce1c0a5260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce200b81c0 tx=0x7fce2003ae50 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 msgr2=0x7fce30198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fce20006010 tx=0x7fce20004040 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 shutdown_connections 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fce1c0a2da0 0x7fce1c0a5260 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce30105840 0x7fce3019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce301015b0 0x7fce30198640 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 --2- 172.21.15.87:0/2921075757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30100bf0 0x7fce30198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 >> 172.21.15.87:0/2921075757 conn(0x7fce300766a0 msgr2=0x7fce3006a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.665+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 shutdown_connections 2024-06-06T08:09:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:39.666+0000 7fce373a5640 1 -- 172.21.15.87:0/2921075757 wait complete. 2024-06-06T08:09:40.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 96220 11.10 MB/sec execute 342 sec latency 490.324 ms 2024-06-06T08:09:40.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:40 smithi195 ceph-mon[37040]: pgmap v687: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-06-06T08:09:40.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:40 smithi181 ceph-mon[36790]: pgmap v687: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-06-06T08:09:40.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:40 smithi087 ceph-mon[32276]: pgmap v687: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 499 op/s 2024-06-06T08:09:41.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 96478 11.11 MB/sec execute 343 sec latency 380.001 ms 2024-06-06T08:09:42.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 96817 11.11 MB/sec execute 344 sec latency 365.237 ms 2024-06-06T08:09:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:42 smithi195 ceph-mon[37040]: pgmap v688: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-06-06T08:09:42.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:42 smithi181 ceph-mon[36790]: pgmap v688: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-06-06T08:09:42.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:42 smithi087 ceph-mon[32276]: pgmap v688: 129 pgs: 129 active+clean; 603 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.6 MiB/s wr, 405 op/s 2024-06-06T08:09:43.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 97177 11.14 MB/sec execute 345 sec latency 229.203 ms 2024-06-06T08:09:44.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 97492 11.15 MB/sec execute 346 sec latency 192.086 ms 2024-06-06T08:09:44.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:44 smithi195 ceph-mon[37040]: pgmap v689: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-06-06T08:09:44.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:44 smithi181 ceph-mon[36790]: pgmap v689: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-06-06T08:09:44.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:44 smithi087 ceph-mon[32276]: pgmap v689: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-06-06T08:09:45.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 97767 11.16 MB/sec execute 347 sec latency 328.691 ms 2024-06-06T08:09:45.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:45 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:09:45.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:45 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:09:45.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:45 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:09:46.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 98144 11.20 MB/sec execute 348 sec latency 285.448 ms 2024-06-06T08:09:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:46 smithi195 ceph-mon[37040]: pgmap v690: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.5 MiB/s wr, 554 op/s 2024-06-06T08:09:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:46 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:09:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:46 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:09:46.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:46 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:09:46.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:46 smithi181 ceph-mon[36790]: pgmap v690: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.5 MiB/s wr, 554 op/s 2024-06-06T08:09:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:46 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:09:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:46 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:09:46.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:46 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:09:46.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:46 smithi087 ceph-mon[32276]: pgmap v690: 129 pgs: 129 active+clean; 577 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.5 MiB/s wr, 554 op/s 2024-06-06T08:09:46.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:46 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:09:46.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:46 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:09:46.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:46 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:09:47.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 98453 11.24 MB/sec execute 349 sec latency 205.497 ms 2024-06-06T08:09:48.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 98706 11.22 MB/sec execute 350 sec latency 605.146 ms 2024-06-06T08:09:48.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:48 smithi195 ceph-mon[37040]: pgmap v691: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.7 MiB/s wr, 563 op/s 2024-06-06T08:09:48.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:48 smithi181 ceph-mon[36790]: pgmap v691: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.7 MiB/s wr, 563 op/s 2024-06-06T08:09:48.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:48 smithi087 ceph-mon[32276]: pgmap v691: 129 pgs: 129 active+clean; 578 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.7 MiB/s wr, 563 op/s 2024-06-06T08:09:49.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 99005 11.23 MB/sec execute 351 sec latency 415.653 ms 2024-06-06T08:09:50.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 99427 11.26 MB/sec execute 352 sec latency 179.334 ms 2024-06-06T08:09:50.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:50 smithi195 ceph-mon[37040]: pgmap v692: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.4 MiB/s wr, 620 op/s 2024-06-06T08:09:50.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:50 smithi181 ceph-mon[36790]: pgmap v692: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.4 MiB/s wr, 620 op/s 2024-06-06T08:09:50.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:50 smithi087 ceph-mon[32276]: pgmap v692: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.4 MiB/s wr, 620 op/s 2024-06-06T08:09:51.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 99783 11.31 MB/sec execute 353 sec latency 215.779 ms 2024-06-06T08:09:52.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 100054 11.31 MB/sec execute 354 sec latency 295.024 ms 2024-06-06T08:09:52.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:52 smithi087 ceph-mon[32276]: pgmap v693: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-06-06T08:09:52.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:52 smithi195 ceph-mon[37040]: pgmap v693: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-06-06T08:09:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 -- 172.21.15.87:0/2618787117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c09b0c0 msgr2=0x7f5a5c0a2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/2618787117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c09b0c0 0x7f5a5c0a2c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a64067110 tx=0x7f5a64096370 comp rx=0 tx=0).stop 2024-06-06T08:09:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 -- 172.21.15.87:0/2618787117 shutdown_connections 2024-06-06T08:09:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/2618787117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c09b0c0 0x7f5a5c0a2c40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/2618787117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a5c096ba0 0x7f5a5c09a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/2618787117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c0965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.799+0000 7f5a6ad73640 1 -- 172.21.15.87:0/2618787117 >> 172.21.15.87:0/2618787117 conn(0x7f5a5c00b880 msgr2=0x7f5a5c00bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.800+0000 7f5a6ad73640 1 -- 172.21.15.87:0/2618787117 shutdown_connections 2024-06-06T08:09:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.800+0000 7f5a6ad73640 1 -- 172.21.15.87:0/2618787117 wait complete. 2024-06-06T08:09:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.801+0000 7f5a6ad73640 1 Processor -- start 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.801+0000 7f5a6ad73640 1 -- start start 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a5c09b0c0 0x7f5a5c149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c0a1d10 0x7f5a5c0a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0b7130 con 0x7f5a5c0961f0 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0b6fb0 con 0x7f5a5c09b0c0 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6ad73640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0b72b0 con 0x7f5a5c0a1d10 2024-06-06T08:09:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44960/0 (socket says 172.21.15.87:44960) 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 -- 172.21.15.87:0/1696032194 learned_addr learned my addr 172.21.15.87:0/1696032194 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a6a572640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c0a1d10 0x7f5a5c0a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 -- 172.21.15.87:0/1696032194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c0a1d10 msgr2=0x7f5a5c0a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c0a1d10 0x7f5a5c0a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 -- 172.21.15.87:0/1696032194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a5c09b0c0 msgr2=0x7f5a5c149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a5c09b0c0 0x7f5a5c149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.802+0000 7f5a69d71640 1 -- 172.21.15.87:0/1696032194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c0a2830 con 0x7f5a5c0961f0 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.803+0000 7f5a69d71640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000d950 tx=0x7f5a6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.803+0000 7f5a5affd640 1 -- 172.21.15.87:0/1696032194 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60017070 con 0x7f5a5c0961f0 2024-06-06T08:09:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.803+0000 7f5a5affd640 1 -- 172.21.15.87:0/1696032194 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60009c30 con 0x7f5a5c0961f0 2024-06-06T08:09:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.804+0000 7f5a5affd640 1 -- 172.21.15.87:0/1696032194 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000f6b0 con 0x7f5a5c0961f0 2024-06-06T08:09:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.804+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c0a2b20 con 0x7f5a5c0961f0 2024-06-06T08:09:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.804+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c13c9d0 con 0x7f5a5c0961f0 2024-06-06T08:09:52.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:52 smithi181 ceph-mon[36790]: pgmap v693: 129 pgs: 129 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-06-06T08:09:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.808+0000 7f5a5affd640 1 -- 172.21.15.87:0/1696032194 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5a6000f850 con 0x7f5a5c0961f0 2024-06-06T08:09:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.809+0000 7f5a5affd640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 0x7f5a500a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.809+0000 7f5a5affd640 1 -- 172.21.15.87:0/1696032194 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7f5a600cf430 con 0x7f5a5c0961f0 2024-06-06T08:09:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.810+0000 7f5a6edc1740 1 Processor -- start 2024-06-06T08:09:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.811+0000 7f5a69570640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 0x7f5a500a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.811+0000 7f5a6edc1740 1 -- start start 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f2467ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a69570640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 0x7f5a500a5270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09a940 tx=0x7f5a54006c40 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467bac60 0x55f2467c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f24671e4f0 con 0x55f2467c5520 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2466bc870 con 0x55f2467ba340 2024-06-06T08:09:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f24671a4b0 con 0x55f2467bac60 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6a572640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6a572640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44962/0 (socket says 172.21.15.87:44962) 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.812+0000 7f5a6a572640 1 -- 172.21.15.87:0/2419851012 learned_addr learned my addr 172.21.15.87:0/2419851012 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a69570640 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467bac60 0x55f2467c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 -- 172.21.15.87:0/2419851012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467bac60 msgr2=0x55f2467c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467bac60 0x55f2467c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 -- 172.21.15.87:0/2419851012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 msgr2=0x55f2467ba720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f2467ba720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 -- 172.21.15.87:0/2419851012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2466d89b0 con 0x55f2467c5520 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a6a572640 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5a64067650 tx=0x7f5a64067910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee792d12f157caa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a4ffff640 1 -- 172.21.15.87:0/2419851012 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640ab070 con 0x55f2467c5520 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.813+0000 7f5a4ffff640 1 -- 172.21.15.87:0/2419851012 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6406b580 con 0x55f2467c5520 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.814+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2419851012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 msgr2=0x55f2467c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.814+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5a64067650 tx=0x7f5a64067910 comp rx=0 tx=0).stop 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.815+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2419851012 shutdown_connections 2024-06-06T08:09:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.815+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467c5520 0x55f2467c7910 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.815+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467bac60 0x55f2467c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.815+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2419851012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f2467ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.815+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2419851012 >> 172.21.15.87:0/2419851012 conn(0x55f2467b6850 msgr2=0x55f2467b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.818+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2419851012 shutdown_connections 2024-06-06T08:09:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.818+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2419851012 wait complete. 2024-06-06T08:09:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.818+0000 7f5a6edc1740 1 -- start start 2024-06-06T08:09:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.818+0000 7f5a6edc1740 1 Processor -- start 2024-06-06T08:09:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f2467c80f0 con 0x55f2467bac60 2024-06-06T08:09:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2466a7ae0 con 0x55f2467c5520 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f2467c8270 con 0x55f2467ba340 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.819+0000 7f5a6edc1740 10 client.0 _ll_register_callbacks cb 0x55f2466e09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44964/0 (socket says 172.21.15.87:44964) 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 -- 172.21.15.87:0/2688987508 learned_addr learned my addr 172.21.15.87:0/2688987508 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a6a572640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 -- 172.21.15.87:0/2688987508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 msgr2=0x55f2467c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 -- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 msgr2=0x55f2467c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 -- 172.21.15.87:0/2688987508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f246432210 con 0x55f2467bac60 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a69570640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5a540103a0 tx=0x7f5a54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c042bad1b8e68ac2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.820+0000 7f5a4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:09:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54024060 con 0x55f2467bac60 2024-06-06T08:09:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7f5a540127a0 con 0x55f2467bac60 2024-06-06T08:09:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a6406b580 con 0x55f2467bac60 2024-06-06T08:09:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5a54010e20 con 0x55f2467bac60 2024-06-06T08:09:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a6edc1740 10 client.18441 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.822+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f2464fe620 con 0x55f2467bac60 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6406b580 con 0x55f2467bac60 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f5a5401e070 con 0x55f2467bac60 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 10 client.18441 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 10 client.18441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 20 client.18441 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2467d5bf0 0x55f2467d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 4 client.18441 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f246731f10 con 0x55f2467d5bf0 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2467d5bf0 0x55f2467d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.823+0000 7f5a4d7fa640 10 client.18441 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.824+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2467d5bf0 0x55f2467d8070 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce6743ca81bf1a7 server_cookie=d4a2d434c9abe0ec in_seq=0 out_seq=0 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/2688987508 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f246731f10 con 0x55f2467d5bf0 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a4d7fa640 10 client.18441 handle_command_reply: tid=0 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 client.18441 shutdown 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 msgr2=0x55f2467c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5a540103a0 tx=0x7f5a54004750 comp rx=0 tx=0).stop 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 shutdown_connections 2024-06-06T08:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f2467d5bf0 0x55f2467d8070 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c2a00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2688987508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 >> 172.21.15.87:0/2688987508 conn(0x55f2467b6850 msgr2=0x55f2467bc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.825+0000 7f5a4d7fa640 0 client.18441 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.826+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 shutdown_connections 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.826+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2688987508 wait complete. 2024-06-06T08:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.826+0000 7f5a6edc1740 20 client.18441 trim_cache size 0 max 16384 2024-06-06T08:09:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.856+0000 7f5a6edc1740 1 Processor -- start 2024-06-06T08:09:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.856+0000 7f5a6edc1740 1 -- start start 2024-06-06T08:09:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c165910 con 0x55f2467bac60 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1631e0 con 0x55f2467c5520 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c097440 con 0x55f2467ba340 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:37628/0 (socket says 172.21.15.87:37628) 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 -- 172.21.15.87:0/2874059407 learned_addr learned my addr 172.21.15.87:0/2874059407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69570640 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a6a572640 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 -- 172.21.15.87:0/2874059407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 msgr2=0x55f2467dd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467dd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 -- 172.21.15.87:0/2874059407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 msgr2=0x55f2467c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.857+0000 7f5a69d71640 1 -- 172.21.15.87:0/2874059407 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2467ba100 con 0x55f2467ba340 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.858+0000 7f5a69d71640 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a60097f20 tx=0x7f5a60005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716ea0e1664fa248 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.858+0000 7f5a4f7fe640 1 -- 172.21.15.87:0/2874059407 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6009f900 con 0x55f2467ba340 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.858+0000 7f5a4f7fe640 1 -- 172.21.15.87:0/2874059407 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a600b3070 con 0x55f2467ba340 2024-06-06T08:09:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.858+0000 7f5a4f7fe640 1 -- 172.21.15.87:0/2874059407 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60097d50 con 0x55f2467ba340 2024-06-06T08:09:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2874059407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 msgr2=0x55f2467bcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a60097f20 tx=0x7f5a60005f70 comp rx=0 tx=0).stop 2024-06-06T08:09:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2874059407 shutdown_connections 2024-06-06T08:09:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467c5520 0x55f2467dd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f2467c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/2874059407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467ba340 0x55f2467bcd00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2874059407 >> 172.21.15.87:0/2874059407 conn(0x55f2467b6850 msgr2=0x55f2467c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2874059407 shutdown_connections 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- 172.21.15.87:0/2874059407 wait complete. 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 -- start start 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.859+0000 7f5a6edc1740 1 Processor -- start 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f246838c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467c5520 0x55f2467c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1616c0 con 0x55f2467bac60 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c162b10 con 0x55f2467ba340 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c161d70 con 0x55f2467c5520 2024-06-06T08:09:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a69570640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a69570640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44990/0 (socket says 172.21.15.87:44990) 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a69570640 1 -- 172.21.15.87:0/4127984542 learned_addr learned my addr 172.21.15.87:0/4127984542 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6edc1740 10 client.0 _ll_register_callbacks cb 0x7f5a540134e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.860+0000 7f5a6a572640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467c5520 0x55f2467c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 -- 172.21.15.87:0/4127984542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467c5520 msgr2=0x55f2467c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69d71640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f246838c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467c5520 0x55f2467c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 -- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 msgr2=0x55f246838c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f246838c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 -- 172.21.15.87:0/4127984542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2467c91c0 con 0x55f2467bac60 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69d71640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f246838c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a69570640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5a540038b0 tx=0x7f5a54012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e650d158ee80cd1c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:09:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54033020 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a600b3070 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9583+0+0 (secure 0 0 0) 0x7f5a5402f680 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5a54003690 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a6edc1740 10 client.18447 fetch_fsmap learned FSMap version 55 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f2468403f0 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.861+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a600b3070 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f5a5401e050 con 0x55f2467bac60 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 10 client.18447 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 10 client.18447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 20 client.18447 populate_metadata read hostname 'smithi087' 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f246840bf0 0x55f246843010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:09:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 4 client.18447 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:09:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.862+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f24672bc20 con 0x55f246840bf0 2024-06-06T08:09:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.863+0000 7f5a69d71640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f246840bf0 0x55f246843010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:09:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.864+0000 7f5a4d7fa640 10 client.18447 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:52.864+0000 7f5a69d71640 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f246840bf0 0x55f246843010 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3914f4003921201 server_cookie=58c8535eac931821 in_seq=0 out_seq=0 2024-06-06T08:09:53.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 100256 11.31 MB/sec execute 355 sec latency 313.629 ms 2024-06-06T08:09:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a4d7fa640 1 -- 172.21.15.87:0/4127984542 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f24672bc20 con 0x55f246840bf0 2024-06-06T08:09:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a4d7fa640 10 client.18447 handle_command_reply: tid=0 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 client.18447 shutdown 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 msgr2=0x55f246839190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5a540038b0 tx=0x7f5a54012c60 comp rx=0 tx=0).stop 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 shutdown_connections 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f246840bf0 0x55f246843010 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f2467c5520 0x55f2467c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a4d7fa640 0 client.18447 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f2467bac60 0x55f246839190 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 --2- 172.21.15.87:0/4127984542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2467ba340 0x55f246838c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 >> 172.21.15.87:0/4127984542 conn(0x55f2467b6850 msgr2=0x55f2467c2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 shutdown_connections 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 1 -- 172.21.15.87:0/4127984542 wait complete. 2024-06-06T08:09:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.782+0000 7f5a6edc1740 20 client.18447 trim_cache size 0 max 16384 2024-06-06T08:09:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:09:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:09:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 msgr2=0x7f5a500a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 0x7f5a500a5270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09a940 tx=0x7f5a54006c40 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 msgr2=0x7f5a5c1497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000d950 tx=0x7f5a6000de10 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 shutdown_connections 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a500a2db0 0x7f5a500a5270 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5c0a1d10 0x7f5a5c0a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a5c09b0c0 0x7f5a5c149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 --2- 172.21.15.87:0/1696032194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a5c0961f0 0x7f5a5c1497f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 >> 172.21.15.87:0/1696032194 conn(0x7f5a5c00b880 msgr2=0x7f5a5c09c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 shutdown_connections 2024-06-06T08:09:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:09:53.783+0000 7f5a6ad73640 1 -- 172.21.15.87:0/1696032194 wait complete. 2024-06-06T08:09:54.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 100419 11.30 MB/sec execute 356 sec latency 778.718 ms 2024-06-06T08:09:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:54 smithi087 ceph-mon[32276]: pgmap v694: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-06-06T08:09:54.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:54 smithi195 ceph-mon[37040]: pgmap v694: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-06-06T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:54 smithi181 ceph-mon[36790]: pgmap v694: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-06-06T08:09:55.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 100743 11.31 MB/sec execute 357 sec latency 243.181 ms 2024-06-06T08:09:56.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 101111 11.33 MB/sec execute 358 sec latency 191.948 ms 2024-06-06T08:09:56.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:56 smithi195 ceph-mon[37040]: pgmap v695: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-06-06T08:09:56.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:56 smithi181 ceph-mon[36790]: pgmap v695: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-06-06T08:09:56.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:56 smithi087 ceph-mon[32276]: pgmap v695: 129 pgs: 129 active+clean; 606 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-06-06T08:09:57.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 101459 11.36 MB/sec execute 359 sec latency 297.934 ms 2024-06-06T08:09:58.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 101782 11.37 MB/sec execute 360 sec latency 268.855 ms 2024-06-06T08:09:58.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:09:58 smithi195 ceph-mon[37040]: pgmap v696: 129 pgs: 129 active+clean; 600 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-06-06T08:09:58.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:09:58 smithi181 ceph-mon[36790]: pgmap v696: 129 pgs: 129 active+clean; 600 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-06-06T08:09:58.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:09:58 smithi087 ceph-mon[32276]: pgmap v696: 129 pgs: 129 active+clean; 600 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-06-06T08:09:59.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 102180 11.39 MB/sec execute 361 sec latency 258.110 ms 2024-06-06T08:10:00.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 102543 11.41 MB/sec execute 362 sec latency 309.035 ms 2024-06-06T08:10:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:00 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T08:09:59.999+0000 7f61096e0700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:10:00.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:00 smithi195 ceph-mon[37040]: pgmap v697: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-06T08:10:00.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:00 smithi195 ceph-mon[37040]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:10:00.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:00 smithi181 ceph-mon[36790]: pgmap v697: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-06T08:10:00.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:00 smithi181 ceph-mon[36790]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:10:00.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:00 smithi087 ceph-mon[32276]: pgmap v697: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-06T08:10:00.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:00 smithi087 ceph-mon[32276]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T08:10:01.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 102782 11.42 MB/sec execute 363 sec latency 376.029 ms 2024-06-06T08:10:02.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 102782 11.39 MB/sec execute 364 sec latency 1376.077 ms 2024-06-06T08:10:02.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:02 smithi195 ceph-mon[37040]: pgmap v698: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-06-06T08:10:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:02 smithi181 ceph-mon[36790]: pgmap v698: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-06-06T08:10:02.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:02 smithi087 ceph-mon[32276]: pgmap v698: 129 pgs: 129 active+clean; 599 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-06-06T08:10:03.187 INFO:tasks.workunit.client.0.smithi087.stdout: 10 102862 11.36 MB/sec execute 365 sec latency 2167.790 ms 2024-06-06T08:10:04.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103126 11.36 MB/sec execute 366 sec latency 280.693 ms 2024-06-06T08:10:04.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:04 smithi195 ceph-mon[37040]: pgmap v699: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-06-06T08:10:04.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:04 smithi181 ceph-mon[36790]: pgmap v699: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-06-06T08:10:04.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:04 smithi087 ceph-mon[32276]: pgmap v699: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-06-06T08:10:05.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103336 11.36 MB/sec execute 367 sec latency 382.588 ms 2024-06-06T08:10:06.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103402 11.34 MB/sec execute 368 sec latency 441.420 ms 2024-06-06T08:10:06.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:06 smithi195 ceph-mon[37040]: pgmap v700: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 388 op/s 2024-06-06T08:10:06.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:06 smithi181 ceph-mon[36790]: pgmap v700: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 388 op/s 2024-06-06T08:10:06.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:06 smithi087 ceph-mon[32276]: pgmap v700: 129 pgs: 129 active+clean; 609 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 388 op/s 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 -- 172.21.15.87:0/1093910596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 msgr2=0x7f49f811d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/1093910596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f811d570 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009cd0 tx=0x7f49f002a9a0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 -- 172.21.15.87:0/1093910596 shutdown_connections 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/1093910596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f811d570 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/1093910596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110570 0x7f49f8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/1093910596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f8110190 0x7f49f81089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.925+0000 7f49fcf7e640 1 -- 172.21.15.87:0/1093910596 >> 172.21.15.87:0/1093910596 conn(0x7f49f806ca70 msgr2=0x7f49f806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.926+0000 7f49fcf7e640 1 -- 172.21.15.87:0/1093910596 shutdown_connections 2024-06-06T08:10:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.926+0000 7f49fcf7e640 1 -- 172.21.15.87:0/1093910596 wait complete. 2024-06-06T08:10:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.927+0000 7f49fcf7e640 1 Processor -- start 2024-06-06T08:10:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 -- start start 2024-06-06T08:10:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 0x7f49f81a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f81a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f81ab940 0x7f49f81abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8120c00 con 0x7f49f811b180 2024-06-06T08:10:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8120a80 con 0x7f49f81ab940 2024-06-06T08:10:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.928+0000 7f49fcf7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8120d80 con 0x7f49f8110190 2024-06-06T08:10:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.931+0000 7f49f659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 0x7f49f81a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.931+0000 7f49f659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 0x7f49f81a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:33908/0 (socket says 172.21.15.87:33908) 2024-06-06T08:10:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.931+0000 7f49f659c640 1 -- 172.21.15.87:0/3837741409 learned_addr learned my addr 172.21.15.87:0/3837741409 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.931+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f81a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.932+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f81ab940 0x7f49f81abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.935+0000 7f49f5d9b640 1 -- 172.21.15.87:0/3837741409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 msgr2=0x7f49f81a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.935+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 0x7f49f81a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.935+0000 7f49f5d9b640 1 -- 172.21.15.87:0/3837741409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f81ab940 msgr2=0x7f49f81abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.935+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f81ab940 0x7f49f81abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.935+0000 7f49f5d9b640 1 -- 172.21.15.87:0/3837741409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81ac430 con 0x7f49f811b180 2024-06-06T08:10:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.941+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f81a75c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fd0 tx=0x7f49ec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.945+0000 7f49e77fe640 1 -- 172.21.15.87:0/3837741409 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec015020 con 0x7f49f811b180 2024-06-06T08:10:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.945+0000 7f49e77fe640 1 -- 172.21.15.87:0/3837741409 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec011d10 con 0x7f49f811b180 2024-06-06T08:10:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.945+0000 7f49e77fe640 1 -- 172.21.15.87:0/3837741409 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f2e0 con 0x7f49f811b180 2024-06-06T08:10:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.946+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81ac720 con 0x7f49f811b180 2024-06-06T08:10:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.946+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f8110c30 con 0x7f49f811b180 2024-06-06T08:10:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.950+0000 7f49ffd6d740 1 Processor -- start 2024-06-06T08:10:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.951+0000 7f49ffd6d740 1 -- start start 2024-06-06T08:10:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8c6e4f0 con 0x55f3e8d15350 2024-06-06T08:10:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8c0c870 con 0x55f3e8d0aa90 2024-06-06T08:10:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.954+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8c6a4b0 con 0x55f3e8d0a170 2024-06-06T08:10:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.955+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.955+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:33912/0 (socket says 172.21.15.87:33912) 2024-06-06T08:10:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.955+0000 7f49f6d9d640 1 -- 172.21.15.87:0/3563759681 learned_addr learned my addr 172.21.15.87:0/3563759681 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.956+0000 7f49f659c640 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f659c640 1 -- 172.21.15.87:0/3563759681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 msgr2=0x55f3e8d0a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f659c640 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f659c640 1 -- 172.21.15.87:0/3563759681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 msgr2=0x55f3e8d17740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:10:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f659c640 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d17740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f659c640 1 -- 172.21.15.87:0/3563759681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e8c289b0 con 0x55f3e8d0aa90 2024-06-06T08:10:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.964+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:10:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.969+0000 7f49f659c640 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d14e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49e80058c0 tx=0x7f49e80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab43afa72489f8c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.975+0000 7f49e77fe640 1 -- 172.21.15.87:0/3837741409 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f49ec018020 con 0x7f49f811b180 2024-06-06T08:10:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.975+0000 7f49e77fe640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 0x7f49e00a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.976+0000 7f49e77fe640 1 -- 172.21.15.87:0/3837741409 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9671+0+0 (secure 0 0 0) 0x7f49ec0cede0 con 0x7f49f811b180 2024-06-06T08:10:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.978+0000 7f49e4ff9640 1 -- 172.21.15.87:0/3563759681 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8004230 con 0x55f3e8d0aa90 2024-06-06T08:10:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.978+0000 7f49e4ff9640 1 -- 172.21.15.87:0/3563759681 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e800be90 con 0x55f3e8d0aa90 2024-06-06T08:10:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.979+0000 7f49ffd6d740 1 -- 172.21.15.87:0/3563759681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 msgr2=0x55f3e8d14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.979+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d14e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49e80058c0 tx=0x7f49e80107a0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.980+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 0x7f49e00a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.986+0000 7f49ffd6d740 1 -- 172.21.15.87:0/3563759681 shutdown_connections 2024-06-06T08:10:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.986+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.986+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d14e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.986+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/3563759681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0a170 0x55f3e8d0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.986+0000 7f49ffd6d740 1 -- 172.21.15.87:0/3563759681 >> 172.21.15.87:0/3563759681 conn(0x55f3e8d06680 msgr2=0x55f3e8d06a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.991+0000 7f49ffd6d740 1 -- 172.21.15.87:0/3563759681 shutdown_connections 2024-06-06T08:10:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.992+0000 7f49ffd6d740 1 -- 172.21.15.87:0/3563759681 wait complete. 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.992+0000 7f49ffd6d740 1 -- start start 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.992+0000 7f49ffd6d740 1 Processor -- start 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.992+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 0x7f49e00a5490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49f810dc40 tx=0x7f49f002f540 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d12cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8d17f80 con 0x55f3e8d15350 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8bf7ae0 con 0x55f3e8d0a170 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8d18100 con 0x55f3e8d0aa90 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.993+0000 7f49ffd6d740 10 client.0 _ll_register_callbacks cb 0x55f3e8c309b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.994+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.994+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:55326/0 (socket says 172.21.15.87:55326) 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.994+0000 7f49f5d9b640 1 -- 172.21.15.87:0/1451731794 learned_addr learned my addr 172.21.15.87:0/1451731794 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.994+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d12cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.995+0000 7f49f6d9d640 1 -- 172.21.15.87:0/1451731794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 msgr2=0x55f3e8d12780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.995+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.995+0000 7f49f6d9d640 1 -- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0a170 msgr2=0x55f3e8d12240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.995+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.995+0000 7f49f6d9d640 1 -- 172.21.15.87:0/1451731794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e8982210 con 0x55f3e8d15350 2024-06-06T08:10:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.996+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d12cc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f49f0035000 tx=0x7f49f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee450b5c34febbb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.996+0000 7f49ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.996+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.997+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0055060 con 0x55f3e8d15350 2024-06-06T08:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.997+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e800be90 con 0x55f3e8d15350 2024-06-06T08:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.997+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9671+0+0 (secure 0 0 0) 0x7f49f0030d40 con 0x55f3e8d15350 2024-06-06T08:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:06.998+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f49f0034040 con 0x55f3e8d15350 2024-06-06T08:10:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.006+0000 7f49ffd6d740 10 client.18456 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.006+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f3e8a4e620 con 0x55f3e8d15350 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.007+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e800be90 con 0x55f3e8d15350 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f49f00396d0 con 0x55f3e8d15350 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 10 client.18456 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 20 client.18456 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d25940 0x55f3e8d27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 4 client.18456 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.008+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e8c81f10 con 0x55f3e8d25940 2024-06-06T08:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.010+0000 7f49f659c640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d25940 0x55f3e8d27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.018+0000 7f49f659c640 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d25940 0x55f3e8d27dc0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99707a69e92a0993 server_cookie=17407e0c39252e9e in_seq=0 out_seq=0 2024-06-06T08:10:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.019+0000 7f49ddffb640 10 client.18456 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.021+0000 7f49ddffb640 1 -- 172.21.15.87:0/1451731794 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f3e8c81f10 con 0x55f3e8d25940 2024-06-06T08:10:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.021+0000 7f49ddffb640 10 client.18456 handle_command_reply: tid=0 2024-06-06T08:10:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.021+0000 7f49ffd6d740 1 client.18456 shutdown 2024-06-06T08:10:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.024+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 msgr2=0x55f3e8d12cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.024+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d12cc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f49f0035000 tx=0x7f49f00095c0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 shutdown_connections 2024-06-06T08:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d25940 0x55f3e8d27dc0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d15350 0x55f3e8d12cc0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/1451731794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 >> 172.21.15.87:0/1451731794 conn(0x55f3e8d06680 msgr2=0x55f3e8d0c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.027+0000 7f49ddffb640 0 client.18456 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.035+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 shutdown_connections 2024-06-06T08:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.039+0000 7f49ffd6d740 1 -- 172.21.15.87:0/1451731794 wait complete. 2024-06-06T08:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.041+0000 7f49ffd6d740 20 client.18456 trim_cache size 0 max 16384 2024-06-06T08:10:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.070+0000 7f49ffd6d740 1 Processor -- start 2024-06-06T08:10:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.072+0000 7f49ffd6d740 1 -- start start 2024-06-06T08:10:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.072+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.073+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.073+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d2dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.073+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8012c40 con 0x55f3e8d0a170 2024-06-06T08:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.073+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8010510 con 0x55f3e8d0aa90 2024-06-06T08:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.073+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49d800e380 con 0x55f3e8d15350 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.074+0000 7f49f659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.074+0000 7f49f659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59652/0 (socket says 172.21.15.87:59652) 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.074+0000 7f49f659c640 1 -- 172.21.15.87:0/2923334669 learned_addr learned my addr 172.21.15.87:0/2923334669 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.075+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d2dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f659c640 1 -- 172.21.15.87:0/2923334669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 msgr2=0x55f3e8d2dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f659c640 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d2dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f659c640 1 -- 172.21.15.87:0/2923334669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 msgr2=0x55f3e8d2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f659c640 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d2d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.076+0000 7f49f659c640 1 -- 172.21.15.87:0/2923334669 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e8d0a880 con 0x55f3e8d0a170 2024-06-06T08:10:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.082+0000 7f49f659c640 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f49e80058c0 tx=0x7f49e8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24ab6d6252c161d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.090+0000 7f49dffff640 1 -- 172.21.15.87:0/2923334669 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8011930 con 0x55f3e8d0a170 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.091+0000 7f49dffff640 1 -- 172.21.15.87:0/2923334669 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8005040 con 0x55f3e8d0a170 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.091+0000 7f49dffff640 1 -- 172.21.15.87:0/2923334669 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80038e0 con 0x55f3e8d0a170 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2923334669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 msgr2=0x55f3e8d12ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f49e80058c0 tx=0x7f49e8013040 comp rx=0 tx=0).stop 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2923334669 shutdown_connections 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d2dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2923334669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d0a170 0x55f3e8d12ae0 secure :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f49e80058c0 tx=0x7f49e8013040 comp rx=0 tx=0).stop 2024-06-06T08:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.092+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2923334669 >> 172.21.15.87:0/2923334669 conn(0x55f3e8d06680 msgr2=0x55f3e8d06a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.097+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2923334669 shutdown_connections 2024-06-06T08:10:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.100+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2923334669 wait complete. 2024-06-06T08:10:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.100+0000 7f49ffd6d740 1 -- start start 2024-06-06T08:10:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 Processor -- start 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d18c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d19160 0x55f3e8d195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49d800e9d0 con 0x55f3e8d19160 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49d800fe40 con 0x55f3e8d0aa90 2024-06-06T08:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49d800f0a0 con 0x55f3e8d15350 2024-06-06T08:10:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.101+0000 7f49ffd6d740 10 client.0 _ll_register_callbacks cb 0x7f49f00098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.104+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d18c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.104+0000 7f49f659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.104+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d18c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:55338/0 (socket says 172.21.15.87:55338) 2024-06-06T08:10:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.104+0000 7f49f5d9b640 1 -- 172.21.15.87:0/2404271116 learned_addr learned my addr 172.21.15.87:0/2404271116 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.104+0000 7f49f6d9d640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d19160 0x55f3e8d195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.105+0000 7f49f659c640 1 -- 172.21.15.87:0/2404271116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 msgr2=0x55f3e8d18c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.105+0000 7f49f659c640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d18c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.105+0000 7f49f659c640 1 -- 172.21.15.87:0/2404271116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d19160 msgr2=0x55f3e8d195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.105+0000 7f49f659c640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d19160 0x55f3e8d195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.105+0000 7f49f659c640 1 -- 172.21.15.87:0/2404271116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e8d8f6a0 con 0x55f3e8d0aa90 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.106+0000 7f49f659c640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d1d950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49e8023000 tx=0x7f49e800b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66c73cc2a9b0c964 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.106+0000 7f49ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.106+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8034070 con 0x55f3e8d0aa90 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.106+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e8005040 con 0x55f3e8d0aa90 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.106+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9671+0+0 (secure 0 0 0) 0x7f49e8005400 con 0x55f3e8d0aa90 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.107+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f49e803f050 con 0x55f3e8d0aa90 2024-06-06T08:10:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.108+0000 7f49ffd6d740 10 client.27344 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.108+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f3e8d90870 con 0x55f3e8d0aa90 2024-06-06T08:10:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.110+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8011e10 con 0x55f3e8d0aa90 2024-06-06T08:10:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.110+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f49e80273d0 con 0x55f3e8d0aa90 2024-06-06T08:10:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 10 client.27344 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 10 client.27344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 20 client.27344 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d90bb0 0x55f3e8d92fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 4 client.27344 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:10:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f49e8005040 con 0x55f3e8d90bb0 2024-06-06T08:10:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.111+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d90bb0 0x55f3e8d92fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.112+0000 7f49f5d9b640 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d90bb0 0x55f3e8d92fa0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7231040e176bed5 server_cookie=d738512287cf88b9 in_seq=0 out_seq=0 2024-06-06T08:10:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:07.114+0000 7f49ddffb640 10 client.27344 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:07.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103435 11.32 MB/sec execute 369 sec latency 833.154 ms 2024-06-06T08:10:07.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:07 smithi195 ceph-mon[37040]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T08:10:07.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:07 smithi181 ceph-mon[36790]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T08:10:07.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:07 smithi087 ceph-mon[32276]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T08:10:08.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103447 11.29 MB/sec execute 370 sec latency 1810.944 ms 2024-06-06T08:10:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.404+0000 7f49ddffb640 1 -- 172.21.15.87:0/2404271116 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f49e8005040 con 0x55f3e8d90bb0 2024-06-06T08:10:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.404+0000 7f49ddffb640 10 client.27344 handle_command_reply: tid=0 2024-06-06T08:10:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.404+0000 7f49ffd6d740 1 client.27344 shutdown 2024-06-06T08:10:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.404+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 msgr2=0x55f3e8d1d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.404+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d1d950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49e8023000 tx=0x7f49e800b510 comp rx=0 tx=0).stop 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 shutdown_connections 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55f3e8d90bb0 0x55f3e8d92fa0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f3e8d19160 0x55f3e8d195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f3e8d15350 0x55f3e8d18c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 --2- 172.21.15.87:0/2404271116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3e8d0aa90 0x55f3e8d1d950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 >> 172.21.15.87:0/2404271116 conn(0x55f3e8d06680 msgr2=0x55f3e8d0bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ddffb640 0 client.27344 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 shutdown_connections 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 1 -- 172.21.15.87:0/2404271116 wait complete. 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.405+0000 7f49ffd6d740 20 client.27344 trim_cache size 0 max 16384 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:10:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:10:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 msgr2=0x7f49e00a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 0x7f49e00a5490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49f810dc40 tx=0x7f49f002f540 comp rx=0 tx=0).stop 2024-06-06T08:10:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 msgr2=0x7f49f81a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f81a75c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fd0 tx=0x7f49ec00b870 comp rx=0 tx=0).stop 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 shutdown_connections 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f49e00a2fd0 0x7f49e00a5490 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f81ab940 0x7f49f81abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49f811b180 0x7f49f81a75c0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 --2- 172.21.15.87:0/3837741409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8110190 0x7f49f81a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 >> 172.21.15.87:0/3837741409 conn(0x7f49f806ca70 msgr2=0x7f49f810de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.406+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 shutdown_connections 2024-06-06T08:10:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:08.407+0000 7f49fcf7e640 1 -- 172.21.15.87:0/3837741409 wait complete. 2024-06-06T08:10:08.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:08 smithi195 ceph-mon[37040]: pgmap v702: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 613 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-06T08:10:08.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:08 smithi181 ceph-mon[36790]: pgmap v702: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 613 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-06T08:10:08.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:08 smithi087 ceph-mon[32276]: pgmap v702: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 613 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.3 MiB/s wr, 424 op/s 2024-06-06T08:10:09.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103487 11.27 MB/sec execute 371 sec latency 2042.578 ms 2024-06-06T08:10:10.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103611 11.26 MB/sec execute 372 sec latency 1870.261 ms 2024-06-06T08:10:10.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:10 smithi195 ceph-mon[37040]: pgmap v703: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:10.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:10 smithi181 ceph-mon[36790]: pgmap v703: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:10.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:10 smithi087 ceph-mon[32276]: pgmap v703: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:11.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103720 11.25 MB/sec execute 373 sec latency 856.958 ms 2024-06-06T08:10:12.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 103912 11.25 MB/sec execute 374 sec latency 772.180 ms 2024-06-06T08:10:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:12 smithi195 ceph-mon[37040]: pgmap v704: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:12.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:12 smithi181 ceph-mon[36790]: pgmap v704: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:12.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:12 smithi087 ceph-mon[32276]: pgmap v704: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 616 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-06-06T08:10:13.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 104082 11.24 MB/sec execute 375 sec latency 753.727 ms 2024-06-06T08:10:14.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 104309 11.26 MB/sec execute 376 sec latency 658.462 ms 2024-06-06T08:10:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:14 smithi195 ceph-mon[37040]: pgmap v705: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 458 op/s 2024-06-06T08:10:14.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:14 smithi181 ceph-mon[36790]: pgmap v705: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 458 op/s 2024-06-06T08:10:14.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:14 smithi087 ceph-mon[32276]: pgmap v705: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 603 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 458 op/s 2024-06-06T08:10:15.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 104443 11.24 MB/sec execute 377 sec latency 531.372 ms 2024-06-06T08:10:16.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 104627 11.24 MB/sec execute 378 sec latency 372.379 ms 2024-06-06T08:10:16.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:16 smithi195 ceph-mon[37040]: pgmap v706: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 599 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 518 op/s 2024-06-06T08:10:16.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:16 smithi181 ceph-mon[36790]: pgmap v706: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 599 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 518 op/s 2024-06-06T08:10:16.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:16 smithi087 ceph-mon[32276]: pgmap v706: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 599 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 518 op/s 2024-06-06T08:10:17.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 104856 11.23 MB/sec execute 379 sec latency 216.384 ms 2024-06-06T08:10:18.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 105161 11.25 MB/sec execute 380 sec latency 192.358 ms 2024-06-06T08:10:18.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:18 smithi195 ceph-mon[37040]: pgmap v707: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 601 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-06-06T08:10:18.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:18 smithi181 ceph-mon[36790]: pgmap v707: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 601 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-06-06T08:10:18.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:18 smithi087 ceph-mon[32276]: pgmap v707: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 601 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-06-06T08:10:19.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 105509 11.25 MB/sec execute 381 sec latency 202.754 ms 2024-06-06T08:10:20.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 105758 11.26 MB/sec execute 382 sec latency 452.277 ms 2024-06-06T08:10:20.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:20 smithi195 ceph-mon[37040]: pgmap v708: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-06T08:10:20.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:20 smithi195 ceph-mon[37040]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T08:10:20.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:20 smithi181 ceph-mon[36790]: pgmap v708: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-06T08:10:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:20 smithi181 ceph-mon[36790]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T08:10:20.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:20 smithi087 ceph-mon[32276]: pgmap v708: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-06T08:10:20.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:20 smithi087 ceph-mon[32276]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T08:10:21.188 INFO:tasks.workunit.client.0.smithi087.stdout: 10 106107 11.30 MB/sec execute 383 sec latency 285.789 ms 2024-06-06T08:10:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3470948792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8106890 msgr2=0x7fcdc8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3470948792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8106890 0x7fcdc8106c70 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009930 tx=0x7fcdbc02a420 comp rx=0 tx=0).stop 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3470948792 shutdown_connections 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3470948792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8101290 0x7fcdc810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3470948792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3470948792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8106890 0x7fcdc8106c70 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3470948792 >> 172.21.15.87:0/3470948792 conn(0x7fcdc8077a20 msgr2=0x7fcdc8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3470948792 shutdown_connections 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.504+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3470948792 wait complete. 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 Processor -- start 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 -- start start 2024-06-06T08:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8106890 0x7fcdc81a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8110c60 con 0x7fcdc8101290 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8110ae0 con 0x7fcdc8106890 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcef3b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8110de0 con 0x7fcdc81008d0 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52300/0 (socket says 172.21.15.87:52300) 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdc7fff640 1 -- 172.21.15.87:0/3552377413 learned_addr learned my addr 172.21.15.87:0/3552377413 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8106890 0x7fcdc81a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdc7fff640 1 -- 172.21.15.87:0/3552377413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 msgr2=0x7fcdc8199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.505+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc7fff640 1 -- 172.21.15.87:0/3552377413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8106890 msgr2=0x7fcdc81a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8106890 0x7fcdc81a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc7fff640 1 -- 172.21.15.87:0/3552377413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc81a45c0 con 0x7fcdc8101290 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e9f0 tx=0x7fcdb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc5ffb640 1 -- 172.21.15.87:0/3552377413 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400cd80 con 0x7fcdc8101290 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc5ffb640 1 -- 172.21.15.87:0/3552377413 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb4010910 con 0x7fcdc8101290 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc81a4790 con 0x7fcdc8101290 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdc5ffb640 1 -- 172.21.15.87:0/3552377413 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4018990 con 0x7fcdc8101290 2024-06-06T08:10:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.506+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a4cb0 con 0x7fcdc8101290 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.509+0000 7fcdc5ffb640 1 -- 172.21.15.87:0/3552377413 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcdb4010ac0 con 0x7fcdc8101290 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.509+0000 7fcdd1c4a740 1 Processor -- start 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.509+0000 7fcdd1c4a740 1 -- start start 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.509+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.509+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5574e00ff4f0 con 0x5574e019b990 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574e009d870 con 0x5574e01aa760 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574e00fb4b0 con 0x5574e019b070 2024-06-06T08:10:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:34786/0 (socket says 172.21.15.87:34786) 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 -- 172.21.15.87:0/803685251 learned_addr learned my addr 172.21.15.87:0/803685251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdc5ffb640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 0x7fcdb80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 -- 172.21.15.87:0/803685251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 msgr2=0x5574e01acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 -- 172.21.15.87:0/803685251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 msgr2=0x5574e01aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 -- 172.21.15.87:0/803685251 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e00b99b0 con 0x5574e019b070 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 0x7fcdb80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdc5ffb640 1 -- 172.21.15.87:0/3552377413 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fcdb40cfc10 con 0x7fcdc8101290 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0011590 tx=0x7fcdc0011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47c3fb9988679715 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdb2ffd640 1 -- 172.21.15.87:0/803685251 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0017050 con 0x5574e019b070 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 0x7fcdb80a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0022000 tx=0x7fcdc0023000 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdb2ffd640 1 -- 172.21.15.87:0/803685251 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc001bb90 con 0x5574e019b070 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.510+0000 7fcdb2ffd640 1 -- 172.21.15.87:0/803685251 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000e3b0 con 0x5574e019b070 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/803685251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 msgr2=0x5574e019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0011590 tx=0x7fcdc0011a50 comp rx=0 tx=0).stop 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/803685251 shutdown_connections 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/803685251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e019b450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/803685251 >> 172.21.15.87:0/803685251 conn(0x5574e0197680 msgr2=0x5574e0197a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/803685251 shutdown_connections 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/803685251 wait complete. 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.511+0000 7fcdd1c4a740 1 -- start start 2024-06-06T08:10:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 Processor -- start 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e01a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5574e01ad390 con 0x5574e019b990 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574e0088ae0 con 0x5574e01aa760 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574e01ad510 con 0x5574e019b070 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdd1c4a740 10 client.0 _ll_register_callbacks cb 0x5574e00c19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e01a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52308/0 (socket says 172.21.15.87:52308) 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 -- 172.21.15.87:0/1280981381 learned_addr learned my addr 172.21.15.87:0/1280981381 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 -- 172.21.15.87:0/1280981381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 msgr2=0x5574e01a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e01a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 -- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 msgr2=0x5574e01a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 -- 172.21.15.87:0/1280981381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574dfe13210 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.512+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40986e0 tx=0x7fcdb409b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e268e94f21ba9836 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4018d30 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fcdb409b160 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc001bb90 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcdb40a00c0 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 10 client.18474 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5574dfedf620 con 0x5574e019b990 2024-06-06T08:10:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc001bb90 con 0x5574e019b990 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcdb40a4c60 con 0x5574e019b990 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 10 client.18474 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 20 client.18474 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e01baa10 0x5574e01bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 4 client.18474 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574e0112f10 con 0x5574e01baa10 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.513+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e01baa10 0x5574e01bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.514+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e01baa10 0x5574e01bce00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a04ffb3547342c server_cookie=3b1ee67d66e198b8 in_seq=0 out_seq=0 2024-06-06T08:10:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.514+0000 7fcda7fff640 10 client.18474 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcda7fff640 1 -- 172.21.15.87:0/1280981381 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5574e0112f10 con 0x5574e01baa10 2024-06-06T08:10:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcda7fff640 10 client.18474 handle_command_reply: tid=0 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 client.18474 shutdown 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 msgr2=0x5574e01a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40986e0 tx=0x7fcdb409b910 comp rx=0 tx=0).stop 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 shutdown_connections 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e01baa10 0x5574e01bce00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b990 0x5574e01a7c70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcda7fff640 0 client.18474 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1280981381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b070 0x5574e01a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 >> 172.21.15.87:0/1280981381 conn(0x5574e0197680 msgr2=0x5574e019d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 shutdown_connections 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1280981381 wait complete. 2024-06-06T08:10:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.518+0000 7fcdd1c4a740 20 client.18474 trim_cache size 0 max 16384 2024-06-06T08:10:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 Processor -- start 2024-06-06T08:10:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 -- start start 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81af6d0 con 0x5574e019b070 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81acfa0 con 0x5574e01aa760 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81aae10 con 0x5574e019b990 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52316/0 (socket says 172.21.15.87:52316) 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1251363820 learned_addr learned my addr 172.21.15.87:0/1251363820 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1251363820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 msgr2=0x5574e01c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1251363820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 msgr2=0x5574e01c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.534+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1251363820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e01a5c80 con 0x5574e019b070 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0096c0 tx=0x7fcdbc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaa9e87a27541a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdb27fc640 1 -- 172.21.15.87:0/1251363820 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc04a070 con 0x5574e019b070 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdb27fc640 1 -- 172.21.15.87:0/1251363820 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc005520 con 0x5574e019b070 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdb27fc640 1 -- 172.21.15.87:0/1251363820 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc035040 con 0x5574e019b070 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1251363820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 msgr2=0x5574e0196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.535+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0096c0 tx=0x7fcdbc005b50 comp rx=0 tx=0).stop 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1251363820 shutdown_connections 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1251363820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e0196660 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1251363820 >> 172.21.15.87:0/1251363820 conn(0x5574e0197680 msgr2=0x5574e0195f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1251363820 shutdown_connections 2024-06-06T08:10:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1251363820 wait complete. 2024-06-06T08:10:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 -- start start 2024-06-06T08:10:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 Processor -- start 2024-06-06T08:10:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.536+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81ab460 con 0x5574e019b070 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81ac8d0 con 0x5574e01aa760 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81abb30 con 0x5574e019b990 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 10 client.0 _ll_register_callbacks cb 0x7fcdb400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52328/0 (socket says 172.21.15.87:52328) 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1227154572 learned_addr learned my addr 172.21.15.87:0/1227154572 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcd4d8640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1227154572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 msgr2=0x5574e01aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 msgr2=0x5574e01af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 -- 172.21.15.87:0/1227154572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574e02246a0 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdcccd7640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009230 tx=0x7fcdbc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0b911210983c8c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcda7fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc04a040 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdbc005520 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fcdbc004240 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc035040 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 10 client.18480 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.537+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5574e0225990 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc005520 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcdbc04d070 con 0x5574e019b070 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 10 client.18480 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 10 client.18480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 20 client.18480 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e0225ff0 0x5574e02283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 4 client.18480 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574e010cc20 con 0x5574e0225ff0 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e0225ff0 0x5574e02283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcda7fff640 10 client.18480 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:21.538+0000 7fcdc7fff640 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e0225ff0 0x5574e02283e0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4db825b3b36e24 server_cookie=127e21cbe3c3493 in_seq=0 out_seq=0 2024-06-06T08:10:22.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 106268 11.30 MB/sec execute 384 sec latency 949.864 ms 2024-06-06T08:10:22.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:22 smithi195 ceph-mon[37040]: pgmap v710: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 458 op/s 2024-06-06T08:10:22.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:22 smithi181 ceph-mon[36790]: pgmap v710: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 458 op/s 2024-06-06T08:10:22.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:22 smithi087 ceph-mon[32276]: pgmap v710: 129 pgs: 129 active+clean; 604 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 458 op/s 2024-06-06T08:10:23.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 106540 11.31 MB/sec execute 385 sec latency 1663.761 ms 2024-06-06T08:10:24.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 106778 11.31 MB/sec execute 386 sec latency 798.951 ms 2024-06-06T08:10:24.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:24 smithi195 ceph-mon[37040]: pgmap v711: 129 pgs: 129 active+clean; 620 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 497 op/s 2024-06-06T08:10:24.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:24 smithi181 ceph-mon[36790]: pgmap v711: 129 pgs: 129 active+clean; 620 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 497 op/s 2024-06-06T08:10:24.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:24 smithi087 ceph-mon[32276]: pgmap v711: 129 pgs: 129 active+clean; 620 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.9 MiB/s wr, 497 op/s 2024-06-06T08:10:25.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 107012 11.30 MB/sec execute 387 sec latency 1728.874 ms 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.020+0000 7fcda7fff640 1 -- 172.21.15.87:0/1227154572 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5574e010cc20 con 0x5574e0225ff0 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.020+0000 7fcda7fff640 10 client.18480 handle_command_reply: tid=0 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.020+0000 7fcdd1c4a740 1 client.18480 shutdown 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.020+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 msgr2=0x5574e01ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.020+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009230 tx=0x7fcdbc004770 comp rx=0 tx=0).stop 2024-06-06T08:10:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 shutdown_connections 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5574e0225ff0 0x5574e02283e0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcda7fff640 0 client.18480 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574e01aa760 0x5574e01af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574e019b990 0x5574e01aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 --2- 172.21.15.87:0/1227154572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5574e019b070 0x5574e01ae5a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 >> 172.21.15.87:0/1227154572 conn(0x5574e0197680 msgr2=0x5574e01967e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 shutdown_connections 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 1 -- 172.21.15.87:0/1227154572 wait complete. 2024-06-06T08:10:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.021+0000 7fcdd1c4a740 20 client.18480 trim_cache size 0 max 16384 2024-06-06T08:10:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:10:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:10:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 msgr2=0x7fcdb80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 0x7fcdb80a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0022000 tx=0x7fcdc0023000 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 msgr2=0x7fcdc8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e9f0 tx=0x7fcdb400eeb0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 shutdown_connections 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdb80a2da0 0x7fcdb80a5260 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc8106890 0x7fcdc81a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdc8101290 0x7fcdc8199540 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 --2- 172.21.15.87:0/3552377413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdc81008d0 0x7fcdc8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 >> 172.21.15.87:0/3552377413 conn(0x7fcdc8077a20 msgr2=0x7fcdc8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 shutdown_connections 2024-06-06T08:10:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:26.022+0000 7fcdcef3b640 1 -- 172.21.15.87:0/3552377413 wait complete. 2024-06-06T08:10:26.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 107287 11.31 MB/sec execute 388 sec latency 2691.580 ms 2024-06-06T08:10:26.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:26 smithi195 ceph-mon[37040]: pgmap v712: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 530 op/s 2024-06-06T08:10:26.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:26 smithi181 ceph-mon[36790]: pgmap v712: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 530 op/s 2024-06-06T08:10:26.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:26 smithi087 ceph-mon[32276]: pgmap v712: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.8 MiB/s wr, 530 op/s 2024-06-06T08:10:27.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 107618 11.34 MB/sec execute 389 sec latency 207.486 ms 2024-06-06T08:10:28.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 107872 11.35 MB/sec execute 390 sec latency 352.545 ms 2024-06-06T08:10:28.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:28 smithi195 ceph-mon[37040]: pgmap v713: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 514 op/s 2024-06-06T08:10:28.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:28 smithi181 ceph-mon[36790]: pgmap v713: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 514 op/s 2024-06-06T08:10:28.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:28 smithi087 ceph-mon[32276]: pgmap v713: 129 pgs: 129 active+clean; 623 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.5 MiB/s wr, 514 op/s 2024-06-06T08:10:29.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 108177 11.37 MB/sec execute 391 sec latency 211.168 ms 2024-06-06T08:10:30.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 108308 11.35 MB/sec execute 392 sec latency 577.200 ms 2024-06-06T08:10:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:30 smithi195 ceph-mon[37040]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.4 MiB/s wr, 554 op/s 2024-06-06T08:10:30.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:30 smithi181 ceph-mon[36790]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.4 MiB/s wr, 554 op/s 2024-06-06T08:10:30.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:30 smithi087 ceph-mon[32276]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.4 MiB/s wr, 554 op/s 2024-06-06T08:10:31.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 108542 11.35 MB/sec execute 393 sec latency 268.588 ms 2024-06-06T08:10:32.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 108899 11.37 MB/sec execute 394 sec latency 180.798 ms 2024-06-06T08:10:32.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:32 smithi195 ceph-mon[37040]: pgmap v715: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-06-06T08:10:32.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:32 smithi181 ceph-mon[36790]: pgmap v715: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-06-06T08:10:32.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:32 smithi087 ceph-mon[32276]: pgmap v715: 129 pgs: 129 active+clean; 621 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-06-06T08:10:33.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 109162 11.38 MB/sec execute 395 sec latency 314.755 ms 2024-06-06T08:10:34.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 109440 11.39 MB/sec execute 396 sec latency 394.587 ms 2024-06-06T08:10:34.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:34 smithi195 ceph-mon[37040]: pgmap v716: 129 pgs: 129 active+clean; 633 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.6 MiB/s wr, 576 op/s 2024-06-06T08:10:34.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:34 smithi181 ceph-mon[36790]: pgmap v716: 129 pgs: 129 active+clean; 633 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.6 MiB/s wr, 576 op/s 2024-06-06T08:10:34.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:34 smithi087 ceph-mon[32276]: pgmap v716: 129 pgs: 129 active+clean; 633 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.6 MiB/s wr, 576 op/s 2024-06-06T08:10:35.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 109640 11.38 MB/sec execute 397 sec latency 426.540 ms 2024-06-06T08:10:36.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 109932 11.39 MB/sec execute 398 sec latency 327.841 ms 2024-06-06T08:10:36.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:36 smithi195 ceph-mon[37040]: pgmap v717: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 382 op/s 2024-06-06T08:10:36.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:36 smithi181 ceph-mon[36790]: pgmap v717: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 382 op/s 2024-06-06T08:10:36.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:36 smithi087 ceph-mon[32276]: pgmap v717: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 382 op/s 2024-06-06T08:10:37.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 110213 11.39 MB/sec execute 399 sec latency 281.218 ms 2024-06-06T08:10:38.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 110548 11.41 MB/sec execute 400 sec latency 270.043 ms 2024-06-06T08:10:38.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:38 smithi195 ceph-mon[37040]: pgmap v718: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-06-06T08:10:38.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:38 smithi181 ceph-mon[36790]: pgmap v718: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-06-06T08:10:38.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:38 smithi087 ceph-mon[32276]: pgmap v718: 129 pgs: 129 active+clean; 634 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-06-06T08:10:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- 172.21.15.87:0/2204704716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100ac0 msgr2=0x7fdbc8106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/2204704716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100ac0 0x7fdbc8106650 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009900 tx=0x7fdbbc02a410 comp rx=0 tx=0).stop 2024-06-06T08:10:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- 172.21.15.87:0/2204704716 shutdown_connections 2024-06-06T08:10:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/2204704716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 0x7fdbc810e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/2204704716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100ac0 0x7fdbc8106650 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/2204704716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc8100110 0x7fdbc81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- 172.21.15.87:0/2204704716 >> 172.21.15.87:0/2204704716 conn(0x7fdbc8076e00 msgr2=0x7fdbc80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- 172.21.15.87:0/2204704716 shutdown_connections 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- 172.21.15.87:0/2204704716 wait complete. 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 Processor -- start 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.117+0000 7fdbce8ba640 1 -- start start 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 0x7fdbc8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc819cab0 0x7fdbc819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8111db0 con 0x7fdbc8100110 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8111c30 con 0x7fdbc819cab0 2024-06-06T08:10:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbce8ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8111f30 con 0x7fdbc8106d80 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48818/0 (socket says 172.21.15.87:48818) 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 -- 172.21.15.87:0/265279641 learned_addr learned my addr 172.21.15.87:0/265279641 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 0x7fdbc8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbcce57640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc819cab0 0x7fdbc819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 -- 172.21.15.87:0/265279641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 msgr2=0x7fdbc8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 0x7fdbc8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 -- 172.21.15.87:0/265279641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc819cab0 msgr2=0x7fdbc819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc819cab0 0x7fdbc819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 -- 172.21.15.87:0/265279641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc819d5a0 con 0x7fdbc8100110 2024-06-06T08:10:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.118+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9f0 tx=0x7fdbb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.119+0000 7fdbc57fa640 1 -- 172.21.15.87:0/265279641 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800cd80 con 0x7fdbc8100110 2024-06-06T08:10:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.119+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc819d890 con 0x7fdbc8100110 2024-06-06T08:10:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.119+0000 7fdbc57fa640 1 -- 172.21.15.87:0/265279641 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb8010910 con 0x7fdbc8100110 2024-06-06T08:10:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.119+0000 7fdbc57fa640 1 -- 172.21.15.87:0/265279641 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80188f0 con 0x7fdbc8100110 2024-06-06T08:10:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.119+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a50a0 con 0x7fdbc8100110 2024-06-06T08:10:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 Processor -- start 2024-06-06T08:10:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 -- start start 2024-06-06T08:10:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbc57fa640 1 -- 172.21.15.87:0/265279641 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdbb8010ac0 con 0x7fdbc8100110 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f990ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f990f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d0f98664f0 con 0x55d0f9902a90 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d0f9804870 con 0x55d0f990d350 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d0f98624b0 con 0x55d0f9902170 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbc7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f990ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:53136/0 (socket says 172.21.15.87:53136) 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbc77fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f990f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.122+0000 7fdbcce57640 1 -- 172.21.15.87:0/1173312027 learned_addr learned my addr 172.21.15.87:0/1173312027 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 -- 172.21.15.87:0/1173312027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 msgr2=0x55d0f990f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f990f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 -- 172.21.15.87:0/1173312027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 msgr2=0x55d0f990ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f990ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 -- 172.21.15.87:0/1173312027 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f98209b0 con 0x55d0f9902170 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f990ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbc57fa640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 0x7fdbb00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009f50 tx=0x7fdbc00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6668ff815554428 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 0x7fdbb00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbae7fc640 1 -- 172.21.15.87:0/1173312027 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0004d20 con 0x55d0f9902170 2024-06-06T08:10:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbc57fa640 1 -- 172.21.15.87:0/265279641 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fdbb80cfc50 con 0x7fdbc8100110 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbae7fc640 1 -- 172.21.15.87:0/1173312027 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0004050 con 0x55d0f9902170 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.123+0000 7fdbae7fc640 1 -- 172.21.15.87:0/1173312027 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0005280 con 0x55d0f9902170 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1173312027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 msgr2=0x55d0f9902550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbc7fff640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 0x7fdbb00a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80cd030 tx=0x7fdbb80d37a0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009f50 tx=0x7fdbc00118d0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1173312027 shutdown_connections 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f990f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f990ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1173312027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f9902550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1173312027 >> 172.21.15.87:0/1173312027 conn(0x55d0f98fe680 msgr2=0x55d0f98fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1173312027 shutdown_connections 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1173312027 wait complete. 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 -- start start 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.124+0000 7fdbd15c7740 1 Processor -- start 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f9902a90 0x55d0f990a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f990d350 0x55d0f990ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55d0f990ff80 con 0x55d0f9902170 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d0f97efae0 con 0x55d0f9902a90 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d0f9910100 con 0x55d0f990d350 2024-06-06T08:10:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 10 client.0 _ll_register_callbacks cb 0x55d0f98289b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48834/0 (socket says 172.21.15.87:48834) 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 -- 172.21.15.87:0/1327505711 learned_addr learned my addr 172.21.15.87:0/1327505711 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f9902a90 0x55d0f990a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f990d350 0x55d0f990ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 -- 172.21.15.87:0/1327505711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f990d350 msgr2=0x55d0f990ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f990d350 0x55d0f990ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 -- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f9902a90 msgr2=0x55d0f990a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f9902a90 0x55d0f990a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 -- 172.21.15.87:0/1327505711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f957a210 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbcce57640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009250 tx=0x7fdbc0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ff753c08d6e7cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc002b020 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fdbc0025450 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.125+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc0004050 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdbc0003e40 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 10 client.18492 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d0f9646620 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0004050 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc0027810 con 0x55d0f9902170 2024-06-06T08:10:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 10 client.18492 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 20 client.18492 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f991d330 0x55d0f991f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 4 client.18492 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f9879f10 con 0x55d0f991d330 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f991d330 0x55d0f991f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.126+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f991d330 0x55d0f991f720 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bf5512e7d8ae05 server_cookie=150e30c5590f80d7 in_seq=0 out_seq=0 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.127+0000 7fdba77fe640 10 client.18492 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdba77fe640 1 -- 172.21.15.87:0/1327505711 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d0f9879f10 con 0x55d0f991d330 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdba77fe640 10 client.18492 handle_command_reply: tid=0 2024-06-06T08:10:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 client.18492 shutdown 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 msgr2=0x55d0f990a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009250 tx=0x7fdbc0009c80 comp rx=0 tx=0).stop 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 shutdown_connections 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f991d330 0x55d0f991f720 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f990d350 0x55d0f990ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f9902a90 0x55d0f990a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdba77fe640 0 client.18492 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/1327505711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f990a1f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 >> 172.21.15.87:0/1327505711 conn(0x55d0f98fe680 msgr2=0x55d0f9904400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 shutdown_connections 2024-06-06T08:10:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 1 -- 172.21.15.87:0/1327505711 wait complete. 2024-06-06T08:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.129+0000 7fdbd15c7740 20 client.18492 trim_cache size 0 max 16384 2024-06-06T08:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 Processor -- start 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 -- start start 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902a90 0x55d0f9904230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9924e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81b1a00 con 0x55d0f9902170 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81af2d0 con 0x55d0f990d350 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81ad140 con 0x55d0f9902a90 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48848/0 (socket says 172.21.15.87:48848) 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 -- 172.21.15.87:0/586546478 learned_addr learned my addr 172.21.15.87:0/586546478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902a90 0x55d0f9904230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9924e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 -- 172.21.15.87:0/586546478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902a90 msgr2=0x55d0f9904230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902a90 0x55d0f9904230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 -- 172.21.15.87:0/586546478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 msgr2=0x55d0f9924e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9924e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.145+0000 7fdbcce57640 1 -- 172.21.15.87:0/586546478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f99191e0 con 0x55d0f9902170 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.146+0000 7fdbcce57640 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011ed0 tx=0x7fdbc0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aefc5d19b9e965 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.146+0000 7fdbadffb640 1 -- 172.21.15.87:0/586546478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc002b070 con 0x55d0f9902170 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.146+0000 7fdbadffb640 1 -- 172.21.15.87:0/586546478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0003fe0 con 0x55d0f9902170 2024-06-06T08:10:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.146+0000 7fdbadffb640 1 -- 172.21.15.87:0/586546478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc003d6b0 con 0x55d0f9902170 2024-06-06T08:10:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 -- 172.21.15.87:0/586546478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 msgr2=0x55d0f99099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011ed0 tx=0x7fdbc0005d30 comp rx=0 tx=0).stop 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 -- 172.21.15.87:0/586546478 shutdown_connections 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9924e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902a90 0x55d0f9904230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/586546478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902170 0x55d0f99099a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.149+0000 7fdbd15c7740 1 -- 172.21.15.87:0/586546478 >> 172.21.15.87:0/586546478 conn(0x55d0f98fe680 msgr2=0x55d0f9903b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.150+0000 7fdbd15c7740 1 -- 172.21.15.87:0/586546478 shutdown_connections 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.150+0000 7fdbd15c7740 1 -- 172.21.15.87:0/586546478 wait complete. 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.150+0000 7fdbd15c7740 1 -- start start 2024-06-06T08:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.150+0000 7fdbd15c7740 1 Processor -- start 2024-06-06T08:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f99156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9910c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81ad790 con 0x55d0f9902a90 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81aec00 con 0x55d0f990d350 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81ade60 con 0x55d0f9902170 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbd15c7740 10 client.0 _ll_register_callbacks cb 0x7fdbb80d3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48858/0 (socket says 172.21.15.87:48858) 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 -- 172.21.15.87:0/2105266977 learned_addr learned my addr 172.21.15.87:0/2105266977 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 -- 172.21.15.87:0/2105266977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 msgr2=0x55d0f99156e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f99156e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 -- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 msgr2=0x55d0f9910c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9910c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 -- 172.21.15.87:0/2105266977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f9911450 con 0x55d0f9902a90 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdbc77fe640 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009230 tx=0x7fdbbc032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932af28b84c0c22 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.151+0000 7fdba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc035040 con 0x55d0f9902a90 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fdbbc005040 con 0x55d0f9902a90 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc0003fe0 con 0x55d0f9902a90 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc054050 con 0x55d0f9902a90 2024-06-06T08:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 10 client.18498 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d0f9987ac0 con 0x55d0f9902a90 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0003fe0 con 0x55d0f9902a90 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fdbbc0035f0 con 0x55d0f9902a90 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 10 client.18498 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 20 client.18498 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f99881e0 0x55d0f998a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 4 client.18498 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.152+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0f9873c20 con 0x55d0f99881e0 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.153+0000 7fdbcce57640 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f99881e0 0x55d0f998a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.153+0000 7fdba77fe640 10 client.18498 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:39.153+0000 7fdbcce57640 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f99881e0 0x55d0f998a600 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54c6102bdbbff26 server_cookie=cf3f3b1c438a10d0 in_seq=0 out_seq=0 2024-06-06T08:10:39.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 110933 11.44 MB/sec execute 401 sec latency 268.974 ms 2024-06-06T08:10:40.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 110994 11.42 MB/sec execute 402 sec latency 772.786 ms 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.212+0000 7fdba77fe640 1 -- 172.21.15.87:0/2105266977 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d0f9873c20 con 0x55d0f99881e0 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.212+0000 7fdba77fe640 10 client.18498 handle_command_reply: tid=0 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.212+0000 7fdbd15c7740 1 client.18498 shutdown 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 msgr2=0x55d0f9915c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009230 tx=0x7fdbbc032e80 comp rx=0 tx=0).stop 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 shutdown_connections 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55d0f99881e0 0x55d0f998a600 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0f990d350 0x55d0f9910c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdba77fe640 0 client.18498 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d0f9902a90 0x55d0f9915c20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 --2- 172.21.15.87:0/2105266977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0f9902170 0x55d0f99156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 >> 172.21.15.87:0/2105266977 conn(0x55d0f98fe680 msgr2=0x55d0f9918a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 shutdown_connections 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 1 -- 172.21.15.87:0/2105266977 wait complete. 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.213+0000 7fdbd15c7740 20 client.18498 trim_cache size 0 max 16384 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:10:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:10:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 msgr2=0x7fdbb00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 0x7fdbb00a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80cd030 tx=0x7fdbb80d37a0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 msgr2=0x7fdbc81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9f0 tx=0x7fdbb800eeb0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 shutdown_connections 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fdbb00a2da0 0x7fdbb00a5260 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbc819cab0 0x7fdbc819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbc8106d80 0x7fdbc8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 --2- 172.21.15.87:0/265279641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbc8100110 0x7fdbc81981f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 >> 172.21.15.87:0/265279641 conn(0x7fdbc8076e00 msgr2=0x7fdbc8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 shutdown_connections 2024-06-06T08:10:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:40.214+0000 7fdbce8ba640 1 -- 172.21.15.87:0/265279641 wait complete. 2024-06-06T08:10:40.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:40 smithi195 ceph-mon[37040]: pgmap v719: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-06-06T08:10:40.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:40 smithi181 ceph-mon[36790]: pgmap v719: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-06-06T08:10:40.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:40 smithi087 ceph-mon[32276]: pgmap v719: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.6 MiB/s wr, 422 op/s 2024-06-06T08:10:41.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 111273 11.43 MB/sec execute 403 sec latency 856.734 ms 2024-06-06T08:10:42.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 111513 11.42 MB/sec execute 404 sec latency 491.793 ms 2024-06-06T08:10:42.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:42 smithi195 ceph-mon[37040]: pgmap v720: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-06T08:10:42.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:42 smithi181 ceph-mon[36790]: pgmap v720: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-06T08:10:42.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:42 smithi087 ceph-mon[32276]: pgmap v720: 129 pgs: 129 active+clean; 648 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-06T08:10:43.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 111846 11.44 MB/sec execute 405 sec latency 439.460 ms 2024-06-06T08:10:44.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 112239 11.47 MB/sec execute 406 sec latency 189.759 ms 2024-06-06T08:10:44.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:44 smithi195 ceph-mon[37040]: pgmap v721: 129 pgs: 129 active+clean; 628 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 486 op/s 2024-06-06T08:10:44.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:44 smithi181 ceph-mon[36790]: pgmap v721: 129 pgs: 129 active+clean; 628 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 486 op/s 2024-06-06T08:10:44.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:44 smithi087 ceph-mon[32276]: pgmap v721: 129 pgs: 129 active+clean; 628 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 486 op/s 2024-06-06T08:10:45.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 112459 11.49 MB/sec execute 407 sec latency 354.925 ms 2024-06-06T08:10:46.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 112743 11.50 MB/sec execute 408 sec latency 346.692 ms 2024-06-06T08:10:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:46 smithi195 ceph-mon[37040]: pgmap v722: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.9 MiB/s wr, 476 op/s 2024-06-06T08:10:46.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:46 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:10:46.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:46 smithi181 ceph-mon[36790]: pgmap v722: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.9 MiB/s wr, 476 op/s 2024-06-06T08:10:46.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:46 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:10:46.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:46 smithi087 ceph-mon[32276]: pgmap v722: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.9 MiB/s wr, 476 op/s 2024-06-06T08:10:46.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:46 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:10:47.189 INFO:tasks.workunit.client.0.smithi087.stdout: 10 112980 11.49 MB/sec execute 409 sec latency 190.211 ms 2024-06-06T08:10:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:10:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:10:47.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:10:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:10:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:10:47.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:10:48.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:10:48.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:10:48.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:10:48.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 113315 11.51 MB/sec execute 410 sec latency 192.724 ms 2024-06-06T08:10:48.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:48 smithi181 ceph-mon[36790]: pgmap v723: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:10:48.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:48 smithi087 ceph-mon[32276]: pgmap v723: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:10:49.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:48 smithi195 ceph-mon[37040]: pgmap v723: 129 pgs: 129 active+clean; 633 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 446 op/s 2024-06-06T08:10:49.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 113582 11.52 MB/sec execute 411 sec latency 540.123 ms 2024-06-06T08:10:49.497 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T08:10:49.498 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-06-06T08:10:49.498 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:10:49.764 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:10:50.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 113839 11.53 MB/sec execute 412 sec latency 383.720 ms 2024-06-06T08:10:50.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:50 smithi181 ceph-mon[36790]: pgmap v724: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-06-06T08:10:50.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:50 smithi087 ceph-mon[32276]: pgmap v724: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-06-06T08:10:51.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:50 smithi195 ceph-mon[37040]: pgmap v724: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-06-06T08:10:51.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114003 11.52 MB/sec execute 413 sec latency 471.390 ms 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 -- 172.21.15.87:0/2292093457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc101d80 msgr2=0x7f7ebc10e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 --2- 172.21.15.87:0/2292093457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc101d80 0x7f7ebc10e270 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009990 tx=0x7f7eac009ca0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 -- 172.21.15.87:0/2292093457 shutdown_connections 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 --2- 172.21.15.87:0/2292093457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc107430 0x7f7ebc107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 --2- 172.21.15.87:0/2292093457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc101d80 0x7f7ebc10e270 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 --2- 172.21.15.87:0/2292093457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.763+0000 7f7ec3909700 1 -- 172.21.15.87:0/2292093457 >> 172.21.15.87:0/2292093457 conn(0x7f7ebc0783d0 msgr2=0x7f7ebc0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:51.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.764+0000 7f7ec3909700 1 -- 172.21.15.87:0/2292093457 shutdown_connections 2024-06-06T08:10:51.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.764+0000 7f7ec3909700 1 -- 172.21.15.87:0/2292093457 wait complete. 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 Processor -- start 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 -- start start 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 0x7f7ebc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 0x7f7ebc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1116a0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc111520 con 0x7f7ebc101d80 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec3909700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc111820 con 0x7f7ebc107430 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec16a5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec1ea6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 0x7f7ebc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec1ea6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 0x7f7ebc1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:57980/0 (socket says 172.21.15.87:57980) 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec16a5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53172/0 (socket says 172.21.15.87:53172) 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.765+0000 7f7ec1ea6700 1 -- 172.21.15.87:0/1741172385 learned_addr learned my addr 172.21.15.87:0/1741172385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec0ea4700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 0x7f7ebc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 -- 172.21.15.87:0/1741172385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 msgr2=0x7f7ebc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 0x7f7ebc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 -- 172.21.15.87:0/1741172385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 msgr2=0x7f7ebc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 0x7f7ebc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 -- 172.21.15.87:0/1741172385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc199100 con 0x7f7ebc1013c0 2024-06-06T08:10:51.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec0ea4700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 0x7f7ebc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec16a5700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400e9f0 tx=0x7f7eb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400cbd0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1993f0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.766+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb400cd70 con 0x7f7ebc1013c0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.767+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4018710 con 0x7f7ebc1013c0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.767+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc102fd0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.768+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc102ae0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.770+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7eb4010430 con 0x7f7ebc1013c0 2024-06-06T08:10:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.771+0000 7f7eba7fc700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 0x7f7ea40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:51.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.771+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7f7eb40cfeb0 con 0x7f7ebc1013c0 2024-06-06T08:10:51.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.771+0000 7f7ec0ea4700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 0x7f7ea40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:51.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.772+0000 7f7ec0ea4700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 0x7f7ea40a52c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc078940 tx=0x7f7eb000b040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:51.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:51.776+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7eb409c650 con 0x7f7ebc1013c0 2024-06-06T08:10:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:10:52.092 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:10:52.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.090+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ebc069180 con 0x7f7ebc1013c0 2024-06-06T08:10:52.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.092+0000 7f7eba7fc700 1 -- 172.21.15.87:0/1741172385 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7f7eb409bf50 con 0x7f7ebc1013c0 2024-06-06T08:10:52.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 msgr2=0x7f7ea40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 0x7f7ea40a52c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc078940 tx=0x7f7eb000b040 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 msgr2=0x7f7ebc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400e9f0 tx=0x7f7eb400ed00 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 shutdown_connections 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7ea40a2e00 0x7f7ea40a52c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ebc107430 0x7f7ebc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ebc1013c0 0x7f7ebc19e7e0 secure :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400e9f0 tx=0x7f7eb400ed00 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 --2- 172.21.15.87:0/1741172385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ebc101d80 0x7f7ebc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.097+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 >> 172.21.15.87:0/1741172385 conn(0x7f7ebc0783d0 msgr2=0x7f7ebc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.099+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 shutdown_connections 2024-06-06T08:10:52.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:52.099+0000 7f7ec3909700 1 -- 172.21.15.87:0/1741172385 wait complete. 2024-06-06T08:10:52.100 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:10:52.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114205 11.51 MB/sec execute 414 sec latency 272.879 ms 2024-06-06T08:10:52.749 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:52 smithi087 ceph-mon[32276]: pgmap v725: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-06-06T08:10:52.749 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:52 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1741172385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:10:52.790 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-06-06T08:10:52.790 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-06-06T08:10:52.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:52 smithi181 ceph-mon[36790]: pgmap v725: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-06-06T08:10:52.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:52 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1741172385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:10:53.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:52 smithi195 ceph-mon[37040]: pgmap v725: 129 pgs: 129 active+clean; 647 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-06-06T08:10:53.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:52 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1741172385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:10:53.086 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:10:53.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114547 11.51 MB/sec execute 415 sec latency 384.523 ms 2024-06-06T08:10:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 -- 172.21.15.87:0/2541948461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1079b0 msgr2=0x7fca0c107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 --2- 172.21.15.87:0/2541948461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1079b0 0x7fca0c107d90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fca08009930 tx=0x7fca0802a420 comp rx=0 tx=0).stop 2024-06-06T08:10:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 -- 172.21.15.87:0/2541948461 shutdown_connections 2024-06-06T08:10:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 --2- 172.21.15.87:0/2541948461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 0x7fca0c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 --2- 172.21.15.87:0/2541948461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1019a0 0x7fca0c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 --2- 172.21.15.87:0/2541948461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1079b0 0x7fca0c107d90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 -- 172.21.15.87:0/2541948461 >> 172.21.15.87:0/2541948461 conn(0x7fca0c0fd470 msgr2=0x7fca0c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 -- 172.21.15.87:0/2541948461 shutdown_connections 2024-06-06T08:10:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.314+0000 7fca14770640 1 -- 172.21.15.87:0/2541948461 wait complete. 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 Processor -- start 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 -- start start 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 0x7fca0c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1079b0 0x7fca0c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111dc0 con 0x7fca0c1019a0 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111c40 con 0x7fca0c1079b0 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.315+0000 7fca14770640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c111f40 con 0x7fca0c102360 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53188/0 (socket says 172.21.15.87:53188) 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 -- 172.21.15.87:0/1262369496 learned_addr learned my addr 172.21.15.87:0/1262369496 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca12d0d640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1079b0 0x7fca0c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca11d0b640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 0x7fca0c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 -- 172.21.15.87:0/1262369496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 msgr2=0x7fca0c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 0x7fca0c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 -- 172.21.15.87:0/1262369496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1079b0 msgr2=0x7fca0c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1079b0 0x7fca0c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 -- 172.21.15.87:0/1262369496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c19d1e0 con 0x7fca0c1019a0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca1250c640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fca08009230 tx=0x7fca08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca037fe640 1 -- 172.21.15.87:0/1262369496 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0803f030 con 0x7fca0c1019a0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca037fe640 1 -- 172.21.15.87:0/1262369496 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08004440 con 0x7fca0c1019a0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c19d470 con 0x7fca0c1019a0 2024-06-06T08:10:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca037fe640 1 -- 172.21.15.87:0/1262369496 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08035070 con 0x7fca0c1019a0 2024-06-06T08:10:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.316+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c1a4d60 con 0x7fca0c1019a0 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.318+0000 7fca037fe640 1 -- 172.21.15.87:0/1262369496 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fca08042030 con 0x7fca0c1019a0 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 Processor -- start 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 -- start start 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca037fe640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 0x7fc9f80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560d425414f0 con 0x560d425dda90 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560d424df870 con 0x560d425e8350 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1747d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560d4253d4b0 con 0x560d425dd170 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca12d0d640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 0x7fc9f80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca1250c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.319+0000 7fca037fe640 1 -- 172.21.15.87:0/1262369496 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fca080efaf0 con 0x7fca0c1019a0 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca12d0d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58012/0 (socket says 172.21.15.87:58012) 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53198/0 (socket says 172.21.15.87:53198) 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 -- 172.21.15.87:0/2748939767 learned_addr learned my addr 172.21.15.87:0/2748939767 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 -- 172.21.15.87:0/2748939767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 msgr2=0x560d425ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 -- 172.21.15.87:0/2748939767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 msgr2=0x560d425e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 -- 172.21.15.87:0/2748939767 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d424fb9b0 con 0x560d425dd170 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca11d0b640 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca12d0d640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 0x7fc9f80a52e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca04005fd0 tx=0x7fca04009e90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca1250c640 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca08032c00 tx=0x7fca0802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50fb60b78b293535 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca00ff9640 1 -- 172.21.15.87:0/2748939767 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0807be60 con 0x560d425dd170 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca00ff9640 1 -- 172.21.15.87:0/2748939767 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08098eb0 con 0x560d425dd170 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.320+0000 7fca00ff9640 1 -- 172.21.15.87:0/2748939767 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08049040 con 0x560d425dd170 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 -- 172.21.15.87:0/2748939767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 msgr2=0x560d425dd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca08032c00 tx=0x7fca0802a420 comp rx=0 tx=0).stop 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 -- 172.21.15.87:0/2748939767 shutdown_connections 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 --2- 172.21.15.87:0/2748939767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dd170 0x560d425dd550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 -- 172.21.15.87:0/2748939767 >> 172.21.15.87:0/2748939767 conn(0x560d425d9680 msgr2=0x560d425d9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 -- 172.21.15.87:0/2748939767 shutdown_connections 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.321+0000 7fca1747d740 1 -- 172.21.15.87:0/2748939767 wait complete. 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 -- start start 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 Processor -- start 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425e8350 0x560d425e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x560d425ead40 con 0x560d425dda90 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560d424caae0 con 0x560d425dd170 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560d425eaec0 con 0x560d425e8350 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1747d740 10 client.0 _ll_register_callbacks cb 0x560d425039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53212/0 (socket says 172.21.15.87:53212) 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 -- 172.21.15.87:0/3499506446 learned_addr learned my addr 172.21.15.87:0/3499506446 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1250c640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca12d0d640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425e8350 0x560d425e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 -- 172.21.15.87:0/3499506446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425e8350 msgr2=0x560d425e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425e8350 0x560d425e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 -- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 msgr2=0x560d425e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 -- 172.21.15.87:0/3499506446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d42255210 con 0x560d425dda90 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca1250c640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc012720 tx=0x7fc9fc012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7886378788f73d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.322+0000 7fc9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004030 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fc9fc0041d0 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca08098eb0 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc9fc025050 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 10 client.18516 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560d42321620 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08098eb0 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fc9fc003a60 con 0x560d425dda90 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 10 client.18516 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 10 client.18516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 20 client.18516 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d425f8690 0x560d425fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 4 client.18516 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d42554f10 con 0x560d425f8690 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.323+0000 7fca1250c640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d425f8690 0x560d425fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.324+0000 7fca1250c640 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d425f8690 0x560d425fab30 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d5db4635072f60 server_cookie=59def200104956c8 in_seq=0 out_seq=0 2024-06-06T08:10:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.324+0000 7fc9f5ffb640 10 client.18516 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/3499506446 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d42554f10 con 0x560d425f8690 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fc9f5ffb640 10 client.18516 handle_command_reply: tid=0 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 client.18516 shutdown 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 msgr2=0x560d425e5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc012720 tx=0x7fc9fc012be0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 shutdown_connections 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d425f8690 0x560d425fab30 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425e8350 0x560d425e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dda90 0x560d425e5850 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 --2- 172.21.15.87:0/3499506446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 >> 172.21.15.87:0/3499506446 conn(0x560d425d9680 msgr2=0x560d425df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fc9f5ffb640 0 client.18516 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 shutdown_connections 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.326+0000 7fca1747d740 1 -- 172.21.15.87:0/3499506446 wait complete. 2024-06-06T08:10:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.327+0000 7fca1747d740 20 client.18516 trim_cache size 0 max 16384 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 Processor -- start 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 -- start start 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425e8350 0x560d425d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1b1980 con 0x560d425e8350 2024-06-06T08:10:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1af250 con 0x560d425dd170 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1747d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1ad0c0 con 0x560d425dda90 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca11d0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58032/0 (socket says 172.21.15.87:58032) 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca11d0b640 1 -- 172.21.15.87:0/3358107139 learned_addr learned my addr 172.21.15.87:0/3358107139 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca1250c640 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.344+0000 7fca12d0d640 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425e8350 0x560d425d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 -- 172.21.15.87:0/3358107139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 msgr2=0x560d425f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 -- 172.21.15.87:0/3358107139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425e8350 msgr2=0x560d425d8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425e8350 0x560d425d8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 -- 172.21.15.87:0/3358107139 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d425d4f70 con 0x560d425dda90 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fca11d0b640 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001eb0 tx=0x7fc9fc0057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5add8d5da38a559 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fc9f7fff640 1 -- 172.21.15.87:0/3358107139 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc022040 con 0x560d425dda90 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fc9f7fff640 1 -- 172.21.15.87:0/3358107139 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc010e00 con 0x560d425dda90 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.345+0000 7fc9f7fff640 1 -- 172.21.15.87:0/3358107139 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00fb80 con 0x560d425dda90 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- 172.21.15.87:0/3358107139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 msgr2=0x560d425dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001eb0 tx=0x7fc9fc0057d0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- 172.21.15.87:0/3358107139 shutdown_connections 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425e8350 0x560d425d8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425dfb30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 --2- 172.21.15.87:0/3358107139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425dd170 0x560d425f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- 172.21.15.87:0/3358107139 >> 172.21.15.87:0/3358107139 conn(0x560d425d9680 msgr2=0x560d425d7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- 172.21.15.87:0/3358107139 shutdown_connections 2024-06-06T08:10:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- 172.21.15.87:0/3358107139 wait complete. 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.346+0000 7fca1747d740 1 -- start start 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 Processor -- start 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1ad710 con 0x560d425dd170 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1aeb80 con 0x560d425e8350 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c1adde0 con 0x560d425dda90 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1747d740 10 client.0 _ll_register_callbacks cb 0x7fc9fc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53228/0 (socket says 172.21.15.87:53228) 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 -- 172.21.15.87:0/993684214 learned_addr learned my addr 172.21.15.87:0/993684214 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca12d0d640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca11d0b640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 -- 172.21.15.87:0/993684214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 msgr2=0x560d425e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 -- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 msgr2=0x560d425e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425e2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 -- 172.21.15.87:0/993684214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d425e29e0 con 0x560d425dd170 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fca1250c640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fca080bbc30 tx=0x7fca08005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514d81e82a0db40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.347+0000 7fc9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080d2070 con 0x560d425dd170 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fca08003d20 con 0x560d425dd170 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9fc010e00 con 0x560d425dd170 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fca08049040 con 0x560d425dd170 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 10 client.18522 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560d42667600 con 0x560d425dd170 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc010e00 con 0x560d425dd170 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fca0809aac0 con 0x560d425dd170 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 10 client.18522 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 10 client.18522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 20 client.18522 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d42667a50 0x560d42669e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 4 client.18522 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d4254ec20 con 0x560d42667a50 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.348+0000 7fca11d0b640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d42667a50 0x560d42669e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.349+0000 7fca11d0b640 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d42667a50 0x560d42669e40 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb99345b80279f0 server_cookie=7beaa74a65714263 in_seq=0 out_seq=0 2024-06-06T08:10:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:53.349+0000 7fc9f5ffb640 10 client.18522 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fc9f5ffb640 1 -- 172.21.15.87:0/993684214 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d4254ec20 con 0x560d42667a50 2024-06-06T08:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fc9f5ffb640 10 client.18522 handle_command_reply: tid=0 2024-06-06T08:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 client.18522 shutdown 2024-06-06T08:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 msgr2=0x560d425f02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fca080bbc30 tx=0x7fca08005a20 comp rx=0 tx=0).stop 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 shutdown_connections 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x560d42667a50 0x560d42669e40 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d425e8350 0x560d425e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d425dda90 0x560d425e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 --2- 172.21.15.87:0/993684214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560d425dd170 0x560d425f02c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 >> 172.21.15.87:0/993684214 conn(0x560d425d9680 msgr2=0x560d425d87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fc9f5ffb640 0 client.18522 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 shutdown_connections 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 1 -- 172.21.15.87:0/993684214 wait complete. 2024-06-06T08:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.121+0000 7fca1747d740 20 client.18522 trim_cache size 0 max 16384 2024-06-06T08:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 msgr2=0x7fc9f80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 0x7fc9f80a52e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca04005fd0 tx=0x7fca04009e90 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 msgr2=0x7fca0c198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fca08009230 tx=0x7fca08004040 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 shutdown_connections 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fc9f80a2e20 0x7fc9f80a52e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca0c1079b0 0x7fca0c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca0c102360 0x7fca0c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 --2- 172.21.15.87:0/1262369496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca0c1019a0 0x7fca0c198240 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 >> 172.21.15.87:0/1262369496 conn(0x7fca0c0fd470 msgr2=0x7fca0c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 shutdown_connections 2024-06-06T08:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:10:54.122+0000 7fca14770640 1 -- 172.21.15.87:0/1262369496 wait complete. 2024-06-06T08:10:54.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114662 11.50 MB/sec execute 416 sec latency 499.721 ms 2024-06-06T08:10:54.710 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:54 smithi087 ceph-mon[32276]: pgmap v726: 129 pgs: 129 active+clean; 657 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-06-06T08:10:54.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:54 smithi181 ceph-mon[36790]: pgmap v726: 129 pgs: 129 active+clean; 657 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-06-06T08:10:55.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:54 smithi195 ceph-mon[37040]: pgmap v726: 129 pgs: 129 active+clean; 657 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-06-06T08:10:55.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 -- 172.21.15.87:0/3444035681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7581099d0 msgr2=0x7fd758109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 --2- 172.21.15.87:0/3444035681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7581099d0 0x7fd758109e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7480099c0 tx=0x7fd748009cd0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 -- 172.21.15.87:0/3444035681 shutdown_connections 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 --2- 172.21.15.87:0/3444035681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd75811bc70 0x7fd75811e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 --2- 172.21.15.87:0/3444035681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd758110ce0 0x7fd758109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 --2- 172.21.15.87:0/3444035681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7581099d0 0x7fd758109e50 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7480099c0 tx=0x7fd748009cd0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.075+0000 7fd760067700 1 -- 172.21.15.87:0/3444035681 >> 172.21.15.87:0/3444035681 conn(0x7fd75806d600 msgr2=0x7fd75806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:55.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.076+0000 7fd760067700 1 -- 172.21.15.87:0/3444035681 shutdown_connections 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.077+0000 7fd760067700 1 -- 172.21.15.87:0/3444035681 wait complete. 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.078+0000 7fd760067700 1 Processor -- start 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.078+0000 7fd760067700 1 -- start start 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd758110ce0 0x7fd7581ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd75811bc70 0x7fd7581ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd758121660 con 0x7fd75811bc70 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581214e0 con 0x7fd7581a7a80 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd760067700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581217e0 con 0x7fd758110ce0 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd75e604700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd75e604700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:60292/0 (socket says 172.21.15.87:60292) 2024-06-06T08:10:55.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd75e604700 1 -- 172.21.15.87:0/3945233392 learned_addr learned my addr 172.21.15.87:0/3945233392 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd75de03700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd758110ce0 0x7fd7581ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.079+0000 7fd75d602700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd75811bc70 0x7fd7581ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.080+0000 7fd75e604700 1 -- 172.21.15.87:0/3945233392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd758110ce0 msgr2=0x7fd7581ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.080+0000 7fd75e604700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd758110ce0 0x7fd7581ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.080+0000 7fd75e604700 1 -- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd75811bc70 msgr2=0x7fd7581ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.080+0000 7fd75e604700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd75811bc70 0x7fd7581ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.080+0000 7fd75e604700 1 -- 172.21.15.87:0/3945233392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7581a8600 con 0x7fd7581a7a80 2024-06-06T08:10:55.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.094+0000 7fd75e604700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b970 tx=0x7fd75000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.096+0000 7fd74effd700 1 -- 172.21.15.87:0/3945233392 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750019070 con 0x7fd7581a7a80 2024-06-06T08:10:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.096+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75807a6f0 con 0x7fd7581a7a80 2024-06-06T08:10:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.097+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75807ac00 con 0x7fd7581a7a80 2024-06-06T08:10:55.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.097+0000 7fd74effd700 1 -- 172.21.15.87:0/3945233392 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd750014440 con 0x7fd7581a7a80 2024-06-06T08:10:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.097+0000 7fd74effd700 1 -- 172.21.15.87:0/3945233392 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500075d0 con 0x7fd7581a7a80 2024-06-06T08:10:55.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.102+0000 7fd76a8eab80 1 Processor -- start 2024-06-06T08:10:55.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.103+0000 7fd74effd700 1 -- 172.21.15.87:0/3945233392 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd75001c070 con 0x7fd7581a7a80 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.104+0000 7fd74effd700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 0x7fd7440a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.104+0000 7fd75d602700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 0x7fd7440a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.105+0000 7fd75d602700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 0x7fd7440a53f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd748000c00 tx=0x7fd7480058e0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.105+0000 7fd74effd700 1 -- 172.21.15.87:0/3945233392 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fd7500cef90 con 0x7fd7581a7a80 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 -- start start 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff1ce0 0x558f26ff20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 0x558f26febf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 0x558f26fec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558f26f3d9b0 con 0x558f26fec470 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558f26ee58c0 con 0x558f26ff2600 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.106+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f26f39970 con 0x558f26ff1ce0 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.107+0000 7fd75d602700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 0x558f26febf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.107+0000 7fd75d602700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 0x558f26febf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:60296/0 (socket says 172.21.15.87:60296) 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.107+0000 7fd75d602700 1 -- 172.21.15.87:0/1047333186 learned_addr learned my addr 172.21.15.87:0/1047333186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.107+0000 7fd75de03700 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff1ce0 0x558f26ff20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.107+0000 7fd75e604700 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 0x558f26fec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 -- 172.21.15.87:0/1047333186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff1ce0 msgr2=0x558f26ff20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff1ce0 0x558f26ff20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 -- 172.21.15.87:0/1047333186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 msgr2=0x558f26febf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 0x558f26febf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 -- 172.21.15.87:0/1047333186 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f26edc0f0 con 0x558f26fec470 2024-06-06T08:10:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.108+0000 7fd75e604700 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 0x558f26fec8b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cc370 tx=0x7fd75009abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cd2a0089cd2021 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.110+0000 7fd743fff700 1 -- 172.21.15.87:0/1047333186 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500a3db0 con 0x558f26fec470 2024-06-06T08:10:55.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.110+0000 7fd743fff700 1 -- 172.21.15.87:0/1047333186 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7500bb0f0 con 0x558f26fec470 2024-06-06T08:10:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.110+0000 7fd743fff700 1 -- 172.21.15.87:0/1047333186 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500a3b80 con 0x558f26fec470 2024-06-06T08:10:55.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.111+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1047333186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 msgr2=0x558f26fec8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.111+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 0x558f26fec8b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cc370 tx=0x7fd75009abb0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.113+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1047333186 shutdown_connections 2024-06-06T08:10:55.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.113+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fec470 0x558f26fec8b0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.113+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff1ce0 0x558f26ff20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.113+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1047333186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff2600 0x558f26febf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.113+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1047333186 >> 172.21.15.87:0/1047333186 conn(0x558f26f16e60 msgr2=0x558f26fe71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:55.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.116+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1047333186 shutdown_connections 2024-06-06T08:10:55.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.118+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1047333186 wait complete. 2024-06-06T08:10:55.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.118+0000 7fd76a8eab80 1 -- start start 2024-06-06T08:10:55.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.118+0000 7fd76a8eab80 1 Processor -- start 2024-06-06T08:10:55.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.122+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fec470 0x558f26ff8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.122+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.123+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.123+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558f26ff9070 con 0x558f26ff2600 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.123+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558f26eddd90 con 0x558f26ff1ce0 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.123+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f26ff91f0 con 0x558f26fec470 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.123+0000 7fd76a8eab80 10 client.0 _ll_register_callbacks cb 0x558f26ed7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.124+0000 7fd75d602700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.124+0000 7fd75d602700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:60310/0 (socket says 172.21.15.87:60310) 2024-06-06T08:10:55.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.124+0000 7fd75d602700 1 -- 172.21.15.87:0/4121984774 learned_addr learned my addr 172.21.15.87:0/4121984774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75e604700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75d602700 1 -- 172.21.15.87:0/4121984774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fec470 msgr2=0x558f26ff8320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:10:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75d602700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fec470 0x558f26ff8320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75d602700 1 -- 172.21.15.87:0/4121984774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 msgr2=0x558f26ff5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75d602700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75d602700 1 -- 172.21.15.87:0/4121984774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f26e8ca70 con 0x558f26ff1ce0 2024-06-06T08:10:55.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.127+0000 7fd75e604700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:10:55.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.128+0000 7fd75d602700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd74801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308082d7481acdd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.128+0000 7fd7417fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:10:55.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.132+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74800fc50 con 0x558f26ff1ce0 2024-06-06T08:10:55.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.132+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fd74800fdf0 con 0x558f26ff1ce0 2024-06-06T08:10:55.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.132+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7500bb0f0 con 0x558f26ff1ce0 2024-06-06T08:10:55.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.134+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd74803a450 con 0x558f26ff1ce0 2024-06-06T08:10:55.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.135+0000 7fd76a8eab80 10 client.27410 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:55.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.135+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558f27003a00 con 0x558f26ff1ce0 2024-06-06T08:10:55.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.138+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7500bb0f0 con 0x558f26ff1ce0 2024-06-06T08:10:55.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.138+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd748030700 con 0x558f26ff1ce0 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 10 client.27410 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 10 client.27410 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 20 client.27410 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f27006040 0x558f27008430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 4 client.27410 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:10:55.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.139+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f26f4aeb0 con 0x558f27006040 2024-06-06T08:10:55.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.142+0000 7fd75de03700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f27006040 0x558f27008430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.154+0000 7fd7417fa700 10 client.27410 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:55.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.154+0000 7fd75de03700 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f27006040 0x558f27008430 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4849f347bcdf39ea server_cookie=7fc5a8f7149e5d3 in_seq=0 out_seq=0 2024-06-06T08:10:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.157+0000 7fd7417fa700 1 -- 172.21.15.87:0/4121984774 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f26f4aeb0 con 0x558f27006040 2024-06-06T08:10:55.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.157+0000 7fd7417fa700 10 client.27410 handle_command_reply: tid=0 2024-06-06T08:10:55.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.157+0000 7fd76a8eab80 1 client.27410 shutdown 2024-06-06T08:10:55.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.159+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 msgr2=0x558f26ff5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.159+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd74801a050 comp rx=0 tx=0).stop 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 shutdown_connections 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f27006040 0x558f27008430 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fec470 0x558f26ff8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4121984774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26ff1ce0 0x558f26ff5690 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd74801a050 comp rx=0 tx=0).stop 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 >> 172.21.15.87:0/4121984774 conn(0x558f26f16e60 msgr2=0x558f26fe6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:55.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.160+0000 7fd7417fa700 0 client.27410 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.161+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 shutdown_connections 2024-06-06T08:10:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.161+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4121984774 wait complete. 2024-06-06T08:10:55.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.161+0000 7fd76a8eab80 20 client.27410 trim_cache size 0 max 16384 2024-06-06T08:10:55.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114960 11.53 MB/sec execute 417 sec latency 421.819 ms 2024-06-06T08:10:55.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.201+0000 7fd76a8eab80 1 Processor -- start 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 -- start start 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26febf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 0x558f26ff3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c011930 con 0x558f26ff2600 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00f1c0 con 0x558f26fec470 2024-06-06T08:10:55.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.202+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00d030 con 0x558f26fefe30 2024-06-06T08:10:55.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.203+0000 7fd75d602700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.203+0000 7fd75e604700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 0x558f26ff3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.203+0000 7fd75d602700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53288/0 (socket says 172.21.15.87:53288) 2024-06-06T08:10:55.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.203+0000 7fd75d602700 1 -- 172.21.15.87:0/1948221634 learned_addr learned my addr 172.21.15.87:0/1948221634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.205+0000 7fd75e604700 1 -- 172.21.15.87:0/1948221634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 msgr2=0x558f26febf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.205+0000 7fd75e604700 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26febf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.205+0000 7fd75e604700 1 -- 172.21.15.87:0/1948221634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 msgr2=0x558f26ff5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.205+0000 7fd75e604700 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.205+0000 7fd75e604700 1 -- 172.21.15.87:0/1948221634 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f26fe7700 con 0x558f26fefe30 2024-06-06T08:10:55.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.209+0000 7fd75e604700 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 0x558f26ff3270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cc3a0 tx=0x7fd750004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fdedaee607408d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.212+0000 7fd7437fe700 1 -- 172.21.15.87:0/1948221634 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500bb0c0 con 0x558f26fefe30 2024-06-06T08:10:55.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.212+0000 7fd7437fe700 1 -- 172.21.15.87:0/1948221634 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75009f120 con 0x558f26fefe30 2024-06-06T08:10:55.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.212+0000 7fd7437fe700 1 -- 172.21.15.87:0/1948221634 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500a3b80 con 0x558f26fefe30 2024-06-06T08:10:55.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.213+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1948221634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 msgr2=0x558f26ff3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:55.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.213+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 0x558f26ff3270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cc3a0 tx=0x7fd750004be0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.214+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1948221634 shutdown_connections 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.214+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26febf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.214+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26fefe30 0x558f26ff3270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.214+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/1948221634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26ff2600 0x558f26ff5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.214+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1948221634 >> 172.21.15.87:0/1948221634 conn(0x558f26f16e60 msgr2=0x558f26ff6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.217+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1948221634 shutdown_connections 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.217+0000 7fd76a8eab80 1 -- 172.21.15.87:0/1948221634 wait complete. 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.217+0000 7fd76a8eab80 1 -- start start 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.217+0000 7fd76a8eab80 1 Processor -- start 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26ffeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff2600 0x558f26ffb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00d680 con 0x558f26fefe30 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00eaf0 con 0x558f26fec470 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c00dd50 con 0x558f26ff2600 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.218+0000 7fd76a8eab80 10 client.0 _ll_register_callbacks cb 0x558f26ed7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.220+0000 7fd75d602700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.220+0000 7fd75d602700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53298/0 (socket says 172.21.15.87:53298) 2024-06-06T08:10:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.220+0000 7fd75d602700 1 -- 172.21.15.87:0/4053224190 learned_addr learned my addr 172.21.15.87:0/4053224190 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75e604700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff2600 0x558f26ffb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75de03700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26ffeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75d602700 1 -- 172.21.15.87:0/4053224190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff2600 msgr2=0x558f26ffb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75d602700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff2600 0x558f26ffb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75d602700 1 -- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 msgr2=0x558f26ffeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75d602700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26ffeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.221+0000 7fd75d602700 1 -- 172.21.15.87:0/4053224190 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f26ffc100 con 0x558f26fefe30 2024-06-06T08:10:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.223+0000 7fd75d602700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd748030cc0 tx=0x7fd748009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ef8269b101853 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:10:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.224+0000 7fd7417fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:10:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.225+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74803aa00 con 0x558f26fefe30 2024-06-06T08:10:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.225+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd75009f120 con 0x558f26fefe30 2024-06-06T08:10:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.226+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9583+0+0 (secure 0 0 0) 0x7fd74803aba0 con 0x558f26fefe30 2024-06-06T08:10:55.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.230+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd748054050 con 0x558f26fefe30 2024-06-06T08:10:55.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.233+0000 7fd76a8eab80 10 client.18537 fetch_fsmap learned FSMap version 55 2024-06-06T08:10:55.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.233+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558f27069c50 con 0x558f26fefe30 2024-06-06T08:10:55.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.234+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75009f120 con 0x558f26fefe30 2024-06-06T08:10:55.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.234+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd748024410 con 0x558f26fefe30 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 10 client.18537 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 10 client.18537 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 20 client.18537 populate_metadata read hostname 'smithi087' 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f2706a350 0x558f2706c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 4 client.18537 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-06-06T08:10:55.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.235+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x558f26f45430 con 0x558f2706a350 2024-06-06T08:10:55.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.237+0000 7fd75de03700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f2706a350 0x558f2706c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:10:55.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.246+0000 7fd75de03700 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f2706a350 0x558f2706c770 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664e4d19a418070e server_cookie=d4ba3d874b558a5d in_seq=0 out_seq=0 2024-06-06T08:10:55.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:55.246+0000 7fd7417fa700 10 client.18537 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:56.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114972 11.50 MB/sec execute 418 sec latency 1173.819 ms 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.579+0000 7fd7417fa700 1 -- 172.21.15.87:0/4053224190 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558f26f45430 con 0x558f2706a350 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.579+0000 7fd7417fa700 10 client.18537 handle_command_reply: tid=0 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.579+0000 7fd76a8eab80 1 client.18537 shutdown 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.579+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 msgr2=0x558f26ffb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.579+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd748030cc0 tx=0x7fd748009310 comp rx=0 tx=0).stop 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 shutdown_connections 2024-06-06T08:10:56.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x558f2706a350 0x558f2706c770 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558f26fec470 0x558f26ffeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f26ff2600 0x558f26ffb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 --2- 172.21.15.87:0/4053224190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558f26fefe30 0x558f26ffb480 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 >> 172.21.15.87:0/4053224190 conn(0x558f26f16e60 msgr2=0x558f26ff6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd7417fa700 0 client.18537 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 shutdown_connections 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 1 -- 172.21.15.87:0/4053224190 wait complete. 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.580+0000 7fd76a8eab80 20 client.18537 trim_cache size 0 max 16384 2024-06-06T08:10:56.581 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 70, 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "9ebcf3dc": { 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 70, 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:10:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 141.8, 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:10:56.583 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:10:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 msgr2=0x7fd7440a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 0x7fd7440a53f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd748000c00 tx=0x7fd7480058e0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 msgr2=0x7fd7581a7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd75000b970 tx=0x7fd75000bd30 comp rx=0 tx=0).stop 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 shutdown_connections 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7581a7a80 0x7fd7581a7f00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd7440a2f30 0x7fd7440a53f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd758110ce0 0x7fd7581ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 --2- 172.21.15.87:0/3945233392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd75811bc70 0x7fd7581ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 >> 172.21.15.87:0/3945233392 conn(0x7fd75806d600 msgr2=0x7fd758119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 shutdown_connections 2024-06-06T08:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:10:56.581+0000 7fd760067700 1 -- 172.21.15.87:0/3945233392 wait complete. 2024-06-06T08:10:56.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:56 smithi181 ceph-mon[36790]: pgmap v727: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.9 MiB/s wr, 357 op/s 2024-06-06T08:10:56.858 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:56 smithi087 ceph-mon[32276]: pgmap v727: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.9 MiB/s wr, 357 op/s 2024-06-06T08:10:57.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:56 smithi195 ceph-mon[37040]: pgmap v727: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.9 MiB/s wr, 357 op/s 2024-06-06T08:10:57.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.48 MB/sec execute 419 sec latency 2149.892 ms 2024-06-06T08:10:57.369 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9ebcf3dc', quiesce took 7.9 seconds. Will release after: 13.85 2024-06-06T08:10:58.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.45 MB/sec execute 420 sec latency 3149.989 ms 2024-06-06T08:10:58.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:10:58 smithi181 ceph-mon[36790]: pgmap v728: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-06-06T08:10:58.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:10:58 smithi087 ceph-mon[32276]: pgmap v728: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-06-06T08:10:59.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:10:58 smithi195 ceph-mon[37040]: pgmap v728: 129 pgs: 129 active+clean; 658 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-06-06T08:10:59.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.42 MB/sec execute 421 sec latency 4150.044 ms 2024-06-06T08:11:00.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.39 MB/sec execute 422 sec latency 5150.095 ms 2024-06-06T08:11:00.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:00 smithi181 ceph-mon[36790]: pgmap v729: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-06-06T08:11:00.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:00 smithi087 ceph-mon[32276]: pgmap v729: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-06-06T08:11:01.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:00 smithi195 ceph-mon[37040]: pgmap v729: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-06-06T08:11:01.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.37 MB/sec execute 423 sec latency 6150.134 ms 2024-06-06T08:11:02.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.34 MB/sec execute 424 sec latency 7150.226 ms 2024-06-06T08:11:02.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:02 smithi181 ceph-mon[36790]: pgmap v730: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-06-06T08:11:02.811 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:02 smithi195 ceph-mon[37040]: pgmap v730: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-06-06T08:11:02.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:02 smithi087 ceph-mon[32276]: pgmap v730: 129 pgs: 129 active+clean; 653 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-06-06T08:11:03.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.31 MB/sec execute 425 sec latency 8150.286 ms 2024-06-06T08:11:04.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.29 MB/sec execute 426 sec latency 9150.393 ms 2024-06-06T08:11:04.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:04 smithi181 ceph-mon[36790]: pgmap v731: 129 pgs: 129 active+clean; 654 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-06-06T08:11:04.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:04 smithi087 ceph-mon[32276]: pgmap v731: 129 pgs: 129 active+clean; 654 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-06-06T08:11:05.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:04 smithi195 ceph-mon[37040]: pgmap v731: 129 pgs: 129 active+clean; 654 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-06-06T08:11:05.190 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.26 MB/sec execute 427 sec latency 10150.519 ms 2024-06-06T08:11:06.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.23 MB/sec execute 428 sec latency 11150.552 ms 2024-06-06T08:11:06.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:06 smithi181 ceph-mon[36790]: pgmap v732: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-06T08:11:06.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:06 smithi087 ceph-mon[32276]: pgmap v732: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-06T08:11:07.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:06 smithi195 ceph-mon[37040]: pgmap v732: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-06T08:11:07.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.21 MB/sec execute 429 sec latency 12150.589 ms 2024-06-06T08:11:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.265+0000 7f69fa17e640 1 -- 172.21.15.87:0/538276201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4102340 msgr2=0x7f69f410e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.265+0000 7f69fa17e640 1 --2- 172.21.15.87:0/538276201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4102340 0x7f69f410e850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009900 tx=0x7f69ec02a400 comp rx=0 tx=0).stop 2024-06-06T08:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.268+0000 7f69fa17e640 1 -- 172.21.15.87:0/538276201 shutdown_connections 2024-06-06T08:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.268+0000 7f69fa17e640 1 --2- 172.21.15.87:0/538276201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4102340 0x7f69f410e850 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.268+0000 7f69fa17e640 1 --2- 172.21.15.87:0/538276201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69f4101980 0x7f69f4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.268+0000 7f69fa17e640 1 --2- 172.21.15.87:0/538276201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4107990 0x7f69f4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.268+0000 7f69fa17e640 1 -- 172.21.15.87:0/538276201 >> 172.21.15.87:0/538276201 conn(0x7f69f40fd450 msgr2=0x7f69f40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.272+0000 7f69fa17e640 1 -- 172.21.15.87:0/538276201 shutdown_connections 2024-06-06T08:11:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.273+0000 7f69fa17e640 1 -- 172.21.15.87:0/538276201 wait complete. 2024-06-06T08:11:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.274+0000 7f69fa17e640 1 Processor -- start 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 -- start start 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 0x7f69f4115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69f4102340 0x7f69f4116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 0x7f69f41110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4075890 con 0x7f69f4102340 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4075710 con 0x7f69f4107990 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.275+0000 7f69fa17e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4075a10 con 0x7f69f4101980 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.276+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 0x7f69f4115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.276+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 0x7f69f4115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49214/0 (socket says 172.21.15.87:49214) 2024-06-06T08:11:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.276+0000 7f69f37fe640 1 -- 172.21.15.87:0/3764305913 learned_addr learned my addr 172.21.15.87:0/3764305913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.276+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 0x7f69f41110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 -- 172.21.15.87:0/3764305913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 msgr2=0x7f69f4115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 0x7f69f4115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 -- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69f4102340 msgr2=0x7f69f4116500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69f4102340 0x7f69f4116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 -- 172.21.15.87:0/3764305913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f4111990 con 0x7f69f4107990 2024-06-06T08:11:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.277+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 0x7f69f41110d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02a920 tx=0x7f69ec003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.282+0000 7f69f0ff9640 1 -- 172.21.15.87:0/3764305913 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec03f070 con 0x7f69f4107990 2024-06-06T08:11:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.282+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f4111c20 con 0x7f69f4107990 2024-06-06T08:11:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.282+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f411beb0 con 0x7f69f4107990 2024-06-06T08:11:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.285+0000 7f69fcf6a740 1 Processor -- start 2024-06-06T08:11:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.285+0000 7f69f0ff9640 1 -- 172.21.15.87:0/3764305913 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec031c20 con 0x7f69f4107990 2024-06-06T08:11:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.285+0000 7f69f0ff9640 1 -- 172.21.15.87:0/3764305913 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec035030 con 0x7f69f4107990 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 -- start start 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecc8350 0x562bfecca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562bfec214f0 con 0x562bfecc8350 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562bfebbf870 con 0x562bfecbda90 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562bfec1d4b0 con 0x562bfecbd170 2024-06-06T08:11:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.287+0000 7f69f0ff9640 1 -- 172.21.15.87:0/3764305913 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69ec042070 con 0x7f69f4107990 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.288+0000 7f69f0ff9640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 0x7f69e00a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.288+0000 7f69f0ff9640 1 -- 172.21.15.87:0/3764305913 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7f69ec0f0f50 con 0x7f69f4107990 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.289+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.289+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49220/0 (socket says 172.21.15.87:49220) 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.289+0000 7f69f2ffd640 1 -- 172.21.15.87:0/108327743 learned_addr learned my addr 172.21.15.87:0/108327743 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.289+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecc8350 0x562bfecca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 -- 172.21.15.87:0/108327743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 msgr2=0x562bfecc7e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc7e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 -- 172.21.15.87:0/108327743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecc8350 msgr2=0x562bfecca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecc8350 0x562bfecca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 -- 172.21.15.87:0/108327743 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfebdb9b0 con 0x562bfecbd170 2024-06-06T08:11:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.290+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009fd0 tx=0x7f69dc009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc1f630f10f9faf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.291+0000 7f69e9ffb640 1 -- 172.21.15.87:0/108327743 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc012040 con 0x562bfecbd170 2024-06-06T08:11:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.291+0000 7f69e9ffb640 1 -- 172.21.15.87:0/108327743 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc007120 con 0x562bfecbd170 2024-06-06T08:11:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.291+0000 7f69e9ffb640 1 -- 172.21.15.87:0/108327743 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc004a30 con 0x562bfecbd170 2024-06-06T08:11:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.293+0000 7f69fcf6a740 1 -- 172.21.15.87:0/108327743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 msgr2=0x562bfecbd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.293+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009fd0 tx=0x7f69dc009e00 comp rx=0 tx=0).stop 2024-06-06T08:11:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.293+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 0x7f69e00a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.293+0000 7f69f3fff640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 0x7f69e00a5440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69ec03b920 tx=0x7f69ec005c90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.296+0000 7f69fcf6a740 1 -- 172.21.15.87:0/108327743 shutdown_connections 2024-06-06T08:11:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.296+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecc8350 0x562bfecca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.296+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.296+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/108327743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecbd550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.296+0000 7f69fcf6a740 1 -- 172.21.15.87:0/108327743 >> 172.21.15.87:0/108327743 conn(0x562bfecb9680 msgr2=0x562bfecb9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.300+0000 7f69fcf6a740 1 -- 172.21.15.87:0/108327743 shutdown_connections 2024-06-06T08:11:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.300+0000 7f69fcf6a740 1 -- 172.21.15.87:0/108327743 wait complete. 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 -- start start 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 Processor -- start 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbd170 0x562bfecc52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562bfeccaf80 con 0x562bfecbda90 2024-06-06T08:11:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562bfebaaae0 con 0x562bfecbd170 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562bfeccb100 con 0x562bfecc8350 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.301+0000 7f69fcf6a740 10 client.0 _ll_register_callbacks cb 0x562bfebe39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38016/0 (socket says 172.21.15.87:38016) 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 -- 172.21.15.87:0/2950962823 learned_addr learned my addr 172.21.15.87:0/2950962823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 -- 172.21.15.87:0/2950962823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 msgr2=0x562bfecc5d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:11:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc5d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 -- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbd170 msgr2=0x562bfecc52f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:11:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbd170 0x562bfecc52f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.302+0000 7f69f37fe640 1 -- 172.21.15.87:0/2950962823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfe935210 con 0x562bfecbda90 2024-06-06T08:11:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.303+0000 7f69f37fe640 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f69e40116c0 tx=0x7f69e4011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7212503c5893cba server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.303+0000 7f69d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.303+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e400fc70 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.303+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7f69e400fe10 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.304+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69dc007120 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.304+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f69e4011e90 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.304+0000 7f69fcf6a740 10 client.18543 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.304+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562bfea01620 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc007120 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f69e400f040 con 0x562bfecbda90 2024-06-06T08:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 10 client.18543 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 10 client.18543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 20 client.18543 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfecd8760 0x562bfecdac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 4 client.18543 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.305+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bfec34f10 con 0x562bfecd8760 2024-06-06T08:11:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.306+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfecd8760 0x562bfecdac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.307+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfecd8760 0x562bfecdac00 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d27463bc64bc35f server_cookie=36501c9ab1ce5e1 in_seq=0 out_seq=0 2024-06-06T08:11:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.309+0000 7f69d2ffd640 10 client.18543 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.310+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2950962823 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562bfec34f10 con 0x562bfecd8760 2024-06-06T08:11:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.310+0000 7f69d2ffd640 10 client.18543 handle_command_reply: tid=0 2024-06-06T08:11:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.310+0000 7f69fcf6a740 1 client.18543 shutdown 2024-06-06T08:11:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.318+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 msgr2=0x562bfecc5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.318+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f69e40116c0 tx=0x7f69e4011b80 comp rx=0 tx=0).stop 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 shutdown_connections 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfecd8760 0x562bfecdac00 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecc5830 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2950962823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbd170 0x562bfecc52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.320+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 >> 172.21.15.87:0/2950962823 conn(0x562bfecb9680 msgr2=0x562bfecbf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.321+0000 7f69d2ffd640 0 client.18543 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.327+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 shutdown_connections 2024-06-06T08:11:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.327+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2950962823 wait complete. 2024-06-06T08:11:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.328+0000 7f69fcf6a740 20 client.18543 trim_cache size 0 max 16384 2024-06-06T08:11:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.366+0000 7f69fcf6a740 1 Processor -- start 2024-06-06T08:11:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.366+0000 7f69fcf6a740 1 -- start start 2024-06-06T08:11:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.366+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 0x562bfecb8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.367+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.367+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.367+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4012c40 con 0x562bfecbd170 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.367+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4010510 con 0x562bfecbda90 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.367+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69d400e380 con 0x562bfecc8350 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.368+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 0x562bfecb8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.368+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.368+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 0x562bfecb8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38028/0 (socket says 172.21.15.87:38028) 2024-06-06T08:11:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.368+0000 7f69f2ffd640 1 -- 172.21.15.87:0/792942355 learned_addr learned my addr 172.21.15.87:0/792942355 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.368+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.378+0000 7f69f2ffd640 1 -- 172.21.15.87:0/792942355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 msgr2=0x562bfecc3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.378+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.378+0000 7f69f2ffd640 1 -- 172.21.15.87:0/792942355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 msgr2=0x562bfecb8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.378+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 0x562bfecb8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.378+0000 7f69f2ffd640 1 -- 172.21.15.87:0/792942355 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfecb3b70 con 0x562bfecc8350 2024-06-06T08:11:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.381+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc4140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69dc007040 tx=0x7f69dc009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e274b7fd34ade server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.382+0000 7f69e97fa640 1 -- 172.21.15.87:0/792942355 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc015cd0 con 0x562bfecc8350 2024-06-06T08:11:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.382+0000 7f69e97fa640 1 -- 172.21.15.87:0/792942355 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc015e70 con 0x562bfecc8350 2024-06-06T08:11:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.382+0000 7f69e97fa640 1 -- 172.21.15.87:0/792942355 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc007120 con 0x562bfecc8350 2024-06-06T08:11:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.383+0000 7f69fcf6a740 1 -- 172.21.15.87:0/792942355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 msgr2=0x562bfecc4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.383+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc4140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69dc007040 tx=0x7f69dc009e00 comp rx=0 tx=0).stop 2024-06-06T08:11:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.387+0000 7f69fcf6a740 1 -- 172.21.15.87:0/792942355 shutdown_connections 2024-06-06T08:11:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.387+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecc8350 0x562bfecc4140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.387+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecbda90 0x562bfecc3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.387+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/792942355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbd170 0x562bfecb8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.387+0000 7f69fcf6a740 1 -- 172.21.15.87:0/792942355 >> 172.21.15.87:0/792942355 conn(0x562bfecb9680 msgr2=0x562bfecc05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.390+0000 7f69fcf6a740 1 -- 172.21.15.87:0/792942355 shutdown_connections 2024-06-06T08:11:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.393+0000 7f69fcf6a740 1 -- 172.21.15.87:0/792942355 wait complete. 2024-06-06T08:11:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.393+0000 7f69fcf6a740 1 -- start start 2024-06-06T08:11:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.393+0000 7f69fcf6a740 1 Processor -- start 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecd03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecc8350 0x562bfecd08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69d400e9d0 con 0x562bfecbda90 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69d400fe40 con 0x562bfecc8350 2024-06-06T08:11:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69d400f0a0 con 0x562bfecbd170 2024-06-06T08:11:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.394+0000 7f69fcf6a740 10 client.0 _ll_register_callbacks cb 0x7f69e4009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.397+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecd03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.397+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecd03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38036/0 (socket says 172.21.15.87:38036) 2024-06-06T08:11:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.397+0000 7f69f37fe640 1 -- 172.21.15.87:0/2310284807 learned_addr learned my addr 172.21.15.87:0/2310284807 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.398+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.398+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecc8350 0x562bfecd08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.402+0000 7f69f2ffd640 1 -- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecc8350 msgr2=0x562bfecd08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.402+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecc8350 0x562bfecd08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.402+0000 7f69f2ffd640 1 -- 172.21.15.87:0/2310284807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 msgr2=0x562bfecd03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.402+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecd03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.402+0000 7f69f2ffd640 1 -- 172.21.15.87:0/2310284807 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfecd0ee0 con 0x562bfecbd170 2024-06-06T08:11:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.406+0000 7f69f2ffd640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecc2d00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x562bfeccac50 tx=0x7f69dc008d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf5ab7235c09cf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.407+0000 7f69d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:11:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.409+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc015cd0 con 0x562bfecbd170 2024-06-06T08:11:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.409+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7f69dc004030 con 0x562bfecbd170 2024-06-06T08:11:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.410+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69dc015e70 con 0x562bfecbd170 2024-06-06T08:11:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.417+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f69dc000e40 con 0x562bfecbd170 2024-06-06T08:11:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.422+0000 7f69fcf6a740 10 client.27616 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.422+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562bfed424e0 con 0x562bfecbd170 2024-06-06T08:11:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.426+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc015e70 con 0x562bfecbd170 2024-06-06T08:11:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f69dc004970 con 0x562bfecbd170 2024-06-06T08:11:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 10 client.27616 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 10 client.27616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 20 client.27616 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfed42ab0 0x562bfed44ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 4 client.27616 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:11:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.427+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562bfec2ec20 con 0x562bfed42ab0 2024-06-06T08:11:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.433+0000 7f69f37fe640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfed42ab0 0x562bfed44ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.434+0000 7f69f37fe640 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfed42ab0 0x562bfed44ed0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8124e8243777d66b server_cookie=6f0b490c5fa63d43 in_seq=0 out_seq=0 2024-06-06T08:11:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.434+0000 7f69d2ffd640 10 client.27616 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:07.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:07 smithi087 ceph-mon[32276]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T08:11:07.859 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:07 smithi087 ceph-mon[32276]: pgmap v734: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T08:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.930+0000 7f69d2ffd640 1 -- 172.21.15.87:0/2310284807 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562bfec2ec20 con 0x562bfed42ab0 2024-06-06T08:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.930+0000 7f69d2ffd640 10 client.27616 handle_command_reply: tid=0 2024-06-06T08:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.930+0000 7f69fcf6a740 1 client.27616 shutdown 2024-06-06T08:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 msgr2=0x562bfecc2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecc2d00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x562bfeccac50 tx=0x7f69dc008d50 comp rx=0 tx=0).stop 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 shutdown_connections 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562bfed42ab0 0x562bfed44ed0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfecc8350 0x562bfecd08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562bfecbda90 0x562bfecd03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 --2- 172.21.15.87:0/2310284807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562bfecbd170 0x562bfecc2d00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 >> 172.21.15.87:0/2310284807 conn(0x562bfecb9680 msgr2=0x562bfecc3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69d2ffd640 0 client.27616 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 shutdown_connections 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 1 -- 172.21.15.87:0/2310284807 wait complete. 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.931+0000 7f69fcf6a740 20 client.27616 trim_cache size 0 max 16384 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:11:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 msgr2=0x7f69e00a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 0x7f69e00a5440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69ec03b920 tx=0x7f69ec005c90 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 msgr2=0x7f69f41110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 0x7f69f41110d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02a920 tx=0x7f69ec003b90 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 shutdown_connections 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f69e00a2f80 0x7f69e00a5440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f4107990 0x7f69f41110d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69f4102340 0x7f69f4116500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 --2- 172.21.15.87:0/3764305913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f4101980 0x7f69f4115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.933+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 >> 172.21.15.87:0/3764305913 conn(0x7f69f40fd450 msgr2=0x7f69f40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.934+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 shutdown_connections 2024-06-06T08:11:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:07.934+0000 7f69fa17e640 1 -- 172.21.15.87:0/3764305913 wait complete. 2024-06-06T08:11:08.009 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:07 smithi195 ceph-mon[37040]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T08:11:08.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:07 smithi195 ceph-mon[37040]: pgmap v734: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T08:11:08.055 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:07 smithi181 ceph-mon[36790]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T08:11:08.056 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:07 smithi181 ceph-mon[36790]: pgmap v734: 129 pgs: 129 active+clean; 655 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2024-06-06T08:11:08.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.18 MB/sec execute 430 sec latency 13150.630 ms 2024-06-06T08:11:09.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.16 MB/sec execute 431 sec latency 14150.675 ms 2024-06-06T08:11:10.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:09 smithi087 ceph-mon[32276]: pgmap v735: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:10.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.13 MB/sec execute 432 sec latency 15150.717 ms 2024-06-06T08:11:10.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:09 smithi195 ceph-mon[37040]: pgmap v735: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:10.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:09 smithi181 ceph-mon[36790]: pgmap v735: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:11.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.10 MB/sec execute 433 sec latency 16150.760 ms 2024-06-06T08:11:11.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:11:11.726 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:12.066 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:11 smithi087 ceph-mon[32276]: pgmap v736: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:12.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.08 MB/sec execute 434 sec latency 17150.804 ms 2024-06-06T08:11:12.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:11 smithi195 ceph-mon[37040]: pgmap v736: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:12.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:11 smithi181 ceph-mon[36790]: pgmap v736: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 652 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-06-06T08:11:13.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.05 MB/sec execute 435 sec latency 18150.846 ms 2024-06-06T08:11:14.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:13 smithi087 ceph-mon[32276]: pgmap v737: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-06-06T08:11:14.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.03 MB/sec execute 436 sec latency 19150.888 ms 2024-06-06T08:11:14.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:13 smithi195 ceph-mon[37040]: pgmap v737: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-06-06T08:11:14.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:13 smithi181 ceph-mon[36790]: pgmap v737: 129 pgs: 11 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 86 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-06-06T08:11:15.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 11.00 MB/sec execute 437 sec latency 20150.922 ms 2024-06-06T08:11:16.094 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:15 smithi087 ceph-mon[32276]: pgmap v738: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-06T08:11:16.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 10.98 MB/sec execute 438 sec latency 21150.958 ms 2024-06-06T08:11:16.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:15 smithi195 ceph-mon[37040]: pgmap v738: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-06T08:11:16.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:15 smithi181 ceph-mon[36790]: pgmap v738: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 -- 172.21.15.87:0/3593207171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8110b80 msgr2=0x7ff1c8109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 --2- 172.21.15.87:0/3593207171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8110b80 0x7ff1c8109490 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009f90 tx=0x7ff1c000b330 comp rx=0 tx=0).stop 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 -- 172.21.15.87:0/3593207171 shutdown_connections 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 --2- 172.21.15.87:0/3593207171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c811bc70 0x7ff1c811e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 --2- 172.21.15.87:0/3593207171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8110b80 0x7ff1c8109490 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 --2- 172.21.15.87:0/3593207171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110ff0 0x7ff1c81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.391+0000 7ff1cc995700 1 -- 172.21.15.87:0/3593207171 >> 172.21.15.87:0/3593207171 conn(0x7ff1c806d470 msgr2=0x7ff1c806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:16.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.396+0000 7ff1cc995700 1 -- 172.21.15.87:0/3593207171 shutdown_connections 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.396+0000 7ff1cc995700 1 -- 172.21.15.87:0/3593207171 wait complete. 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.396+0000 7ff1cc995700 1 Processor -- start 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 -- start start 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 0x7ff1c811aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 0x7ff1c811b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c811bc70 0x7ff1c8114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8111a60 con 0x7ff1c811bc70 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81118e0 con 0x7ff1c8110b80 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1cc995700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8111be0 con 0x7ff1c8110ff0 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 0x7ff1c811b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.397+0000 7ff1c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 0x7ff1c811aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 0x7ff1c811b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:49260/0 (socket says 172.21.15.87:49260) 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c6ffd700 1 -- 172.21.15.87:0/1386733726 learned_addr learned my addr 172.21.15.87:0/1386733726 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:16.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c77fe700 1 -- 172.21.15.87:0/1386733726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 msgr2=0x7ff1c811b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c77fe700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 0x7ff1c811b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c77fe700 1 -- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c811bc70 msgr2=0x7ff1c8114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c77fe700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c811bc70 0x7ff1c8114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.398+0000 7ff1c77fe700 1 -- 172.21.15.87:0/1386733726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c8115750 con 0x7ff1c8110b80 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.399+0000 7ff1c77fe700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 0x7ff1c811aef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009f90 tx=0x7ff1c000cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.399+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0007ac0 con 0x7ff1c8110b80 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.399+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0022070 con 0x7ff1c8110b80 2024-06-06T08:11:16.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.400+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c001b6c0 con 0x7ff1c8110b80 2024-06-06T08:11:16.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.401+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c81159e0 con 0x7ff1c8110b80 2024-06-06T08:11:16.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.401+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c807a6f0 con 0x7ff1c8110b80 2024-06-06T08:11:16.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.406+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff1c0007c60 con 0x7ff1c8110b80 2024-06-06T08:11:16.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.406+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c806e8b0 con 0x7ff1c8110b80 2024-06-06T08:11:16.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.407+0000 7ff1c4ff9700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 0x7ff1b00a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:16.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.407+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7ff1c000f070 con 0x7ff1c8110b80 2024-06-06T08:11:16.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.410+0000 7ff1c6ffd700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 0x7ff1b00a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:16.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.413+0000 7ff1c6ffd700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 0x7ff1b00a5390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8116730 tx=0x7ff1b800a400 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:16.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.416+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1c009c7f0 con 0x7ff1c8110b80 2024-06-06T08:11:16.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.659+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1c81164a0 con 0x7ff1c8110b80 2024-06-06T08:11:16.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.660+0000 7ff1c4ff9700 1 -- 172.21.15.87:0/1386733726 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7ff1c009c0f0 con 0x7ff1c8110b80 2024-06-06T08:11:16.661 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:11:16.661 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:11:16.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 msgr2=0x7ff1b00a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:16.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 0x7ff1b00a5390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8116730 tx=0x7ff1b800a400 comp rx=0 tx=0).stop 2024-06-06T08:11:16.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 msgr2=0x7ff1c811aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:16.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 0x7ff1c811aef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009f90 tx=0x7ff1c000cfa0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 shutdown_connections 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c8110b80 0x7ff1c811aef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff1b00a2ed0 0x7ff1b00a5390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c811bc70 0x7ff1c8114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 --2- 172.21.15.87:0/1386733726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1c8110ff0 0x7ff1c811b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 >> 172.21.15.87:0/1386733726 conn(0x7ff1c806d470 msgr2=0x7ff1c8072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 shutdown_connections 2024-06-06T08:11:16.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:16.662+0000 7ff1cc995700 1 -- 172.21.15.87:0/1386733726 wait complete. 2024-06-06T08:11:16.665 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:11:16.999 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:16 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1386733726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:17.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 10.95 MB/sec execute 439 sec latency 22151.086 ms 2024-06-06T08:11:17.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:16 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1386733726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:17.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:16 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1386733726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:17.360 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id 9ebcf3dc --release --await' 2024-06-06T08:11:17.361 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id 9ebcf3dc --release --await 2024-06-06T08:11:17.598 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:18.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:17 smithi087 ceph-mon[32276]: pgmap v739: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-06-06T08:11:18.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 10.93 MB/sec execute 440 sec latency 23151.188 ms 2024-06-06T08:11:18.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:17 smithi195 ceph-mon[37040]: pgmap v739: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-06-06T08:11:18.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:17 smithi181 ceph-mon[36790]: pgmap v739: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 625 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-06-06T08:11:19.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 114976 10.90 MB/sec execute 441 sec latency 24151.254 ms 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 -- 172.21.15.87:0/1990564287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb498101620 msgr2=0x7fb498101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1990564287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb498101620 0x7fb498101a00 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009990 tx=0x7fb48c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 -- 172.21.15.87:0/1990564287 shutdown_connections 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1990564287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4981066e0 0x7fb49810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1990564287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb498101620 0x7fb498101a00 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1990564287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 0x7fb498106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 -- 172.21.15.87:0/1990564287 >> 172.21.15.87:0/1990564287 conn(0x7fb498078330 msgr2=0x7fb498078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 -- 172.21.15.87:0/1990564287 shutdown_connections 2024-06-06T08:11:19.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.428+0000 7fb49f23b700 1 -- 172.21.15.87:0/1990564287 wait complete. 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 Processor -- start 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 -- start start 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb498101620 0x7fb498198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 0x7fb498199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981117a0 con 0x7fb4981066e0 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb498111620 con 0x7fb498101620 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49f23b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb498111920 con 0x7fb498101fd0 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49d7d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49d7d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51232/0 (socket says 172.21.15.87:51232) 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb49d7d8700 1 -- 172.21.15.87:0/1825548118 learned_addr learned my addr 172.21.15.87:0/1825548118 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:19.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.429+0000 7fb497fff700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 0x7fb498199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb498101620 0x7fb498198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 -- 172.21.15.87:0/1825548118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 msgr2=0x7fb498199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 0x7fb498199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 -- 172.21.15.87:0/1825548118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb498101620 msgr2=0x7fb498198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb498101620 0x7fb498198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 -- 172.21.15.87:0/1825548118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49819ddd0 con 0x7fb4981066e0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb49000b970 tx=0x7fb49000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb495ffb700 1 -- 172.21.15.87:0/1825548118 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490019070 con 0x7fb4981066e0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49819e0c0 con 0x7fb4981066e0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb495ffb700 1 -- 172.21.15.87:0/1825548118 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49000ce10 con 0x7fb4981066e0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb495ffb700 1 -- 172.21.15.87:0/1825548118 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490007370 con 0x7fb4981066e0 2024-06-06T08:11:19.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.430+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4981a5940 con 0x7fb4981066e0 2024-06-06T08:11:19.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.432+0000 7fb495ffb700 1 -- 172.21.15.87:0/1825548118 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb49001c070 con 0x7fb4981066e0 2024-06-06T08:11:19.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.433+0000 7fb495ffb700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 0x7fb4880a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.433+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 0x7fb4880a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.433+0000 7fb495ffb700 1 -- 172.21.15.87:0/1825548118 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7fb4900ceb90 con 0x7fb4981066e0 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.433+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 0x7fb4880a5250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009750 tx=0x7fb48c00b540 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 Processor -- start 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 -- start start 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c8352c3060 con 0x55c8353d6500 2024-06-06T08:11:19.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c835321950 con 0x55c8353d0c50 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c835289560 con 0x55c8353d6e20 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51234/0 (socket says 172.21.15.87:51234) 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 -- 172.21.15.87:0/1431545639 learned_addr learned my addr 172.21.15.87:0/1431545639 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb497fff700 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 -- 172.21.15.87:0/1431545639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 msgr2=0x55c8353d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 -- 172.21.15.87:0/1431545639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 msgr2=0x55c8353d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.434+0000 7fb49cfd7700 1 -- 172.21.15.87:0/1431545639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8352709d0 con 0x55c8353d6500 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00ba60 tx=0x7fb48c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9fe2d8d74d5f36 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb482ffd700 1 -- 172.21.15.87:0/1431545639 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00be20 con 0x55c8353d6500 2024-06-06T08:11:19.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb482ffd700 1 -- 172.21.15.87:0/1431545639 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c00f940 con 0x55c8353d6500 2024-06-06T08:11:19.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb482ffd700 1 -- 172.21.15.87:0/1431545639 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c024980 con 0x55c8353d6500 2024-06-06T08:11:19.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/1431545639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 msgr2=0x55c8353d68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00ba60 tx=0x7fb48c01f000 comp rx=0 tx=0).stop 2024-06-06T08:11:19.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/1431545639 shutdown_connections 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d68e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/1431545639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.435+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/1431545639 >> 172.21.15.87:0/1431545639 conn(0x55c8352fd180 msgr2=0x55c8353cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/1431545639 shutdown_connections 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/1431545639 wait complete. 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- start start 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 Processor -- start 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 0x55c8353da3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55c8352c20c0 con 0x55c8353d6500 2024-06-06T08:11:19.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c83531d910 con 0x55c8353d6e20 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c8353dd740 con 0x55c8353d0c50 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.436+0000 7fb4a9abeb80 10 client.0 _ll_register_callbacks cb 0x55c8352b76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51242/0 (socket says 172.21.15.87:51242) 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 -- 172.21.15.87:0/4280024737 learned_addr learned my addr 172.21.15.87:0/4280024737 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 0x55c8353da3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 -- 172.21.15.87:0/4280024737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 msgr2=0x55c8353dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 -- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 msgr2=0x55c8353da3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 0x55c8353da3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 -- 172.21.15.87:0/4280024737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8352e7850 con 0x55c8353d6500 2024-06-06T08:11:19.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 0x55c8353da3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb497fff700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb4840115e0 tx=0x7fb4840118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba8e62a4c9deae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb477fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484014650 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7fb4840147f0 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c00f940 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb484026050 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb4a9abeb80 10 client.18564 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.437+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c8353e7f20 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c00f940 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fb48401dc80 con 0x55c8353d6500 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 10 client.18564 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 10 client.18564 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 20 client.18564 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:19.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c8353ea560 0x55c8353ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 4 client.18564 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83532ee50 con 0x55c8353ea560 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.438+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c8353ea560 0x55c8353ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c8353ea560 0x55c8353ec950 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a79170bddcc8cbf server_cookie=e347d091f28079d3 in_seq=0 out_seq=0 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb477fff700 10 client.18564 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb477fff700 1 -- 172.21.15.87:0/4280024737 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c83532ee50 con 0x55c8353ea560 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb477fff700 10 client.18564 handle_command_reply: tid=0 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 client.18564 shutdown 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 msgr2=0x55c8353d9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb4840115e0 tx=0x7fb4840118f0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 shutdown_connections 2024-06-06T08:11:19.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c8353ea560 0x55c8353ec950 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6e20 0x55c8353da3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb477fff700 0 client.18564 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.439+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/4280024737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353d9e90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.440+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 >> 172.21.15.87:0/4280024737 conn(0x55c8352fd180 msgr2=0x55c8353cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.440+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 shutdown_connections 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.440+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/4280024737 wait complete. 2024-06-06T08:11:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.440+0000 7fb4a9abeb80 20 client.18564 trim_cache size 0 max 16384 2024-06-06T08:11:19.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.459+0000 7fb4a9abeb80 1 Processor -- start 2024-06-06T08:11:19.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.459+0000 7fb4a9abeb80 1 -- start start 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6500 0x55c8353cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981b1310 con 0x55c8353d6e20 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981aeba0 con 0x55c8353d6500 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981aca10 con 0x55c8353d0c50 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51256/0 (socket says 172.21.15.87:51256) 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 -- 172.21.15.87:0/722270660 learned_addr learned my addr 172.21.15.87:0/722270660 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:19.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb497fff700 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6500 0x55c8353cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 -- 172.21.15.87:0/722270660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 msgr2=0x55c8353dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 -- 172.21.15.87:0/722270660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6500 msgr2=0x55c8353cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6500 0x55c8353cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 -- 172.21.15.87:0/722270660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8353d8420 con 0x55c8353d6e20 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bfd0 tx=0x7fb49009a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1251f4f3bc09fefa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4827fc700 1 -- 172.21.15.87:0/722270660 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49009f0f0 con 0x55c8353d6e20 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4827fc700 1 -- 172.21.15.87:0/722270660 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4900a39b0 con 0x55c8353d6e20 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.460+0000 7fb4827fc700 1 -- 172.21.15.87:0/722270660 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900a3780 con 0x55c8353d6e20 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/722270660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 msgr2=0x55c8353d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bfd0 tx=0x7fb49009a720 comp rx=0 tx=0).stop 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/722270660 shutdown_connections 2024-06-06T08:11:19.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6e20 0x55c8353d0350 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d0c50 0x55c8353dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/722270660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d6500 0x55c8353cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/722270660 >> 172.21.15.87:0/722270660 conn(0x55c8352fd180 msgr2=0x55c8353cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/722270660 shutdown_connections 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/722270660 wait complete. 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.461+0000 7fb4a9abeb80 1 -- start start 2024-06-06T08:11:19.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 Processor -- start 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981ad060 con 0x55c8353d6500 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981ae4d0 con 0x55c8353d0c50 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981ad730 con 0x55c8353d6e20 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb4a9abeb80 10 client.0 _ll_register_callbacks cb 0x55c8352b76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:19.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb497fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51270/0 (socket says 172.21.15.87:51270) 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb497fff700 1 -- 172.21.15.87:0/537104254 learned_addr learned my addr 172.21.15.87:0/537104254 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb49d7d8700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.462+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 -- 172.21.15.87:0/537104254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 msgr2=0x55c8353dff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353dff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 -- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 msgr2=0x55c8353e30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353e30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 -- 172.21.15.87:0/537104254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8353e0750 con 0x55c8353d6500 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb497fff700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x55c8353d1780 tx=0x7fb4840138c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2ec060f1f8a0f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb477fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48402f020 con 0x55c8353d6500 2024-06-06T08:11:19.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9671+0+0 (secure 0 0 0) 0x7fb484013c90 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4900a39b0 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb48403a050 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb4a9abeb80 10 client.18570 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c835456190 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.463+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4900a39b0 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fb48402a430 con 0x55c8353d6500 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 10 client.18570 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 10 client.18570 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 20 client.18570 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c835456890 0x55c835458cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 4 client.18570 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9ebcf3dc", "release": true, "await": true}] 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9ebcf3dc", "release": true, "await": true}) -- 0x55c8353293d0 con 0x55c835456890 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c835456890 0x55c835458cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:19.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb477fff700 10 client.18570 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:19.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.464+0000 7fb49cfd7700 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c835456890 0x55c835458cb0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729aff60717df132 server_cookie=cfb24400c25790d3 in_seq=0 out_seq=0 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb477fff700 1 -- 172.21.15.87:0/537104254 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c8353293d0 con 0x55c835456890 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb477fff700 10 client.18570 handle_command_reply: tid=0 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 client.18570 shutdown 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 msgr2=0x55c8353df9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x55c8353d1780 tx=0x7fb4840138c0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 shutdown_connections 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55c835456890 0x55c835458cb0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8353d0c50 0x55c8353e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8353d6e20 0x55c8353dff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb477fff700 0 client.18570 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 --2- 172.21.15.87:0/537104254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c8353d6500 0x55c8353df9d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 >> 172.21.15.87:0/537104254 conn(0x55c8352fd180 msgr2=0x55c8353cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 shutdown_connections 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 1 -- 172.21.15.87:0/537104254 wait complete. 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.537+0000 7fb4a9abeb80 20 client.18570 trim_cache size 0 max 16384 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:11:19.538 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 72, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "9ebcf3dc": { 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 72, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 141.8, 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:11:19.539 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:19.540 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 msgr2=0x7fb4880a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 0x7fb4880a5250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009750 tx=0x7fb48c00b540 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 msgr2=0x7fb49819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb49000b970 tx=0x7fb49000bc80 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 shutdown_connections 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb498101620 0x7fb498198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fb4880a2d90 0x7fb4880a5250 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4981066e0 0x7fb49819d690 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 --2- 172.21.15.87:0/1825548118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb498101fd0 0x7fb498199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:19.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 >> 172.21.15.87:0/1825548118 conn(0x7fb498078330 msgr2=0x7fb498107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:19.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 shutdown_connections 2024-06-06T08:11:19.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:19.538+0000 7fb49f23b700 1 -- 172.21.15.87:0/1825548118 wait complete. 2024-06-06T08:11:20.191 INFO:tasks.workunit.client.0.smithi087.stdout: 10 115164 10.90 MB/sec execute 442 sec latency 24519.708 ms 2024-06-06T08:11:20.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:19 smithi195 ceph-mon[37040]: pgmap v740: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 3.6 MiB/s wr, 227 op/s 2024-06-06T08:11:20.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:19 smithi181 ceph-mon[36790]: pgmap v740: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 3.6 MiB/s wr, 227 op/s 2024-06-06T08:11:20.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:19 smithi087 ceph-mon[32276]: pgmap v740: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 3.6 MiB/s wr, 227 op/s 2024-06-06T08:11:20.367 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9ebcf3dc', total seconds elapsed: 30.9 2024-06-06T08:11:20.367 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-06-06T08:11:20.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:11:20.872 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 -- 172.21.15.87:0/1361867209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f8114af0 msgr2=0x7f96f8116ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/1361867209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f8114af0 0x7f96f8116ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009cd0 tx=0x7f96f002a9c0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 -- 172.21.15.87:0/1361867209 shutdown_connections 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/1361867209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f8114af0 0x7f96f8116ee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/1361867209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f81092c0 0x7f96f81145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/1361867209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 0x7f96f8108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.072+0000 7f96ff8bd640 1 -- 172.21.15.87:0/1361867209 >> 172.21.15.87:0/1361867209 conn(0x7f96f806ca70 msgr2=0x7f96f806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.073+0000 7f96ff8bd640 1 -- 172.21.15.87:0/1361867209 shutdown_connections 2024-06-06T08:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.073+0000 7f96ff8bd640 1 -- 172.21.15.87:0/1361867209 wait complete. 2024-06-06T08:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 Processor -- start 2024-06-06T08:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 -- start start 2024-06-06T08:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 0x7f96f8114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f81092c0 0x7f96f81145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8118440 con 0x7f96f8114af0 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81182c0 con 0x7f96f81089a0 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96ff8bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81185c0 con 0x7f96f81092c0 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96fd659640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 0x7f96f8114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96fde5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96fde5a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51274/0 (socket says 172.21.15.87:51274) 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.074+0000 7f96fde5a640 1 -- 172.21.15.87:0/4244457824 learned_addr learned my addr 172.21.15.87:0/4244457824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 -- 172.21.15.87:0/4244457824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f81092c0 msgr2=0x7f96f81145a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f81092c0 0x7f96f81145a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 -- 172.21.15.87:0/4244457824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 msgr2=0x7f96f8114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 0x7f96f8114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 -- 172.21.15.87:0/4244457824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f810ebc0 con 0x7f96f8114af0 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.075+0000 7f96fde5a640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009bf0 tx=0x7f96f0030920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.076+0000 7f96ee7fc640 1 -- 172.21.15.87:0/4244457824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0047070 con 0x7f96f8114af0 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.076+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f810ee50 con 0x7f96f8114af0 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.076+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f8126650 con 0x7f96f8114af0 2024-06-06T08:11:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.077+0000 7f96ee7fc640 1 -- 172.21.15.87:0/4244457824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0004040 con 0x7f96f8114af0 2024-06-06T08:11:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.077+0000 7f96ee7fc640 1 -- 172.21.15.87:0/4244457824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0034070 con 0x7f96f8114af0 2024-06-06T08:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.081+0000 7f96ee7fc640 1 -- 172.21.15.87:0/4244457824 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f96f004a070 con 0x7f96f8114af0 2024-06-06T08:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.082+0000 7f96ee7fc640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 0x7f96e40a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f96ee7fc640 1 -- 172.21.15.87:0/4244457824 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f96f00f0590 con 0x7f96f8114af0 2024-06-06T08:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 Processor -- start 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 -- start start 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279170 0x564aaf279550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf286740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564aaf1dd4f0 con 0x564aaf284350 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564aaf17b870 con 0x564aaf279170 2024-06-06T08:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.083+0000 7f97025d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564aaf1d94b0 con 0x564aaf279a90 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47338/0 (socket says 172.21.15.87:47338) 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 -- 172.21.15.87:0/3552654028 learned_addr learned my addr 172.21.15.87:0/3552654028 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 -- 172.21.15.87:0/3552654028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279170 msgr2=0x564aaf279550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279170 0x564aaf279550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 -- 172.21.15.87:0/3552654028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 msgr2=0x564aaf286740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf286740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.084+0000 7f96fce58640 1 -- 172.21.15.87:0/3552654028 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aaf1979b0 con 0x564aaf279a90 2024-06-06T08:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.085+0000 7f96fd659640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 0x7f96e40a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.085+0000 7f96fce58640 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96e80096f0 tx=0x7f96e8009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f67c5506df905b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.085+0000 7f96fd659640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 0x7f96e40a5490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009950 tx=0x7f96f4009110 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.090+0000 7f96e37fe640 1 -- 172.21.15.87:0/3552654028 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8015020 con 0x564aaf279a90 2024-06-06T08:11:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.090+0000 7f96e37fe640 1 -- 172.21.15.87:0/3552654028 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e800bdb0 con 0x564aaf279a90 2024-06-06T08:11:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f96e37fe640 1 -- 172.21.15.87:0/3552654028 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e80103c0 con 0x564aaf279a90 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 -- 172.21.15.87:0/3552654028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 msgr2=0x564aaf283e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96e80096f0 tx=0x7f96e8009bb0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 -- 172.21.15.87:0/3552654028 shutdown_connections 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf286740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279a90 0x564aaf283e10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 --2- 172.21.15.87:0/3552654028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279170 0x564aaf279550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.091+0000 7f97025d5740 1 -- 172.21.15.87:0/3552654028 >> 172.21.15.87:0/3552654028 conn(0x564aaf275680 msgr2=0x564aaf275a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.092+0000 7f97025d5740 1 -- 172.21.15.87:0/3552654028 shutdown_connections 2024-06-06T08:11:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.092+0000 7f97025d5740 1 -- 172.21.15.87:0/3552654028 wait complete. 2024-06-06T08:11:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.092+0000 7f97025d5740 1 -- start start 2024-06-06T08:11:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 Processor -- start 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf2812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf281830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf281d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564aaf286f80 con 0x564aaf279a90 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564aaf166ae0 con 0x564aaf284350 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564aaf287100 con 0x564aaf279170 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f96fce58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf281830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f96fce58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf281830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51290/0 (socket says 172.21.15.87:51290) 2024-06-06T08:11:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f96fce58640 1 -- 172.21.15.87:0/826217374 learned_addr learned my addr 172.21.15.87:0/826217374 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f96fd659640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf2812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.093+0000 7f97025d5740 10 client.0 _ll_register_callbacks cb 0x564aaf19f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fde5a640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf281d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 -- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 msgr2=0x564aaf281d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf281d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 -- 172.21.15.87:0/826217374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 msgr2=0x564aaf281830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf281830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 -- 172.21.15.87:0/826217374 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aaeef1210 con 0x564aaf279170 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96fd659640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf2812f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96f4015730 tx=0x7f96f40047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f492462d8c37ef3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.094+0000 7f96e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.095+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4023020 con 0x564aaf279170 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.095+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f96f4013510 con 0x564aaf279170 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.095+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e800bdb0 con 0x564aaf279170 2024-06-06T08:11:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.096+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f96f401b040 con 0x564aaf279170 2024-06-06T08:11:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.097+0000 7f97025d5740 10 client.27646 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.097+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564aaefbd620 con 0x564aaf279170 2024-06-06T08:11:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.098+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4013bf0 con 0x564aaf279170 2024-06-06T08:11:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f96f4030020 con 0x564aaf279170 2024-06-06T08:11:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 10 client.27646 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 10 client.27646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 20 client.27646 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf296850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 4 client.27646 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.099+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f96e800bdb0 con 0x564aaf294440 2024-06-06T08:11:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.100+0000 7f96fce58640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf296850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.101+0000 7f96fce58640 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf296850 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa265832ce5ba9d server_cookie=11096d6fbfba4581 in_seq=0 out_seq=0 2024-06-06T08:11:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.101+0000 7f96e0ff9640 10 client.27646 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.105+0000 7f96e0ff9640 1 -- 172.21.15.87:0/826217374 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f96e800bdb0 con 0x564aaf294440 2024-06-06T08:11:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.105+0000 7f96e0ff9640 10 client.27646 handle_command_reply: tid=0 2024-06-06T08:11:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.107+0000 7f97025d5740 1 client.27646 shutdown 2024-06-06T08:11:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.108+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 msgr2=0x564aaf2812f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.108+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf2812f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96f4015730 tx=0x7f96f40047d0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 shutdown_connections 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf296850 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f96e0ff9640 0 client.27646 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf281d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf281830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 --2- 172.21.15.87:0/826217374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf2812f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.111+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 >> 172.21.15.87:0/826217374 conn(0x564aaf275680 msgr2=0x564aaf27b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.113+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 shutdown_connections 2024-06-06T08:11:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.114+0000 7f97025d5740 1 -- 172.21.15.87:0/826217374 wait complete. 2024-06-06T08:11:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.117+0000 7f97025d5740 20 client.27646 trim_cache size 0 max 16384 2024-06-06T08:11:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 Processor -- start 2024-06-06T08:11:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 -- start start 2024-06-06T08:11:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279a90 0x564aaf283230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf29c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc012ad0 con 0x564aaf284350 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f96fd659640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f96fd659640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47352/0 (socket says 172.21.15.87:47352) 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f96fd659640 1 -- 172.21.15.87:0/1659458813 learned_addr learned my addr 172.21.15.87:0/1659458813 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f96fde5a640 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf29c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0103a0 con 0x564aaf279a90 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.141+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc00e210 con 0x564aaf279170 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 -- 172.21.15.87:0/1659458813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279a90 msgr2=0x564aaf283230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279a90 0x564aaf283230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 -- 172.21.15.87:0/1659458813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 msgr2=0x564aaf29c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf29c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 -- 172.21.15.87:0/1659458813 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aaf27f840 con 0x564aaf279170 2024-06-06T08:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.142+0000 7f96fd659640 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96f4004700 tx=0x7f96f40284f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13b98b85a60901a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.145+0000 7f96e2ffd640 1 -- 172.21.15.87:0/1659458813 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4028b80 con 0x564aaf279170 2024-06-06T08:11:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.145+0000 7f96e2ffd640 1 -- 172.21.15.87:0/1659458813 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4028d20 con 0x564aaf279170 2024-06-06T08:11:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.145+0000 7f96e2ffd640 1 -- 172.21.15.87:0/1659458813 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4031870 con 0x564aaf279170 2024-06-06T08:11:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.146+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 msgr2=0x564aaf281b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.146+0000 7f97025d5740 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96f4004700 tx=0x7f96f40284f0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.147+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 shutdown_connections 2024-06-06T08:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.147+0000 7f97025d5740 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf284350 0x564aaf29c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.147+0000 7f97025d5740 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf279a90 0x564aaf283230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.147+0000 7f97025d5740 1 --2- 172.21.15.87:0/1659458813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf281b90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.147+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 >> 172.21.15.87:0/1659458813 conn(0x564aaf275680 msgr2=0x564aaf282b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.150+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 shutdown_connections 2024-06-06T08:11:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.151+0000 7f97025d5740 1 -- 172.21.15.87:0/1659458813 wait complete. 2024-06-06T08:11:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.152+0000 7f97025d5740 1 -- start start 2024-06-06T08:11:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.152+0000 7f97025d5740 1 Processor -- start 2024-06-06T08:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.152+0000 7f97025d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf287b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.152+0000 7f97025d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f97025d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf2f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fd659640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf287b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fd659640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf287b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47358/0 (socket says 172.21.15.87:47358) 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fce58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fce58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51298/0 (socket says 172.21.15.87:51298) 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fd659640 1 -- 172.21.15.87:0/3291277378 learned_addr learned my addr 172.21.15.87:0/3291277378 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc00e860 con 0x564aaf279a90 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc00fcd0 con 0x564aaf284350 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc00ef30 con 0x564aaf279170 2024-06-06T08:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f97025d5740 10 client.0 _ll_register_callbacks cb 0x7f96f4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.153+0000 7f96fde5a640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf2f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 -- 172.21.15.87:0/3291277378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 msgr2=0x564aaf287b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf287b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 -- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 msgr2=0x564aaf2f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf2f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aaf2f76b0 con 0x564aaf279a90 2024-06-06T08:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fde5a640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf2f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96fce58640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006df0 tx=0x7f96e8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced5ccd531a19938 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.154+0000 7f96e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8015050 con 0x564aaf279a90 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f96e800bad0 con 0x564aaf279a90 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f4028d20 con 0x564aaf279a90 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f96e800bcd0 con 0x564aaf279a90 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f97025d5740 10 client.18582 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.156+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564aaf2fea50 con 0x564aaf279a90 2024-06-06T08:11:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e8004520 con 0x564aaf279a90 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f96e802c070 con 0x564aaf279a90 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f97025d5740 10 client.18582 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f97025d5740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f97025d5740 20 client.18582 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.157+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf300f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.158+0000 7f97025d5740 4 client.18582 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.158+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f96f4028d20 con 0x564aaf294440 2024-06-06T08:11:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.158+0000 7f96fd659640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf300f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.159+0000 7f96fd659640 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf300f40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea60c2dbc3b2687f server_cookie=fed05ee548ceb65a in_seq=0 out_seq=0 2024-06-06T08:11:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.159+0000 7f96e0ff9640 10 client.18582 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:21.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 115329 10.89 MB/sec execute 443 sec latency 434.683 ms 2024-06-06T08:11:21.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:20 smithi195 ceph-mon[37040]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T08:11:21.300 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:20 smithi087 ceph-mon[32276]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T08:11:21.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:20 smithi181 ceph-mon[36790]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T08:11:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.597+0000 7f96e0ff9640 1 -- 172.21.15.87:0/3291277378 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f96f4028d20 con 0x564aaf294440 2024-06-06T08:11:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.597+0000 7f96e0ff9640 10 client.18582 handle_command_reply: tid=0 2024-06-06T08:11:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.597+0000 7f97025d5740 1 client.18582 shutdown 2024-06-06T08:11:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.597+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 msgr2=0x564aaf2880b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.597+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006df0 tx=0x7f96e8011b00 comp rx=0 tx=0).stop 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 shutdown_connections 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564aaf294440 0x564aaf300f40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564aaf284350 0x564aaf2f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f96e0ff9640 0 client.18582 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564aaf279a90 0x564aaf2880b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 --2- 172.21.15.87:0/3291277378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564aaf279170 0x564aaf287b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 >> 172.21.15.87:0/3291277378 conn(0x564aaf275680 msgr2=0x564aaf2833b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 shutdown_connections 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 1 -- 172.21.15.87:0/3291277378 wait complete. 2024-06-06T08:11:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.598+0000 7f97025d5740 20 client.18582 trim_cache size 0 max 16384 2024-06-06T08:11:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:11:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:11:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:11:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 msgr2=0x7f96e40a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 0x7f96e40a5490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009950 tx=0x7f96f4009110 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 msgr2=0x7f96f810e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009bf0 tx=0x7f96f0030920 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 shutdown_connections 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f96e40a2fd0 0x7f96e40a5490 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009950 tx=0x7f96f4009110 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96f8114af0 0x7f96f810e300 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96f81092c0 0x7f96f81145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 --2- 172.21.15.87:0/4244457824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96f81089a0 0x7f96f8114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.599+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 >> 172.21.15.87:0/4244457824 conn(0x7f96f806ca70 msgr2=0x7f96f8070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.600+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 shutdown_connections 2024-06-06T08:11:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:21.600+0000 7f96ff8bd640 1 -- 172.21.15.87:0/4244457824 wait complete. 2024-06-06T08:11:22.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 115524 10.89 MB/sec execute 444 sec latency 625.505 ms 2024-06-06T08:11:22.225 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:21 smithi087 ceph-mon[32276]: pgmap v742: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.2 MiB/s wr, 189 op/s 2024-06-06T08:11:22.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:21 smithi195 ceph-mon[37040]: pgmap v742: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.2 MiB/s wr, 189 op/s 2024-06-06T08:11:22.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:21 smithi181 ceph-mon[36790]: pgmap v742: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.2 MiB/s wr, 189 op/s 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.658+0000 7f20b9b11700 1 -- 172.21.15.87:0/2461282568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 msgr2=0x7f20b4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.658+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2461282568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4102940 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009990 tx=0x7f20a8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 -- 172.21.15.87:0/2461282568 shutdown_connections 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2461282568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 0x7f20b410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2461282568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2461282568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4102940 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 -- 172.21.15.87:0/2461282568 >> 172.21.15.87:0/2461282568 conn(0x7f20b40795f0 msgr2=0x7f20b4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:22.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 -- 172.21.15.87:0/2461282568 shutdown_connections 2024-06-06T08:11:22.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.659+0000 7f20b9b11700 1 -- 172.21.15.87:0/2461282568 wait complete. 2024-06-06T08:11:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.660+0000 7f20b9b11700 1 Processor -- start 2024-06-06T08:11:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.660+0000 7f20b9b11700 1 -- start start 2024-06-06T08:11:22.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 0x7f20b4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41127f0 con 0x7f20b41024e0 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4112670 con 0x7f20b4102e80 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b9b11700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4112970 con 0x7f20b4108650 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:47378/0 (socket says 172.21.15.87:47378) 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51314/0 (socket says 172.21.15.87:51314) 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 -- 172.21.15.87:0/2530927722 learned_addr learned my addr 172.21.15.87:0/2530927722 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b2ffd700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 0x7f20b4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 -- 172.21.15.87:0/2530927722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 msgr2=0x7f20b4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 0x7f20b4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 -- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 msgr2=0x7f20b4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 -- 172.21.15.87:0/2530927722 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41a71b0 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b37fe700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.661+0000 7f20b3fff700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00efc0 tx=0x7f20ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.662+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0092e0 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.662+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41a7440 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.662+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ac004bb0 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.662+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a7980 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.662+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004020 con 0x7f20b4108650 2024-06-06T08:11:22.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.663+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b4069180 con 0x7f20b4108650 2024-06-06T08:11:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.664+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f20ac01b030 con 0x7f20b4108650 2024-06-06T08:11:22.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.665+0000 7f20b0ff9700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 0x7f209c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:22.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.665+0000 7f20b37fe700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 0x7f209c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:22.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.665+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f20ac0cfb60 con 0x7f20b4108650 2024-06-06T08:11:22.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.665+0000 7f20b37fe700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 0x7f209c0a52d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fd0 tx=0x7f20a4009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:22.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.669+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20ac022020 con 0x7f20b4108650 2024-06-06T08:11:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.990+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20b402dca0 con 0x7f20b4108650 2024-06-06T08:11:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.991+0000 7f20b0ff9700 1 -- 172.21.15.87:0/2530927722 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7f20ac010040 con 0x7f20b4108650 2024-06-06T08:11:22.996 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:11:22.996 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:11:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.997+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 msgr2=0x7f209c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.997+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 0x7f209c0a52d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fd0 tx=0x7f20a4009260 comp rx=0 tx=0).stop 2024-06-06T08:11:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.997+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 msgr2=0x7f20b4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:22.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.997+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00efc0 tx=0x7f20ac00c3a0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 shutdown_connections 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f209c0a2e10 0x7f209c0a52d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b4108650 0x7f20b4075210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b41024e0 0x7f20b4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 --2- 172.21.15.87:0/2530927722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b4102e80 0x7f20b4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:22.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.998+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 >> 172.21.15.87:0/2530927722 conn(0x7f20b40795f0 msgr2=0x7f20b4101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:22.999+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 shutdown_connections 2024-06-06T08:11:22.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:23.000+0000 7f20b9b11700 1 -- 172.21.15.87:0/2530927722 wait complete. 2024-06-06T08:11:23.002 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:11:23.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 115844 10.90 MB/sec execute 445 sec latency 313.537 ms 2024-06-06T08:11:23.727 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-06-06T08:11:23.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-06-06T08:11:24.073 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:24.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116119 10.90 MB/sec execute 446 sec latency 351.935 ms 2024-06-06T08:11:24.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:23 smithi087 ceph-mon[32276]: pgmap v743: 129 pgs: 129 active+clean; 616 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 173 op/s 2024-06-06T08:11:24.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:23 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2530927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:23 smithi195 ceph-mon[37040]: pgmap v743: 129 pgs: 129 active+clean; 616 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 173 op/s 2024-06-06T08:11:24.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:23 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2530927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:23 smithi181 ceph-mon[36790]: pgmap v743: 129 pgs: 129 active+clean; 616 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 173 op/s 2024-06-06T08:11:24.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:23 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2530927722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:25.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116443 10.94 MB/sec execute 447 sec latency 397.293 ms 2024-06-06T08:11:25.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.969+0000 7f87dee40700 1 -- 172.21.15.87:0/2883307031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810ae30 msgr2=0x7f87d810b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:25.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.969+0000 7f87dee40700 1 --2- 172.21.15.87:0/2883307031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810ae30 0x7f87d810b210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098f0 tx=0x7f87cc009c00 comp rx=0 tx=0).stop 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 -- 172.21.15.87:0/2883307031 shutdown_connections 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 --2- 172.21.15.87:0/2883307031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d81143e0 0x7f87d81167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 --2- 172.21.15.87:0/2883307031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810ae30 0x7f87d810b210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 --2- 172.21.15.87:0/2883307031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d810b750 0x7f87d8113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 -- 172.21.15.87:0/2883307031 >> 172.21.15.87:0/2883307031 conn(0x7f87d806d600 msgr2=0x7f87d806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:25.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.970+0000 7f87dee40700 1 -- 172.21.15.87:0/2883307031 shutdown_connections 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.971+0000 7f87dee40700 1 -- 172.21.15.87:0/2883307031 wait complete. 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 Processor -- start 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 -- start start 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810b750 0x7f87d8113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d81143e0 0x7f87d81a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87d811b000 con 0x7f87d810ae30 2024-06-06T08:11:25.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87d811ae80 con 0x7f87d81143e0 2024-06-06T08:11:25.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dee40700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87d811b180 con 0x7f87d810b750 2024-06-06T08:11:25.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.972+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51332/0 (socket says 172.21.15.87:51332) 2024-06-06T08:11:25.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1218665087 learned_addr learned my addr 172.21.15.87:0/1218665087 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1218665087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810b750 msgr2=0x7f87d8113540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810b750 0x7f87d8113540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1218665087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d81143e0 msgr2=0x7f87d81a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.973+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d81143e0 0x7f87d81a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.974+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1218665087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d81a3fe0 con 0x7f87d810ae30 2024-06-06T08:11:25.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.974+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005950 tx=0x7f87cc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:25.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.975+0000 7f87d5ffb700 1 -- 172.21.15.87:0/1218665087 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0052e0 con 0x7f87d810ae30 2024-06-06T08:11:25.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.975+0000 7f87d5ffb700 1 -- 172.21.15.87:0/1218665087 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc00ba40 con 0x7f87d810ae30 2024-06-06T08:11:25.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.976+0000 7f87d5ffb700 1 -- 172.21.15.87:0/1218665087 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003cb0 con 0x7f87d810ae30 2024-06-06T08:11:25.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.977+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d81a4270 con 0x7f87d810ae30 2024-06-06T08:11:25.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.977+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81a46d0 con 0x7f87d810ae30 2024-06-06T08:11:25.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.981+0000 7f87d5ffb700 1 -- 172.21.15.87:0/1218665087 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f87cc00b560 con 0x7f87d810ae30 2024-06-06T08:11:25.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.982+0000 7f87d5ffb700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 0x7f87c00a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 0x7f87c00a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87d5ffb700 1 -- 172.21.15.87:0/1218665087 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f87cc0d0d50 con 0x7f87d810ae30 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87e96c3b80 1 Processor -- start 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 0x7f87c00a54d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87d8114170 tx=0x7f87c8009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87e96c3b80 1 -- start start 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.983+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f217469b0 con 0x556f217f5470 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f216ee8c0 con 0x556f217face0 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f21742970 con 0x556f217fb600 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87dd3dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87dd3dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51348/0 (socket says 172.21.15.87:51348) 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87dd3dd700 1 -- 172.21.15.87:0/516875515 learned_addr learned my addr 172.21.15.87:0/516875515 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 -- 172.21.15.87:0/516875515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 msgr2=0x556f217fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 -- 172.21.15.87:0/516875515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 msgr2=0x556f217f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 -- 172.21.15.87:0/516875515 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f216e50f0 con 0x556f217fb600 2024-06-06T08:11:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87dd3dd700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:11:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.984+0000 7f87d7fff700 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217f4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87c800eab0 tx=0x7f87c8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1129b08b4b9f35f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.985+0000 7f87beffd700 1 -- 172.21.15.87:0/516875515 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8020070 con 0x556f217fb600 2024-06-06T08:11:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.985+0000 7f87beffd700 1 -- 172.21.15.87:0/516875515 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c8010d20 con 0x556f217fb600 2024-06-06T08:11:25.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.985+0000 7f87beffd700 1 -- 172.21.15.87:0/516875515 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80052f0 con 0x556f217fb600 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- 172.21.15.87:0/516875515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 msgr2=0x556f217f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217f4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87c800eab0 tx=0x7f87c8011000 comp rx=0 tx=0).stop 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- 172.21.15.87:0/516875515 shutdown_connections 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/516875515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217f4f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- 172.21.15.87:0/516875515 >> 172.21.15.87:0/516875515 conn(0x556f2171fe60 msgr2=0x556f217f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- 172.21.15.87:0/516875515 shutdown_connections 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- 172.21.15.87:0/516875515 wait complete. 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 -- start start 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.986+0000 7f87e96c3b80 1 Processor -- start 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556f218020c0 con 0x556f217f5470 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f216e6d90 con 0x556f217face0 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f21802240 con 0x556f217fb600 2024-06-06T08:11:25.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87e96c3b80 10 client.0 _ll_register_callbacks cb 0x556f216e0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51362/0 (socket says 172.21.15.87:51362) 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 -- 172.21.15.87:0/624259352 learned_addr learned my addr 172.21.15.87:0/624259352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dd3dd700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87d7fff700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 -- 172.21.15.87:0/624259352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 msgr2=0x556f217febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 -- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 msgr2=0x556f217fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.987+0000 7f87dcbdc700 1 -- 172.21.15.87:0/624259352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f21695a70 con 0x556f217f5470 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f87cc099850 tx=0x7f87cc003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67ee4f9efdd8ac6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0ce4a0 con 0x556f217f5470 2024-06-06T08:11:25.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f87cc00e3e0 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c8010d20 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f87cc00e5e0 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87e96c3b80 10 client.18600 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556f2180c9d0 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.988+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c8010d20 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f87cc02b070 con 0x556f217f5470 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 10 client.18600 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 10 client.18600 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 20 client.18600 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2180eea0 0x556f21811290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 4 client.18600 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f21753eb0 con 0x556f2180eea0 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.989+0000 7f87d7fff700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2180eea0 0x556f21811290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:25.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.990+0000 7f87d7fff700 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2180eea0 0x556f21811290 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db009c289e3b69a server_cookie=38ed4662df9bbf4f in_seq=0 out_seq=0 2024-06-06T08:11:25.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.990+0000 7f87b7fff700 10 client.18600 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87b7fff700 1 -- 172.21.15.87:0/624259352 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f21753eb0 con 0x556f2180eea0 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87b7fff700 10 client.18600 handle_command_reply: tid=0 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 client.18600 shutdown 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 msgr2=0x556f21801310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f87cc099850 tx=0x7f87cc003ed0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 shutdown_connections 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2180eea0 0x556f21811290 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f217febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21801310 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/624259352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.992+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 >> 172.21.15.87:0/624259352 conn(0x556f2171fe60 msgr2=0x556f217efdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.993+0000 7f87b7fff700 0 client.18600 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.993+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 shutdown_connections 2024-06-06T08:11:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.993+0000 7f87e96c3b80 1 -- 172.21.15.87:0/624259352 wait complete. 2024-06-06T08:11:25.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:25.993+0000 7f87e96c3b80 20 client.18600 trim_cache size 0 max 16384 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.016+0000 7f87e96c3b80 1 Processor -- start 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.016+0000 7f87e96c3b80 1 -- start start 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f21816b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b74f0 con 0x556f217f5470 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b4d80 con 0x556f217face0 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b2bf0 con 0x556f217fb600 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51378/0 (socket says 172.21.15.87:51378) 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 -- 172.21.15.87:0/2100998094 learned_addr learned my addr 172.21.15.87:0/2100998094 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dd3dd700 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f21816b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87d7fff700 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 -- 172.21.15.87:0/2100998094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 msgr2=0x556f21816b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f21816b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 -- 172.21.15.87:0/2100998094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 msgr2=0x556f217fa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 -- 172.21.15.87:0/2100998094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f21801ad0 con 0x556f217f5470 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0992e0 tx=0x7f87cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93154bdd69fca8f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87be7fc700 1 -- 172.21.15.87:0/2100998094 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00bde0 con 0x556f217f5470 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87be7fc700 1 -- 172.21.15.87:0/2100998094 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc09d580 con 0x556f217f5470 2024-06-06T08:11:26.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.017+0000 7f87be7fc700 1 -- 172.21.15.87:0/2100998094 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc09d220 con 0x556f217f5470 2024-06-06T08:11:26.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 -- 172.21.15.87:0/2100998094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 msgr2=0x556f217f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:26.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0992e0 tx=0x7f87cc009c00 comp rx=0 tx=0).stop 2024-06-06T08:11:26.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 -- 172.21.15.87:0/2100998094 shutdown_connections 2024-06-06T08:11:26.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217fb600 0x556f21816b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f217f4f30 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/2100998094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217face0 0x556f217fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 -- 172.21.15.87:0/2100998094 >> 172.21.15.87:0/2100998094 conn(0x556f2171fe60 msgr2=0x556f217fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 -- 172.21.15.87:0/2100998094 shutdown_connections 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.018+0000 7f87e96c3b80 1 -- 172.21.15.87:0/2100998094 wait complete. 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 -- start start 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 Processor -- start 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 0x556f21806040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217fb600 0x556f218065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b3240 con 0x556f217f5470 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b46b0 con 0x556f217fb600 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b3910 con 0x556f217face0 2024-06-06T08:11:26.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87dcbdc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51390/0 (socket says 172.21.15.87:51390) 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1986747759 learned_addr learned my addr 172.21.15.87:0/1986747759 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.019+0000 7f87e96c3b80 10 client.0 _ll_register_callbacks cb 0x556f216e0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dd3dd700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217fb600 0x556f218065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 0x556f21806040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1986747759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 msgr2=0x556f21806040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 0x556f21806040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217fb600 msgr2=0x556f218065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217fb600 0x556f218065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 -- 172.21.15.87:0/1986747759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f21806e20 con 0x556f217f5470 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 0x556f21806040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87dcbdc700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009810 tx=0x7f87cc09d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb6ac6a9d0c1a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00bde0 con 0x556f217f5470 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87cc09d580 con 0x556f217f5470 2024-06-06T08:11:26.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f87cc00e3e0 con 0x556f217f5470 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f87cc0a1230 con 0x556f217f5470 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87e96c3b80 10 client.18606 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556f21806e20 con 0x556f217f5470 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.020+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc09d580 con 0x556f217f5470 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f87cc02b070 con 0x556f217f5470 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 10 client.18606 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 10 client.18606 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 20 client.18606 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2187aee0 0x556f2187d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 4 client.18606 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x556f2174e430 con 0x556f2187aee0 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2187aee0 0x556f2187d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87b7fff700 10 client.18606 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:26.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.021+0000 7f87d7fff700 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2187aee0 0x556f2187d2d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8457402ce79806 server_cookie=dfe07471927b7516 in_seq=0 out_seq=0 2024-06-06T08:11:26.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116657 10.93 MB/sec execute 448 sec latency 532.337 ms 2024-06-06T08:11:26.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:25 smithi195 ceph-mon[37040]: pgmap v744: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:26.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:25 smithi181 ceph-mon[36790]: pgmap v744: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:25 smithi087 ceph-mon[32276]: pgmap v744: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:26.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87b7fff700 1 -- 172.21.15.87:0/1986747759 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556f2174e430 con 0x556f2187aee0 2024-06-06T08:11:26.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87b7fff700 10 client.18606 handle_command_reply: tid=0 2024-06-06T08:11:26.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 client.18606 shutdown 2024-06-06T08:11:26.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 msgr2=0x556f21807c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:26.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009810 tx=0x7f87cc09d450 comp rx=0 tx=0).stop 2024-06-06T08:11:26.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 shutdown_connections 2024-06-06T08:11:26.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x556f2187aee0 0x556f2187d2d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f217fb600 0x556f218065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f217f5470 0x556f21807c90 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87b7fff700 0 client.18606 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 --2- 172.21.15.87:0/1986747759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f217face0 0x556f21806040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 >> 172.21.15.87:0/1986747759 conn(0x556f2171fe60 msgr2=0x556f217fa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 shutdown_connections 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.932+0000 7f87e96c3b80 1 -- 172.21.15.87:0/1986747759 wait complete. 2024-06-06T08:11:26.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87e96c3b80 20 client.18606 trim_cache size 0 max 16384 2024-06-06T08:11:26.936 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:11:26.936 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:11:26.936 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:11:26.936 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 74, 2024-06-06T08:11:26.936 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "bee6dbd1": { 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 74, 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:11:26.937 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 141.8, 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:11:26.938 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "QUIESCED", 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:26.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:26.940 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:26.940 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:11:26.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 msgr2=0x7f87c00a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:26.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 0x7f87c00a54d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87d8114170 tx=0x7f87c8009260 comp rx=0 tx=0).stop 2024-06-06T08:11:26.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 msgr2=0x7f87d8113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:26.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005950 tx=0x7f87cc005040 comp rx=0 tx=0).stop 2024-06-06T08:11:26.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 shutdown_connections 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d81143e0 0x7f87d81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f87c00a3010 0x7f87c00a54d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d810ae30 0x7f87d8113000 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 --2- 172.21.15.87:0/1218665087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87d810b750 0x7f87d8113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.933+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 >> 172.21.15.87:0/1218665087 conn(0x7f87d806d600 msgr2=0x7f87d81184f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:26.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.934+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 shutdown_connections 2024-06-06T08:11:26.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:26.934+0000 7f87dee40700 1 -- 172.21.15.87:0/1218665087 wait complete. 2024-06-06T08:11:27.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.91 MB/sec execute 449 sec latency 1282.412 ms 2024-06-06T08:11:27.814 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bee6dbd1', quiesce took 7.4 seconds. Will release after: 14.35 2024-06-06T08:11:28.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:27 smithi087 ceph-mon[32276]: pgmap v745: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:28.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.88 MB/sec execute 450 sec latency 2020.755 ms 2024-06-06T08:11:28.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:27 smithi195 ceph-mon[37040]: pgmap v745: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:28.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:27 smithi181 ceph-mon[36790]: pgmap v745: 129 pgs: 129 active+clean; 621 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-06-06T08:11:29.194 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.86 MB/sec execute 451 sec latency 3020.799 ms 2024-06-06T08:11:30.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.84 MB/sec execute 452 sec latency 4020.905 ms 2024-06-06T08:11:30.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:29 smithi195 ceph-mon[37040]: pgmap v746: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-06-06T08:11:30.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:29 smithi181 ceph-mon[36790]: pgmap v746: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-06-06T08:11:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:29 smithi087 ceph-mon[32276]: pgmap v746: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-06-06T08:11:31.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.81 MB/sec execute 453 sec latency 5021.006 ms 2024-06-06T08:11:32.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.79 MB/sec execute 454 sec latency 6021.113 ms 2024-06-06T08:11:32.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:31 smithi195 ceph-mon[37040]: pgmap v747: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-06-06T08:11:32.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:31 smithi181 ceph-mon[36790]: pgmap v747: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-06-06T08:11:32.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:31 smithi087 ceph-mon[32276]: pgmap v747: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-06-06T08:11:33.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.77 MB/sec execute 455 sec latency 7021.200 ms 2024-06-06T08:11:34.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.74 MB/sec execute 456 sec latency 8021.318 ms 2024-06-06T08:11:34.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:33 smithi195 ceph-mon[37040]: pgmap v748: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-06-06T08:11:34.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:33 smithi181 ceph-mon[36790]: pgmap v748: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-06-06T08:11:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:33 smithi087 ceph-mon[32276]: pgmap v748: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-06-06T08:11:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 -- 172.21.15.87:0/4170415054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a81068a0 msgr2=0x7fe8a8106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 --2- 172.21.15.87:0/4170415054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a81068a0 0x7fe8a8106c80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe898009930 tx=0x7fe89802a420 comp rx=0 tx=0).stop 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 -- 172.21.15.87:0/4170415054 shutdown_connections 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 --2- 172.21.15.87:0/4170415054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 0x7fe8a810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 --2- 172.21.15.87:0/4170415054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a8100890 0x7fe8a8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.702+0000 7fe8ad963640 1 --2- 172.21.15.87:0/4170415054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a81068a0 0x7fe8a8106c80 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 -- 172.21.15.87:0/4170415054 >> 172.21.15.87:0/4170415054 conn(0x7fe8a8077a50 msgr2=0x7fe8a8077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 -- 172.21.15.87:0/4170415054 shutdown_connections 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 -- 172.21.15.87:0/4170415054 wait complete. 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 Processor -- start 2024-06-06T08:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 -- start start 2024-06-06T08:11:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 0x7fe8a8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.703+0000 7fe8ad963640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 0x7fe8a819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ad963640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110c70 con 0x7fe8a8100890 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ad963640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110af0 con 0x7fe8a8101250 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ad963640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110df0 con 0x7fe8a81068a0 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39382/0 (socket says 172.21.15.87:39382) 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/3215691481 learned_addr learned my addr 172.21.15.87:0/3215691481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 0x7fe8a8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 0x7fe8a819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/3215691481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 msgr2=0x7fe8a819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 0x7fe8a819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/3215691481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 msgr2=0x7fe8a8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 0x7fe8a8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/3215691481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a819d1b0 con 0x7fe8a8100890 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 0x7fe8a819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe898009a50 tx=0x7fe898004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ac961640 1 -- 172.21.15.87:0/3215691481 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89803f030 con 0x7fe8a8100890 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a819d440 con 0x7fe8a8100890 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ac961640 1 -- 172.21.15.87:0/3215691481 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898003ce0 con 0x7fe8a8100890 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ac961640 1 -- 172.21.15.87:0/3215691481 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898035070 con 0x7fe8a8100890 2024-06-06T08:11:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.704+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a4d30 con 0x7fe8a8100890 2024-06-06T08:11:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8ac961640 1 -- 172.21.15.87:0/3215691481 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe898042030 con 0x7fe8a8100890 2024-06-06T08:11:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 Processor -- start 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 -- start start 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8ac961640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 0x7fe8940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5561933b64f0 con 0x556193452170 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556193354870 con 0x556193452a90 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8b0674740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5561933b24b0 con 0x55619345d350 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39394/0 (socket says 172.21.15.87:39394) 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 0x7fe8940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.707+0000 7fe8a77fe640 1 -- 172.21.15.87:0/1104230528 learned_addr learned my addr 172.21.15.87:0/1104230528 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8ac961640 1 -- 172.21.15.87:0/3215691481 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7fe8980efdf0 con 0x7fe8a8100890 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 -- 172.21.15.87:0/1104230528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 msgr2=0x55619345f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 -- 172.21.15.87:0/1104230528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 msgr2=0x55619345ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 -- 172.21.15.87:0/1104230528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561933709b0 con 0x556193452170 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 0x7fe8940a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe8a806b330 tx=0x7fe89000a2d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00126e0 tx=0x7fe8a000a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eca23de56caa1a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe89dffb640 1 -- 172.21.15.87:0/1104230528 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0018070 con 0x556193452170 2024-06-06T08:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe89dffb640 1 -- 172.21.15.87:0/1104230528 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a0010dd0 con 0x556193452170 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.708+0000 7fe89dffb640 1 -- 172.21.15.87:0/1104230528 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00056c0 con 0x556193452170 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- 172.21.15.87:0/1104230528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 msgr2=0x556193452550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00126e0 tx=0x7fe8a000a880 comp rx=0 tx=0).stop 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- 172.21.15.87:0/1104230528 shutdown_connections 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1104230528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x556193452550 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- 172.21.15.87:0/1104230528 >> 172.21.15.87:0/1104230528 conn(0x55619344e680 msgr2=0x55619344ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- 172.21.15.87:0/1104230528 shutdown_connections 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- 172.21.15.87:0/1104230528 wait complete. 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 -- start start 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.709+0000 7fe8b0674740 1 Processor -- start 2024-06-06T08:11:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55619345ff80 con 0x556193452170 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55619333fae0 con 0x556193452a90 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556193460100 con 0x55619345d350 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 10 client.0 _ll_register_callbacks cb 0x5561933789b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39400/0 (socket says 172.21.15.87:39400) 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/2810105081 learned_addr learned my addr 172.21.15.87:0/2810105081 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/2810105081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 msgr2=0x55619345acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 msgr2=0x55619345a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 -- 172.21.15.87:0/2810105081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561930ca210 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe898031550 tx=0x7fe8980bbcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85c4d14062354ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898049040 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7fe8980b8560 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a0010dd0 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe8980ba6a0 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 10 client.18618 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.710+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556193196620 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a0010dd0 con 0x556193452170 2024-06-06T08:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fe89803b460 con 0x556193452170 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 10 client.18618 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 10 client.18618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 20 client.18618 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55619346d530 0x55619346f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 4 client.18618 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561933c9f10 con 0x55619346d530 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.711+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55619346d530 0x55619346f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.712+0000 7fe886ffd640 10 client.18618 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.712+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55619346d530 0x55619346f9f0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3517c7888426975 server_cookie=1dbd4570733f042d in_seq=0 out_seq=0 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.712+0000 7fe886ffd640 1 -- 172.21.15.87:0/2810105081 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561933c9f10 con 0x55619346d530 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.712+0000 7fe886ffd640 10 client.18618 handle_command_reply: tid=0 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.712+0000 7fe8b0674740 1 client.18618 shutdown 2024-06-06T08:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 msgr2=0x55619345a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe898031550 tx=0x7fe8980bbcc0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 shutdown_connections 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55619346d530 0x55619346f9f0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55619345d350 0x55619345acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556193452a90 0x55619345a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe886ffd640 0 client.18618 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 --2- 172.21.15.87:0/2810105081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452170 0x55619345a270 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 >> 172.21.15.87:0/2810105081 conn(0x55619344e680 msgr2=0x556193454400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 shutdown_connections 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 1 -- 172.21.15.87:0/2810105081 wait complete. 2024-06-06T08:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.713+0000 7fe8b0674740 20 client.18618 trim_cache size 0 max 16384 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 Processor -- start 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 -- start start 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x556193475300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x556193453b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b1910 con 0x556193452a90 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81af1e0 con 0x55619345d350 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8b0674740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81ad050 con 0x556193452170 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39416/0 (socket says 172.21.15.87:39416) 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 -- 172.21.15.87:0/1855616952 learned_addr learned my addr 172.21.15.87:0/1855616952 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x556193475300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 -- 172.21.15.87:0/1855616952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 msgr2=0x556193475300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x556193475300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 -- 172.21.15.87:0/1855616952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 msgr2=0x556193453b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x556193453b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x556193453b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 -- 172.21.15.87:0/1855616952 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556193470070 con 0x556193452a90 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x556193475300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x556193453b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.728+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x5561934702a0 tx=0x7fe890016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b054016e7506b780 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe89d7fa640 1 -- 172.21.15.87:0/1855616952 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890016070 con 0x556193452a90 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe89d7fa640 1 -- 172.21.15.87:0/1855616952 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890010d40 con 0x556193452a90 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe89d7fa640 1 -- 172.21.15.87:0/1855616952 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8900242c0 con 0x556193452a90 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- 172.21.15.87:0/1855616952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 msgr2=0x556193475840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x5561934702a0 tx=0x7fe890016000 comp rx=0 tx=0).stop 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- 172.21.15.87:0/1855616952 shutdown_connections 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x556193453b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193475840 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 --2- 172.21.15.87:0/1855616952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x556193475300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- 172.21.15.87:0/1855616952 >> 172.21.15.87:0/1855616952 conn(0x55619344e680 msgr2=0x5561934554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- 172.21.15.87:0/1855616952 shutdown_connections 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- 172.21.15.87:0/1855616952 wait complete. 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.729+0000 7fe8b0674740 1 -- start start 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 Processor -- start 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x5561934608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x5561934d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81ad6a0 con 0x556193452a90 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81aeb10 con 0x55619345d350 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81add70 con 0x556193452170 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8b0674740 10 client.0 _ll_register_callbacks cb 0x7fe8980f00f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39422/0 (socket says 172.21.15.87:39422) 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 -- 172.21.15.87:0/805982539 learned_addr learned my addr 172.21.15.87:0/805982539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x5561934d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 -- 172.21.15.87:0/805982539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 msgr2=0x5561934608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x5561934608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 -- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 msgr2=0x5561934d62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x5561934d62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 -- 172.21.15.87:0/805982539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556193453100 con 0x556193452a90 2024-06-06T08:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe8a77fe640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x5561934d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.730+0000 7fe89e7fc640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe89000fe90 tx=0x7fe8900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a186eb1e3b53038c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89002c6d0 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7fe89002c870 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe890010d40 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe89002ca70 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 10 client.18624 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5561934d78f0 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890010d40 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fe89001b410 con 0x556193452a90 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 10 client.18624 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 10 client.18624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 20 client.18624 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561934d8090 0x5561934da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 4 client.18624 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:11:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561933c3c20 con 0x5561934d8090 2024-06-06T08:11:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.731+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561934d8090 0x5561934da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.732+0000 7fe8a6ffd640 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561934d8090 0x5561934da4b0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ee99c3d1aa8363 server_cookie=3a57699e18516276 in_seq=0 out_seq=0 2024-06-06T08:11:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:34.732+0000 7fe886ffd640 10 client.18624 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe886ffd640 1 -- 172.21.15.87:0/805982539 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561933c3c20 con 0x5561934d8090 2024-06-06T08:11:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe886ffd640 10 client.18624 handle_command_reply: tid=0 2024-06-06T08:11:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 client.18624 shutdown 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 msgr2=0x556193460e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe89000fe90 tx=0x7fe8900048f0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 shutdown_connections 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5561934d8090 0x5561934da4b0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55619345d350 0x5561934d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556193452a90 0x556193460e30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 --2- 172.21.15.87:0/805982539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556193452170 0x5561934608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 >> 172.21.15.87:0/805982539 conn(0x55619344e680 msgr2=0x55619345baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe886ffd640 0 client.18624 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 shutdown_connections 2024-06-06T08:11:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.075+0000 7fe8b0674740 1 -- 172.21.15.87:0/805982539 wait complete. 2024-06-06T08:11:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8b0674740 20 client.18624 trim_cache size 0 max 16384 2024-06-06T08:11:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:11:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:11:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 msgr2=0x7fe8940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 0x7fe8940a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe8a806b330 tx=0x7fe89000a2d0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 msgr2=0x7fe8a8198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe898009a50 tx=0x7fe898004270 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 shutdown_connections 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe8940a2da0 0x7fe8940a5260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8a81068a0 0x7fe8a819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a8101250 0x7fe8a8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 --2- 172.21.15.87:0/3215691481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a8100890 0x7fe8a8198210 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 >> 172.21.15.87:0/3215691481 conn(0x7fe8a8077a50 msgr2=0x7fe8a80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 shutdown_connections 2024-06-06T08:11:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:35.076+0000 7fe8ad963640 1 -- 172.21.15.87:0/3215691481 wait complete. 2024-06-06T08:11:35.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.72 MB/sec execute 457 sec latency 9021.362 ms 2024-06-06T08:11:36.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.69 MB/sec execute 458 sec latency 10021.443 ms 2024-06-06T08:11:36.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:35 smithi195 ceph-mon[37040]: pgmap v749: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-06-06T08:11:36.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:35 smithi181 ceph-mon[36790]: pgmap v749: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-06-06T08:11:36.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:35 smithi087 ceph-mon[32276]: pgmap v749: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-06-06T08:11:37.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.67 MB/sec execute 459 sec latency 11021.515 ms 2024-06-06T08:11:38.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.65 MB/sec execute 460 sec latency 12021.618 ms 2024-06-06T08:11:38.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:37 smithi195 ceph-mon[37040]: pgmap v750: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 139 op/s 2024-06-06T08:11:38.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:37 smithi181 ceph-mon[36790]: pgmap v750: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 139 op/s 2024-06-06T08:11:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:37 smithi087 ceph-mon[32276]: pgmap v750: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 139 op/s 2024-06-06T08:11:39.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.63 MB/sec execute 461 sec latency 13021.669 ms 2024-06-06T08:11:40.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.60 MB/sec execute 462 sec latency 14021.718 ms 2024-06-06T08:11:40.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:39 smithi195 ceph-mon[37040]: pgmap v751: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-06-06T08:11:40.309 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:39 smithi181 ceph-mon[36790]: pgmap v751: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-06-06T08:11:40.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:39 smithi087 ceph-mon[32276]: pgmap v751: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-06-06T08:11:41.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.58 MB/sec execute 463 sec latency 15021.776 ms 2024-06-06T08:11:42.165 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:11:42.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.56 MB/sec execute 464 sec latency 16021.826 ms 2024-06-06T08:11:42.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:41 smithi195 ceph-mon[37040]: pgmap v752: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 828 KiB/s wr, 109 op/s 2024-06-06T08:11:42.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:41 smithi181 ceph-mon[36790]: pgmap v752: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 828 KiB/s wr, 109 op/s 2024-06-06T08:11:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:41 smithi087 ceph-mon[32276]: pgmap v752: 129 pgs: 129 active+clean; 642 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 828 KiB/s wr, 109 op/s 2024-06-06T08:11:42.420 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:43.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.53 MB/sec execute 465 sec latency 17021.862 ms 2024-06-06T08:11:44.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.51 MB/sec execute 466 sec latency 18021.933 ms 2024-06-06T08:11:44.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:43 smithi195 ceph-mon[37040]: pgmap v753: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 921 KiB/s wr, 114 op/s 2024-06-06T08:11:44.299 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:43 smithi087 ceph-mon[32276]: pgmap v753: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 921 KiB/s wr, 114 op/s 2024-06-06T08:11:44.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:43 smithi181 ceph-mon[36790]: pgmap v753: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 921 KiB/s wr, 114 op/s 2024-06-06T08:11:44.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/3941725505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 msgr2=0x7f2d2810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:44.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/3941725505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d2810e240 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b0a0 tx=0x7f2d2000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/3941725505 shutdown_connections 2024-06-06T08:11:44.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/3941725505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d2810e240 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/3941725505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28107400 0x7f2d281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/3941725505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28101390 0x7f2d28101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/3941725505 >> 172.21.15.87:0/3941725505 conn(0x7f2d28078340 msgr2=0x7f2d28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:44.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/3941725505 shutdown_connections 2024-06-06T08:11:44.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.544+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/3941725505 wait complete. 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.545+0000 7f2d2d1d0700 1 Processor -- start 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.545+0000 7f2d2d1d0700 1 -- start start 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28101390 0x7f2d28199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28107400 0x7f2d281a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281116f0 con 0x7f2d28101d50 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28111570 con 0x7f2d28107400 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28111870 con 0x7f2d28101390 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:44.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42914/0 (socket says 172.21.15.87:42914) 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 -- 172.21.15.87:0/1928969702 learned_addr learned my addr 172.21.15.87:0/1928969702 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2759e700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28107400 0x7f2d281a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d26d9d700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28101390 0x7f2d28199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 -- 172.21.15.87:0/1928969702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28101390 msgr2=0x7f2d28199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28101390 0x7f2d28199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 -- 172.21.15.87:0/1928969702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28107400 msgr2=0x7f2d281a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28107400 0x7f2d281a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 -- 172.21.15.87:0/1928969702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a4fd0 con 0x7f2d28101d50 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2659c700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b530 tx=0x7f2d1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c015020 con 0x7f2d28101d50 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a5260 con 0x7f2d28101d50 2024-06-06T08:11:44.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c011d10 con 0x7f2d28101d50 2024-06-06T08:11:44.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.546+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00f2e0 con 0x7f2d28101d50 2024-06-06T08:11:44.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.547+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a5770 con 0x7f2d28101d50 2024-06-06T08:11:44.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.547+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d28069180 con 0x7f2d28101d50 2024-06-06T08:11:44.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.549+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2d1c018020 con 0x7f2d28101d50 2024-06-06T08:11:44.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.549+0000 7f2d0ffff700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 0x7f2d100a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:44.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.549+0000 7f2d26d9d700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 0x7f2d100a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:44.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.549+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f2d1c0cee50 con 0x7f2d28101d50 2024-06-06T08:11:44.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.550+0000 7f2d26d9d700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 0x7f2d100a5250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005950 tx=0x7f2d1800b410 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:44.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.554+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2d1c01f020 con 0x7f2d28101d50 2024-06-06T08:11:44.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.792+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d2802dca0 con 0x7f2d28101d50 2024-06-06T08:11:44.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.793+0000 7f2d0ffff700 1 -- 172.21.15.87:0/1928969702 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7f2d1c09b760 con 0x7f2d28101d50 2024-06-06T08:11:44.793 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:11:44.793 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 msgr2=0x7f2d100a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 0x7f2d100a5250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005950 tx=0x7f2d1800b410 comp rx=0 tx=0).stop 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 msgr2=0x7f2d28199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b530 tx=0x7f2d1c00b840 comp rx=0 tx=0).stop 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 shutdown_connections 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28107400 0x7f2d281a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2d100a2d90 0x7f2d100a5250 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28101390 0x7f2d28199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 --2- 172.21.15.87:0/1928969702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d28101d50 0x7f2d28199f50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:44.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.795+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 >> 172.21.15.87:0/1928969702 conn(0x7f2d28078340 msgr2=0x7f2d280ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:44.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.796+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 shutdown_connections 2024-06-06T08:11:44.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:44.796+0000 7f2d2d1d0700 1 -- 172.21.15.87:0/1928969702 wait complete. 2024-06-06T08:11:44.796 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:11:45.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:44 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1928969702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:45.195 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.49 MB/sec execute 467 sec latency 19022.009 ms 2024-06-06T08:11:45.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:44 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1928969702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:45.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:44 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1928969702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:11:46.033 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --set-id bee6dbd1 --release --await' 2024-06-06T08:11:46.033 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --set-id bee6dbd1 --release --await 2024-06-06T08:11:46.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.47 MB/sec execute 468 sec latency 20022.061 ms 2024-06-06T08:11:46.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:45 smithi195 ceph-mon[37040]: pgmap v754: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 859 KiB/s wr, 105 op/s 2024-06-06T08:11:46.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:45 smithi181 ceph-mon[36790]: pgmap v754: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 859 KiB/s wr, 105 op/s 2024-06-06T08:11:46.316 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:11:46.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:45 smithi087 ceph-mon[32276]: pgmap v754: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 859 KiB/s wr, 105 op/s 2024-06-06T08:11:47.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.44 MB/sec execute 469 sec latency 21022.098 ms 2024-06-06T08:11:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.168+0000 7f770af90640 1 -- 172.21.15.87:0/1690835629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 msgr2=0x7f770410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.168+0000 7f770af90640 1 --2- 172.21.15.87:0/1690835629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f770410d750 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009cd0 tx=0x7f76fc02a910 comp rx=0 tx=0).stop 2024-06-06T08:11:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- 172.21.15.87:0/1690835629 shutdown_connections 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- 172.21.15.87:0/1690835629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f770410d750 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- 172.21.15.87:0/1690835629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77041008d0 0x7f7704100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- 172.21.15.87:0/1690835629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7704106890 0x7f7704106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- 172.21.15.87:0/1690835629 >> 172.21.15.87:0/1690835629 conn(0x7f7704077a20 msgr2=0x7f7704077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- 172.21.15.87:0/1690835629 shutdown_connections 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- 172.21.15.87:0/1690835629 wait complete. 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 Processor -- start 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- start start 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77041008d0 0x7f7704197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7704106890 0x7f770419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7704110cc0 con 0x7f7704101290 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7704110b40 con 0x7f77041008d0 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f770af90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7704110e40 con 0x7f7704106890 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.169+0000 7f7703fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37656/0 (socket says 172.21.15.87:37656) 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 -- 172.21.15.87:0/3641605568 learned_addr learned my addr 172.21.15.87:0/3641605568 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7708d2c640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77041008d0 0x7f7704197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 -- 172.21.15.87:0/3641605568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7704106890 msgr2=0x7f770419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7704106890 0x7f770419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 -- 172.21.15.87:0/3641605568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77041008d0 msgr2=0x7f7704197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77041008d0 0x7f7704197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 -- 172.21.15.87:0/3641605568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770419c110 con 0x7f7704101290 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7703fff640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f76f000b450 tx=0x7f76f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7701ffb640 1 -- 172.21.15.87:0/3641605568 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0015020 con 0x7f7704101290 2024-06-06T08:11:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770419c400 con 0x7f7704101290 2024-06-06T08:11:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7701ffb640 1 -- 172.21.15.87:0/3641605568 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f0011d70 con 0x7f7704101290 2024-06-06T08:11:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f7701ffb640 1 -- 172.21.15.87:0/3641605568 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f000f340 con 0x7f7704101290 2024-06-06T08:11:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.170+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77041a3c90 con 0x7f7704101290 2024-06-06T08:11:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.172+0000 7f7701ffb640 1 -- 172.21.15.87:0/3641605568 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f76f0018020 con 0x7f7704101290 2024-06-06T08:11:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.173+0000 7f7701ffb640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 0x7f76f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.173+0000 7f7701ffb640 1 -- 172.21.15.87:0/3641605568 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f76f00cf260 con 0x7f7704101290 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.173+0000 7f7708d2c640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 0x7f76f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.173+0000 7f770dd80740 1 Processor -- start 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.173+0000 7f770dd80740 1 -- start start 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f7708d2c640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 0x7f76f40a5260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009bd0 tx=0x7f76f8005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ff347514f0 con 0x55ff347fc4a0 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff346ef870 con 0x55ff347ef850 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770dd80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff3474d4b0 con 0x55ff347f0a30 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37662/0 (socket says 172.21.15.87:37662) 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 -- 172.21.15.87:0/212920770 learned_addr learned my addr 172.21.15.87:0/212920770 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f7708d2c640 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f7703fff640 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 -- 172.21.15.87:0/212920770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 msgr2=0x55ff347ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 -- 172.21.15.87:0/212920770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 msgr2=0x55ff347efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 -- 172.21.15.87:0/212920770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3470b9b0 con 0x55ff347fc4a0 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f770952d640 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f76fc005e30 tx=0x7f76fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d400b12b22dfa161 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.174+0000 7f76eeffd640 1 -- 172.21.15.87:0/212920770 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc031b80 con 0x55ff347fc4a0 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f76eeffd640 1 -- 172.21.15.87:0/212920770 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc03f070 con 0x55ff347fc4a0 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f76eeffd640 1 -- 172.21.15.87:0/212920770 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc004040 con 0x55ff347fc4a0 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 -- 172.21.15.87:0/212920770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 msgr2=0x55ff347fe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f76fc005e30 tx=0x7f76fc005fd0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 -- 172.21.15.87:0/212920770 shutdown_connections 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347fe890 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 --2- 172.21.15.87:0/212920770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 -- 172.21.15.87:0/212920770 >> 172.21.15.87:0/212920770 conn(0x55ff347eb890 msgr2=0x55ff347ebc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 -- 172.21.15.87:0/212920770 shutdown_connections 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.175+0000 7f770dd80740 1 -- 172.21.15.87:0/212920770 wait complete. 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 -- start start 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 Processor -- start 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 0x55ff347f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347f0a30 0x55ff347f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ff347ff2d0 con 0x55ff347fc4a0 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff346daae0 con 0x55ff347f0a30 2024-06-06T08:11:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff347ff450 con 0x55ff347ef850 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770dd80740 10 client.0 _ll_register_callbacks cb 0x55ff347139b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37674/0 (socket says 172.21.15.87:37674) 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f770952d640 1 -- 172.21.15.87:0/2566233044 learned_addr learned my addr 172.21.15.87:0/2566233044 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f7703fff640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347f0a30 0x55ff347f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.176+0000 7f7708d2c640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 0x55ff347f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 -- 172.21.15.87:0/2566233044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 msgr2=0x55ff347f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 0x55ff347f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 -- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347f0a30 msgr2=0x55ff347f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347f0a30 0x55ff347f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 -- 172.21.15.87:0/2566233044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff34465210 con 0x55ff347fc4a0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f7708d2c640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 0x55ff347f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770952d640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009bf0 tx=0x7f76fc031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec64e72bf79a297 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc003970 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f76fc003b10 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76fc03f070 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f76fc003d10 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 10 client.18642 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ff34531620 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc03f070 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f76fc02f970 con 0x55ff347fc4a0 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 10 client.18642 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 20 client.18642 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.177+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff3480c780 0x55ff3480eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.178+0000 7f770dd80740 4 client.18642 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.178+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff34764f10 con 0x55ff3480c780 2024-06-06T08:11:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.178+0000 7f7708d2c640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff3480c780 0x55ff3480eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.178+0000 7f76e3fff640 10 client.18642 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.178+0000 7f7708d2c640 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff3480c780 0x55ff3480eb70 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7bab77f27e3199a server_cookie=2c29792b7cfef9ac in_seq=0 out_seq=0 2024-06-06T08:11:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f76e3fff640 1 -- 172.21.15.87:0/2566233044 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff34764f10 con 0x55ff3480c780 2024-06-06T08:11:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f76e3fff640 10 client.18642 handle_command_reply: tid=0 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 client.18642 shutdown 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 msgr2=0x55ff347f9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009bf0 tx=0x7f76fc031500 comp rx=0 tx=0).stop 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 shutdown_connections 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff3480c780 0x55ff3480eb70 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347f9e60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347f0a30 0x55ff347f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f76e3fff640 0 client.18642 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 --2- 172.21.15.87:0/2566233044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347ef850 0x55ff347f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 >> 172.21.15.87:0/2566233044 conn(0x55ff347eb890 msgr2=0x55ff347f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 shutdown_connections 2024-06-06T08:11:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 1 -- 172.21.15.87:0/2566233044 wait complete. 2024-06-06T08:11:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.179+0000 7f770dd80740 20 client.18642 trim_cache size 0 max 16384 2024-06-06T08:11:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 Processor -- start 2024-06-06T08:11:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 -- start start 2024-06-06T08:11:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77041af700 con 0x55ff347fc4a0 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f77041acfd0 con 0x55ff347ef850 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770dd80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77041aae40 con 0x55ff347f0a30 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.194+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37688/0 (socket says 172.21.15.87:37688) 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 -- 172.21.15.87:0/893767977 learned_addr learned my addr 172.21.15.87:0/893767977 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f7703fff640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f7708d2c640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 -- 172.21.15.87:0/893767977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 msgr2=0x55ff347ea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ea250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 -- 172.21.15.87:0/893767977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 msgr2=0x55ff347e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 -- 172.21.15.87:0/893767977 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff347f0830 con 0x55ff347fc4a0 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f7708d2c640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f770952d640 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009430 tx=0x7f76fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c4cea7f4a711f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f76ee7fc640 1 -- 172.21.15.87:0/893767977 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc005c40 con 0x55ff347fc4a0 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f76ee7fc640 1 -- 172.21.15.87:0/893767977 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc03f070 con 0x55ff347fc4a0 2024-06-06T08:11:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.195+0000 7f76ee7fc640 1 -- 172.21.15.87:0/893767977 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0042a0 con 0x55ff347fc4a0 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- 172.21.15.87:0/893767977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 msgr2=0x55ff347ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009430 tx=0x7f76fc004a40 comp rx=0 tx=0).stop 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- 172.21.15.87:0/893767977 shutdown_connections 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347fc4a0 0x55ff347ea790 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347f0a30 0x55ff347ea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- 172.21.15.87:0/893767977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff347e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- 172.21.15.87:0/893767977 >> 172.21.15.87:0/893767977 conn(0x55ff347eb890 msgr2=0x55ff347ebc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- 172.21.15.87:0/893767977 shutdown_connections 2024-06-06T08:11:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- 172.21.15.87:0/893767977 wait complete. 2024-06-06T08:11:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- start start 2024-06-06T08:11:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 Processor -- start 2024-06-06T08:11:48.199 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116664 10.42 MB/sec execute 470 sec latency 22022.131 ms 2024-06-06T08:11:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff34804910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347fc4a0 0x55ff34813600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77041ab490 con 0x55ff347f0a30 2024-06-06T08:11:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f77041ac900 con 0x55ff347ef850 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.196+0000 7f770dd80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77041abb60 con 0x55ff347fc4a0 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f770dd80740 10 client.0 _ll_register_callbacks cb 0x7f76f00092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37690/0 (socket says 172.21.15.87:37690) 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 -- 172.21.15.87:0/1148929260 learned_addr learned my addr 172.21.15.87:0/1148929260 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7708d2c640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff34804910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 -- 172.21.15.87:0/1148929260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347fc4a0 msgr2=0x55ff34813600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347fc4a0 0x55ff34813600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 -- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 msgr2=0x55ff34804910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff34804910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 -- 172.21.15.87:0/1148929260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff34813e00 con 0x55ff347f0a30 2024-06-06T08:11:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f7703fff640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ccb90 tx=0x7f76f0010400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ba35aaa94c8647 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f76e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f00b9060 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f76f00b4170 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76fc03f070 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f76f009bb70 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f770dd80740 10 client.18648 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ff34876c50 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.197+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc03f070 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f76f00ccce0 con 0x55ff347f0a30 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 10 client.18648 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 10 client.18648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 20 client.18648 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff348773b0 0x55ff348797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 4 client.18648 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:11:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff3475ec20 con 0x55ff348773b0 2024-06-06T08:11:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f7708d2c640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff348773b0 0x55ff348797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f7708d2c640 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff348773b0 0x55ff348797d0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bdb7da4debf2584 server_cookie=7cbf968419691802 in_seq=0 out_seq=0 2024-06-06T08:11:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.198+0000 7f76e3fff640 10 client.18648 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:47 smithi195 ceph-mon[37040]: pgmap v755: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 783 KiB/s wr, 102 op/s 2024-06-06T08:11:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:11:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:11:48.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:11:48.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:47 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:11:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f76e3fff640 1 -- 172.21.15.87:0/1148929260 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ff3475ec20 con 0x55ff348773b0 2024-06-06T08:11:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f76e3fff640 10 client.18648 handle_command_reply: tid=0 2024-06-06T08:11:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 client.18648 shutdown 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 msgr2=0x55ff34804e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ccb90 tx=0x7f76f0010400 comp rx=0 tx=0).stop 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 shutdown_connections 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55ff348773b0 0x55ff348797d0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff347fc4a0 0x55ff34813600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff347f0a30 0x55ff34804e50 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f76e3fff640 0 client.18648 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 --2- 172.21.15.87:0/1148929260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff347ef850 0x55ff34804910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.265+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 >> 172.21.15.87:0/1148929260 conn(0x55ff347eb890 msgr2=0x55ff347e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 shutdown_connections 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770dd80740 1 -- 172.21.15.87:0/1148929260 wait complete. 2024-06-06T08:11:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770dd80740 20 client.18648 trim_cache size 0 max 16384 2024-06-06T08:11:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:11:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:11:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:11:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 msgr2=0x7f76f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 0x7f76f40a5260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009bd0 tx=0x7f76f8005f90 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 msgr2=0x7f7704197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.266+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f76f000b450 tx=0x7f76f000b910 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 shutdown_connections 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f76f40a2da0 0x7f76f40a5260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7704106890 0x7f770419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7704101290 0x7f7704197660 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 --2- 172.21.15.87:0/3641605568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77041008d0 0x7f7704197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 >> 172.21.15.87:0/3641605568 conn(0x7f7704077a20 msgr2=0x7f7704103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 shutdown_connections 2024-06-06T08:11:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:11:48.267+0000 7f770af90640 1 -- 172.21.15.87:0/3641605568 wait complete. 2024-06-06T08:11:48.280 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:47 smithi087 ceph-mon[32276]: pgmap v755: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 783 KiB/s wr, 102 op/s 2024-06-06T08:11:48.280 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:11:48.280 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:11:48.280 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:11:48.280 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:47 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:11:48.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:47 smithi181 ceph-mon[36790]: pgmap v755: 129 pgs: 129 active+clean; 643 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 783 KiB/s wr, 102 op/s 2024-06-06T08:11:48.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:11:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:11:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:11:48.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:47 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:11:48.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 -- 172.21.15.87:0/2251495545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 msgr2=0x7f67c410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 --2- 172.21.15.87:0/2251495545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c410f370 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b0d0 tx=0x7f67bc00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 -- 172.21.15.87:0/2251495545 shutdown_connections 2024-06-06T08:11:48.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 --2- 172.21.15.87:0/2251495545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c410f370 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 --2- 172.21.15.87:0/2251495545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 0x7f67c4108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 --2- 172.21.15.87:0/2251495545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 -- 172.21.15.87:0/2251495545 >> 172.21.15.87:0/2251495545 conn(0x7f67c4077200 msgr2=0x7f67c4077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 -- 172.21.15.87:0/2251495545 shutdown_connections 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.389+0000 7f67cc788700 1 -- 172.21.15.87:0/2251495545 wait complete. 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 Processor -- start 2024-06-06T08:11:48.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 -- start start 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 0x7f67c4102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112800 con 0x7f67c406b580 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112680 con 0x7f67c406abc0 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cc788700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112980 con 0x7f67c41085a0 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37694/0 (socket says 172.21.15.87:37694) 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67c9d23700 1 -- 172.21.15.87:0/2409331542 learned_addr learned my addr 172.21.15.87:0/2409331542 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67cad25700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 0x7f67c4102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67ca524700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.390+0000 7f67c9d23700 1 -- 172.21.15.87:0/2409331542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 msgr2=0x7f67c4102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 0x7f67c4102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67c9d23700 1 -- 172.21.15.87:0/2409331542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 msgr2=0x7f67c41017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c41017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67c9d23700 1 -- 172.21.15.87:0/2409331542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a7080 con 0x7f67c406b580 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67ca524700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d950 tx=0x7f67b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67bb7fe700 1 -- 172.21.15.87:0/2409331542 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4016070 con 0x7f67c406b580 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a7370 con 0x7f67c406b580 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67bb7fe700 1 -- 172.21.15.87:0/2409331542 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b4009e20 con 0x7f67c406b580 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67bb7fe700 1 -- 172.21.15.87:0/2409331542 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b400f620 con 0x7f67c406b580 2024-06-06T08:11:48.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.391+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a7900 con 0x7f67c406b580 2024-06-06T08:11:48.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.394+0000 7f67bb7fe700 1 -- 172.21.15.87:0/2409331542 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f67b4019070 con 0x7f67c406b580 2024-06-06T08:11:48.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.394+0000 7f67bb7fe700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 0x7f67b00a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.395+0000 7f67ca524700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 0x7f67b00a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.395+0000 7f67bb7fe700 1 -- 172.21.15.87:0/2409331542 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f67b40cf1a0 con 0x7f67c406b580 2024-06-06T08:11:48.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.395+0000 7f67ca524700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 0x7f67b00a5130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009c90 tx=0x7f67c0009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.395+0000 7f67d700bb80 1 Processor -- start 2024-06-06T08:11:48.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 -- start start 2024-06-06T08:11:48.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x563666737020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666730e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x563666682910 con 0x5636667313d0 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56366662a820 con 0x563666737560 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67d700bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56366667e8d0 con 0x563666736c40 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666730e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37704/0 (socket says 172.21.15.87:37704) 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67ca524700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x563666737020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 -- 172.21.15.87:0/282767520 learned_addr learned my addr 172.21.15.87:0/282767520 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 -- 172.21.15.87:0/282767520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 msgr2=0x563666737020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x563666737020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 -- 172.21.15.87:0/282767520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 msgr2=0x563666730e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666730e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67cad25700 1 -- 172.21.15.87:0/282767520 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563666621050 con 0x5636667313d0 2024-06-06T08:11:48.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.396+0000 7f67ca524700 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x563666737020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:11:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67cad25700 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009d50 tx=0x7f67bc009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56986c0d075b422e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67b8ff9700 1 -- 172.21.15.87:0/282767520 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc009460 con 0x5636667313d0 2024-06-06T08:11:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67b8ff9700 1 -- 172.21.15.87:0/282767520 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67bc003a00 con 0x5636667313d0 2024-06-06T08:11:48.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67b8ff9700 1 -- 172.21.15.87:0/282767520 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0057c0 con 0x5636667313d0 2024-06-06T08:11:48.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67d700bb80 1 -- 172.21.15.87:0/282767520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 msgr2=0x563666731810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.397+0000 7f67d700bb80 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009d50 tx=0x7f67bc009d80 comp rx=0 tx=0).stop 2024-06-06T08:11:48.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- 172.21.15.87:0/282767520 shutdown_connections 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666731810 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x563666737020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- 172.21.15.87:0/282767520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666730e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- 172.21.15.87:0/282767520 >> 172.21.15.87:0/282767520 conn(0x56366665bdc0 msgr2=0x56366672c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- 172.21.15.87:0/282767520 shutdown_connections 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- 172.21.15.87:0/282767520 wait complete. 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- start start 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 Processor -- start 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636667313d0 0x56366673d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666737560 0x56366673aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x56366673df70 con 0x563666736c40 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563666622cf0 con 0x5636667313d0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56366673e0f0 con 0x563666737560 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67d700bb80 10 client.0 _ll_register_callbacks cb 0x56366661ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37712/0 (socket says 172.21.15.87:37712) 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67c9d23700 1 -- 172.21.15.87:0/3326432 learned_addr learned my addr 172.21.15.87:0/3326432 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.398+0000 7f67ca524700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636667313d0 0x56366673d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67cad25700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666737560 0x56366673aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 -- 172.21.15.87:0/3326432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666737560 msgr2=0x56366673aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666737560 0x56366673aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 -- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636667313d0 msgr2=0x56366673d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636667313d0 0x56366673d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 -- 172.21.15.87:0/3326432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636665d19d0 con 0x563666736c40 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67c9d23700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f67b40ccad0 tx=0x7f67b409f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23d4b4d63443784 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4020070 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f67b40a3a80 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67bc003a00 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f67b40b4240 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67d700bb80 10 client.18660 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563666748880 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67bc003a00 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.399+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f67b4023060 con 0x563666736c40 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 10 client.18660 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 10 client.18660 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 20 client.18660 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56366674aec0 0x56366674d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 4 client.18660 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366668fe10 con 0x56366674aec0 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67ca524700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56366674aec0 0x56366674d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67ca524700 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56366674aec0 0x56366674d2b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3cc8f9046c5140 server_cookie=d04cffac9d1a9384 in_seq=0 out_seq=0 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.400+0000 7f67adffb700 10 client.18660 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67adffb700 1 -- 172.21.15.87:0/3326432 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56366668fe10 con 0x56366674aec0 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67adffb700 10 client.18660 handle_command_reply: tid=0 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 client.18660 shutdown 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 msgr2=0x56366673a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f67b40ccad0 tx=0x7f67b409f650 comp rx=0 tx=0).stop 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 shutdown_connections 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x56366674aec0 0x56366674d2b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636667313d0 0x56366673d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666737560 0x56366673aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67adffb700 0 client.18660 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 --2- 172.21.15.87:0/3326432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x56366673a560 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 >> 172.21.15.87:0/3326432 conn(0x56366665bdc0 msgr2=0x56366672bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 shutdown_connections 2024-06-06T08:11:48.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 1 -- 172.21.15.87:0/3326432 wait complete. 2024-06-06T08:11:48.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.401+0000 7f67d700bb80 20 client.18660 trim_cache size 0 max 16384 2024-06-06T08:11:48.421 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 Processor -- start 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 -- start start 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x56366673a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666737a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41b2dc0 con 0x5636667313d0 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41b0650 con 0x563666737560 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67d700bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41ae4c0 con 0x563666736c40 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67ca524700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67ca524700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37718/0 (socket says 172.21.15.87:37718) 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67ca524700 1 -- 172.21.15.87:0/4080757268 learned_addr learned my addr 172.21.15.87:0/4080757268 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.421+0000 7f67c9d23700 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x56366673a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67cad25700 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666737a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 -- 172.21.15.87:0/4080757268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 msgr2=0x56366673a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x56366673a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 -- 172.21.15.87:0/4080757268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 msgr2=0x563666737a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666737a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 -- 172.21.15.87:0/4080757268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563666738af0 con 0x5636667313d0 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67ca524700 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009b50 tx=0x7f67c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc9d831e5fc81d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67affff700 1 -- 172.21.15.87:0/4080757268 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0011e40 con 0x5636667313d0 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67affff700 1 -- 172.21.15.87:0/4080757268 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c002a900 con 0x5636667313d0 2024-06-06T08:11:48.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67affff700 1 -- 172.21.15.87:0/4080757268 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c001fbf0 con 0x5636667313d0 2024-06-06T08:11:48.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67d700bb80 1 -- 172.21.15.87:0/4080757268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 msgr2=0x563666730e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.422+0000 7f67d700bb80 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009b50 tx=0x7f67c0013000 comp rx=0 tx=0).stop 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- 172.21.15.87:0/4080757268 shutdown_connections 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x563666737a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5636667313d0 0x563666730e90 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- 172.21.15.87:0/4080757268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563666736c40 0x56366673a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- 172.21.15.87:0/4080757268 >> 172.21.15.87:0/4080757268 conn(0x56366665bdc0 msgr2=0x5636667346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- 172.21.15.87:0/4080757268 shutdown_connections 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- 172.21.15.87:0/4080757268 wait complete. 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- start start 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 Processor -- start 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5636667313d0 0x5636667438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x5636667408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41aeb10 con 0x563666736c40 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41aff80 con 0x563666737560 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41af1e0 con 0x5636667313d0 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67d700bb80 10 client.0 _ll_register_callbacks cb 0x56366661ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67c9d23700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37732/0 (socket says 172.21.15.87:37732) 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67ca524700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5636667313d0 0x5636667438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67c9d23700 1 -- 172.21.15.87:0/2142386175 learned_addr learned my addr 172.21.15.87:0/2142386175 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.423+0000 7f67cad25700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x5636667408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 -- 172.21.15.87:0/2142386175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5636667313d0 msgr2=0x5636667438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5636667313d0 0x5636667438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 -- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 msgr2=0x5636667408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x5636667408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 -- 172.21.15.87:0/2142386175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563666740eb0 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67c9d23700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67b409afa0 tx=0x7f67b40a3e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68284a85b9c98e2c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4020070 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f67b409f160 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c002a900 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f67b4097540 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67d700bb80 10 client.18666 fetch_fsmap learned FSMap version 55 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5636667b6900 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c002a900 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f67b4096ff0 con 0x563666736c40 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.424+0000 7f67d700bb80 10 client.18666 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67d700bb80 10 client.18666 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67d700bb80 20 client.18666 populate_metadata read hostname 'smithi087' 2024-06-06T08:11:48.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5636667b6f80 0x5636667b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:11:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67d700bb80 4 client.18666 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bee6dbd1", "release": true, "await": true}] 2024-06-06T08:11:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bee6dbd1", "release": true, "await": true}) -- 0x56366668a390 con 0x5636667b6f80 2024-06-06T08:11:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67ca524700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5636667b6f80 0x5636667b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:11:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67ca524700 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5636667b6f80 0x5636667b93a0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759c133cdcf78784 server_cookie=d7d50a8ee7ffa837 in_seq=0 out_seq=0 2024-06-06T08:11:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.425+0000 7f67adffb700 10 client.18666 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.481+0000 7f67adffb700 1 -- 172.21.15.87:0/2142386175 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56366668a390 con 0x5636667b6f80 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.481+0000 7f67adffb700 10 client.18666 handle_command_reply: tid=0 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.481+0000 7f67d700bb80 1 client.18666 shutdown 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 msgr2=0x5636667403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67b409afa0 tx=0x7f67b40a3e30 comp rx=0 tx=0).stop 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 shutdown_connections 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5636667b6f80 0x5636667b93a0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563666737560 0x5636667408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5636667313d0 0x5636667438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67adffb700 0 client.18666 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 --2- 172.21.15.87:0/2142386175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563666736c40 0x5636667403b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 >> 172.21.15.87:0/2142386175 conn(0x56366665bdc0 msgr2=0x56366672b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 shutdown_connections 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 1 -- 172.21.15.87:0/2142386175 wait complete. 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67d700bb80 20 client.18666 trim_cache size 0 max 16384 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 76, 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": { 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "bee6dbd1": { 2024-06-06T08:11:48.483 INFO:teuthology.orchestra.run.smithi087.stdout: "version": 76, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age_ref": 0.0, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "timeout": 90.0, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "expiration": 141.8, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "members": { 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "file:/": { 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "excluded": false, 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "state": { 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RELEASED", 2024-06-06T08:11:48.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:11:48.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 msgr2=0x7f67b00a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 0x7f67b00a5130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009c90 tx=0x7f67c0009260 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 msgr2=0x7f67c4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d950 tx=0x7f67b400dc60 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 shutdown_connections 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c406abc0 0x7f67c41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f67b00a2c70 0x7f67b00a5130 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41085a0 0x7f67c4102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 --2- 172.21.15.87:0/2409331542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67c406b580 0x7f67c4101cf0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.482+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 >> 172.21.15.87:0/2409331542 conn(0x7f67c4077200 msgr2=0x7f67c4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.483+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 shutdown_connections 2024-06-06T08:11:48.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:11:48.483+0000 7f67cc788700 1 -- 172.21.15.87:0/2409331542 wait complete. 2024-06-06T08:11:49.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 116846 10.43 MB/sec execute 471 sec latency 22336.260 ms 2024-06-06T08:11:49.288 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bee6dbd1', total seconds elapsed: 28.9 2024-06-06T08:11:50.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 117107 10.43 MB/sec execute 472 sec latency 404.535 ms 2024-06-06T08:11:50.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:49 smithi195 ceph-mon[37040]: pgmap v756: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-06-06T08:11:50.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:49 smithi181 ceph-mon[36790]: pgmap v756: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-06-06T08:11:50.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:49 smithi087 ceph-mon[32276]: pgmap v756: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-06-06T08:11:51.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 117357 10.44 MB/sec execute 473 sec latency 369.343 ms 2024-06-06T08:11:52.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 117555 10.44 MB/sec execute 474 sec latency 473.274 ms 2024-06-06T08:11:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:51 smithi195 ceph-mon[37040]: pgmap v757: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 159 KiB/s wr, 11 op/s 2024-06-06T08:11:52.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:51 smithi181 ceph-mon[36790]: pgmap v757: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 159 KiB/s wr, 11 op/s 2024-06-06T08:11:52.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:51 smithi087 ceph-mon[32276]: pgmap v757: 129 pgs: 129 active+clean; 644 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 159 KiB/s wr, 11 op/s 2024-06-06T08:11:53.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 117791 10.44 MB/sec execute 475 sec latency 361.158 ms 2024-06-06T08:11:54.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 118095 10.45 MB/sec execute 476 sec latency 364.537 ms 2024-06-06T08:11:54.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:54 smithi195 ceph-mon[37040]: pgmap v758: 129 pgs: 129 active+clean; 646 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 945 KiB/s wr, 80 op/s 2024-06-06T08:11:54.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:54 smithi181 ceph-mon[36790]: pgmap v758: 129 pgs: 129 active+clean; 646 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 945 KiB/s wr, 80 op/s 2024-06-06T08:11:54.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:54 smithi087 ceph-mon[32276]: pgmap v758: 129 pgs: 129 active+clean; 646 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 945 KiB/s wr, 80 op/s 2024-06-06T08:11:55.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 118372 10.45 MB/sec execute 477 sec latency 434.426 ms 2024-06-06T08:11:56.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 118715 10.47 MB/sec execute 478 sec latency 396.764 ms 2024-06-06T08:11:56.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:56 smithi195 ceph-mon[37040]: pgmap v759: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-06-06T08:11:56.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:56 smithi181 ceph-mon[36790]: pgmap v759: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-06-06T08:11:56.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:56 smithi087 ceph-mon[32276]: pgmap v759: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-06-06T08:11:57.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 118923 10.49 MB/sec execute 479 sec latency 454.357 ms 2024-06-06T08:11:58.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119112 10.48 MB/sec execute 480 sec latency 267.394 ms 2024-06-06T08:11:58.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:11:58 smithi195 ceph-mon[37040]: pgmap v760: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-06-06T08:11:58.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:11:58 smithi181 ceph-mon[36790]: pgmap v760: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-06-06T08:11:58.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:11:58 smithi087 ceph-mon[32276]: pgmap v760: 129 pgs: 129 active+clean; 649 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-06-06T08:11:59.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119462 10.49 MB/sec execute 481 sec latency 179.222 ms 2024-06-06T08:12:00.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119775 10.51 MB/sec execute 482 sec latency 180.799 ms 2024-06-06T08:12:00.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:00 smithi195 ceph-mon[37040]: pgmap v761: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-06-06T08:12:00.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:00 smithi181 ceph-mon[36790]: pgmap v761: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-06-06T08:12:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:00 smithi087 ceph-mon[32276]: pgmap v761: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-06-06T08:12:01.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119839 10.49 MB/sec execute 483 sec latency 916.745 ms 2024-06-06T08:12:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 -- 172.21.15.87:0/3487174175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 msgr2=0x7f701010d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 --2- 172.21.15.87:0/3487174175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f701010d790 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7008009cd0 tx=0x7f700802a8e0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 -- 172.21.15.87:0/3487174175 shutdown_connections 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 --2- 172.21.15.87:0/3487174175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f701010d790 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 --2- 172.21.15.87:0/3487174175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 0x7f7010100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 --2- 172.21.15.87:0/3487174175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 0x7f7010106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 -- 172.21.15.87:0/3487174175 >> 172.21.15.87:0/3487174175 conn(0x7f7010076c20 msgr2=0x7f7010077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 -- 172.21.15.87:0/3487174175 shutdown_connections 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.352+0000 7f7016be7640 1 -- 172.21.15.87:0/3487174175 wait complete. 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 Processor -- start 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 -- start start 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 0x7f7010198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 0x7f701019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7010110ce0 con 0x7f7010101280 2024-06-06T08:12:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7010110b60 con 0x7f70101068d0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7016be7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7010110e60 con 0x7f70101008c0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f700ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f700ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59564/0 (socket says 172.21.15.87:59564) 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f700ffff640 1 -- 172.21.15.87:0/4212050870 learned_addr learned my addr 172.21.15.87:0/4212050870 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7015184640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 0x7f701019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.353+0000 7f7014983640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 0x7f7010198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 -- 172.21.15.87:0/4212050870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 msgr2=0x7f7010198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 0x7f7010198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 -- 172.21.15.87:0/4212050870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 msgr2=0x7f701019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 0x7f701019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 -- 172.21.15.87:0/4212050870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701019d160 con 0x7f7010101280 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700ffff640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f700400d770 tx=0x7f700400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700dffb640 1 -- 172.21.15.87:0/4212050870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004017070 con 0x7f7010101280 2024-06-06T08:12:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701019d450 con 0x7f7010101280 2024-06-06T08:12:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700dffb640 1 -- 172.21.15.87:0/4212050870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7004005500 con 0x7f7010101280 2024-06-06T08:12:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f700dffb640 1 -- 172.21.15.87:0/4212050870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004004e00 con 0x7f7010101280 2024-06-06T08:12:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.354+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a4ce0 con 0x7f7010101280 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.356+0000 7f700dffb640 1 -- 172.21.15.87:0/4212050870 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7004004fa0 con 0x7f7010101280 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 Processor -- start 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 -- start start 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f700dffb640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 0x7f6ff80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7015184640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 0x7f6ff80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50eda740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bd50e314f0 con 0x55bd50ecd170 2024-06-06T08:12:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd50dcf870 con 0x55bd50ecda90 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f70199d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd50e2d4b0 con 0x55bd50ed8350 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f700dffb640 1 -- 172.21.15.87:0/4212050870 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f70040cf2f0 con 0x7f7010101280 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59580/0 (socket says 172.21.15.87:59580) 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 -- 172.21.15.87:0/2023341025 learned_addr learned my addr 172.21.15.87:0/2023341025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7015184640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50eda740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f700ffff640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 -- 172.21.15.87:0/2023341025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 msgr2=0x55bd50eda740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50eda740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 -- 172.21.15.87:0/2023341025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 msgr2=0x55bd50ed7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 -- 172.21.15.87:0/2023341025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd50deb9b0 con 0x55bd50ecd170 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f700ffff640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:12:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7015184640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 0x7f6ff80a5260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7008004ab0 tx=0x7f7008031430 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f7014983640 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f700000dc30 tx=0x7f700000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59465b5bcf17dec4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.357+0000 7f6ff6ffd640 1 -- 172.21.15.87:0/2023341025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000011440 con 0x55bd50ecd170 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f6ff6ffd640 1 -- 172.21.15.87:0/2023341025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000017070 con 0x55bd50ecd170 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f6ff6ffd640 1 -- 172.21.15.87:0/2023341025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000109c0 con 0x55bd50ecd170 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 -- 172.21.15.87:0/2023341025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 msgr2=0x55bd50ecd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f700000dc30 tx=0x7f700000f730 comp rx=0 tx=0).stop 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 -- 172.21.15.87:0/2023341025 shutdown_connections 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50eda740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 --2- 172.21.15.87:0/2023341025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ecd550 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 -- 172.21.15.87:0/2023341025 >> 172.21.15.87:0/2023341025 conn(0x55bd50ec9680 msgr2=0x55bd50ec9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 -- 172.21.15.87:0/2023341025 shutdown_connections 2024-06-06T08:12:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.358+0000 7f70199d5740 1 -- 172.21.15.87:0/2023341025 wait complete. 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 -- start start 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 Processor -- start 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ed5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bd50edaf80 con 0x55bd50ecda90 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd50dbaae0 con 0x55bd50ecd170 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd50edb100 con 0x55bd50ed8350 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f70199d5740 10 client.0 _ll_register_callbacks cb 0x55bd50df39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59592/0 (socket says 172.21.15.87:59592) 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 -- 172.21.15.87:0/1131894117 learned_addr learned my addr 172.21.15.87:0/1131894117 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f7014983640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ed5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f7015184640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 -- 172.21.15.87:0/1131894117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 msgr2=0x55bd50ed5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 -- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 msgr2=0x55bd50ed5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ed5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.359+0000 7f700ffff640 1 -- 172.21.15.87:0/1131894117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd50b45210 con 0x55bd50ecda90 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f700ffff640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7004009610 tx=0x7f7004004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c38628682b9ba2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004020040 con 0x55bd50ecda90 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f7004009cc0 con 0x55bd50ecda90 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7000017070 con 0x55bd50ecda90 2024-06-06T08:12:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f70040a4060 con 0x55bd50ecda90 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 10 client.18678 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55bd50c11620 con 0x55bd50ecda90 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000017070 con 0x55bd50ecda90 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f700409a840 con 0x55bd50ecda90 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 10 client.18678 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 10 client.18678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 20 client.18678 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50ee8420 0x55bd50eea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 4 client.18678 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd50e44f10 con 0x55bd50ee8420 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.360+0000 7f7014983640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50ee8420 0x55bd50eea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.361+0000 7f6feffff640 10 client.18678 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.361+0000 7f7014983640 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50ee8420 0x55bd50eea810 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2885994f14126ab3 server_cookie=c992e566efdd6407 in_seq=0 out_seq=0 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.361+0000 7f6feffff640 1 -- 172.21.15.87:0/1131894117 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bd50e44f10 con 0x55bd50ee8420 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.361+0000 7f6feffff640 10 client.18678 handle_command_reply: tid=0 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.361+0000 7f70199d5740 1 client.18678 shutdown 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 msgr2=0x55bd50ed5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7004009610 tx=0x7f7004004800 comp rx=0 tx=0).stop 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 shutdown_connections 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50ee8420 0x55bd50eea810 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f6feffff640 0 client.18678 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecda90 0x55bd50ed5750 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 --2- 172.21.15.87:0/1131894117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ed5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 >> 172.21.15.87:0/1131894117 conn(0x55bd50ec9680 msgr2=0x55bd50ecf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 shutdown_connections 2024-06-06T08:12:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 1 -- 172.21.15.87:0/1131894117 wait complete. 2024-06-06T08:12:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.362+0000 7f70199d5740 20 client.18678 trim_cache size 0 max 16384 2024-06-06T08:12:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 Processor -- start 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 -- start start 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ef0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ecda90 0x55bd50ef0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70101b1820 con 0x55bd50ed8350 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70101af0f0 con 0x55bd50ecd170 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f70199d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70101acf60 con 0x55bd50ecda90 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59606/0 (socket says 172.21.15.87:59606) 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 -- 172.21.15.87:0/3076948306 learned_addr learned my addr 172.21.15.87:0/3076948306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7014983640 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ef0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f700ffff640 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ecda90 0x55bd50ef0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 -- 172.21.15.87:0/3076948306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ecda90 msgr2=0x55bd50ef0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ecda90 0x55bd50ef0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 -- 172.21.15.87:0/3076948306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 msgr2=0x55bd50ef0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ef0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 -- 172.21.15.87:0/3076948306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd50ee41e0 con 0x55bd50ed8350 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.377+0000 7f7015184640 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7008009cd0 tx=0x7f7008047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb8baf9e9156206 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f6ff67fc640 1 -- 172.21.15.87:0/3076948306 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008056070 con 0x55bd50ed8350 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f6ff67fc640 1 -- 172.21.15.87:0/3076948306 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7008043050 con 0x55bd50ed8350 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f6ff67fc640 1 -- 172.21.15.87:0/3076948306 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008041040 con 0x55bd50ed8350 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 -- 172.21.15.87:0/3076948306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 msgr2=0x55bd50ed51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7008009cd0 tx=0x7f7008047000 comp rx=0 tx=0).stop 2024-06-06T08:12:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 -- 172.21.15.87:0/3076948306 shutdown_connections 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ed8350 0x55bd50ed51d0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ecda90 0x55bd50ef0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 --2- 172.21.15.87:0/3076948306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecd170 0x55bd50ef0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 -- 172.21.15.87:0/3076948306 >> 172.21.15.87:0/3076948306 conn(0x55bd50ec9680 msgr2=0x55bd50ed0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 -- 172.21.15.87:0/3076948306 shutdown_connections 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.378+0000 7f70199d5740 1 -- 172.21.15.87:0/3076948306 wait complete. 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 -- start start 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 Processor -- start 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70101ad5b0 con 0x55bd50ecd170 2024-06-06T08:12:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70101aea20 con 0x55bd50ecda90 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70101adc80 con 0x55bd50ed8350 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f70199d5740 10 client.0 _ll_register_callbacks cb 0x7f7004097690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59616/0 (socket says 172.21.15.87:59616) 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 -- 172.21.15.87:0/1189957586 learned_addr learned my addr 172.21.15.87:0/1189957586 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7015184640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f700ffff640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 -- 172.21.15.87:0/1189957586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 msgr2=0x55bd50ed2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 -- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 msgr2=0x55bd50ed2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.379+0000 7f7014983640 1 -- 172.21.15.87:0/1189957586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd50f520d0 con 0x55bd50ecd170 2024-06-06T08:12:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f7014983640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f7000010570 tx=0x7f7000010500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24938f88151abe21 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000036060 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 9583+0+0 (secure 0 0 0) 0x7f7000004240 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008043050 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f700003d050 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 10 client.18684 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55bd50f53340 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7008043050 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f7000010610 con 0x55bd50ecd170 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 10 client.18684 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 10 client.18684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 20 client.18684 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50f53730 0x55bd50f55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 4 client.18684 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:12:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.380+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd50e3ec20 con 0x55bd50f53730 2024-06-06T08:12:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.381+0000 7f700ffff640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50f53730 0x55bd50f55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.381+0000 7f6feffff640 10 client.18684 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.381+0000 7f700ffff640 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50f53730 0x55bd50f55b50 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e80a327c4b754e server_cookie=e82c74ce5ca2edd5 in_seq=0 out_seq=0 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f6feffff640 1 -- 172.21.15.87:0/1189957586 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd50e3ec20 con 0x55bd50f53730 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f6feffff640 10 client.18684 handle_command_reply: tid=0 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 client.18684 shutdown 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 msgr2=0x55bd50ed2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f7000010570 tx=0x7f7000010500 comp rx=0 tx=0).stop 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 shutdown_connections 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd50f53730 0x55bd50f55b50 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd50ed8350 0x55bd50ed2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd50ecda90 0x55bd50ed2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 --2- 172.21.15.87:0/1189957586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd50ecd170 0x55bd50ed2200 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f6feffff640 0 client.18684 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 >> 172.21.15.87:0/1189957586 conn(0x55bd50ec9680 msgr2=0x55bd50ed51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 shutdown_connections 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 1 -- 172.21.15.87:0/1189957586 wait complete. 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f70199d5740 20 client.18684 trim_cache size 0 max 16384 2024-06-06T08:12:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:12:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:12:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:12:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 msgr2=0x7f6ff80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 0x7f6ff80a5260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7008004ab0 tx=0x7f7008031430 comp rx=0 tx=0).stop 2024-06-06T08:12:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 msgr2=0x7f70101986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f700400d770 tx=0x7f700400dc30 comp rx=0 tx=0).stop 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 shutdown_connections 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6ff80a2da0 0x7f6ff80a5260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70101068d0 0x7f701019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7010101280 0x7f70101986b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 --2- 172.21.15.87:0/4212050870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101008c0 0x7f7010198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 >> 172.21.15.87:0/4212050870 conn(0x7f7010076c20 msgr2=0x7f701006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.962+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 shutdown_connections 2024-06-06T08:12:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:01.963+0000 7f7016be7640 1 -- 172.21.15.87:0/4212050870 wait complete. 2024-06-06T08:12:02.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119839 10.47 MB/sec execute 484 sec latency 1916.782 ms 2024-06-06T08:12:02.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:02 smithi181 ceph-mon[36790]: pgmap v762: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-06-06T08:12:02.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:02 smithi087 ceph-mon[32276]: pgmap v762: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-06-06T08:12:02.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:02 smithi195 ceph-mon[37040]: pgmap v762: 129 pgs: 129 active+clean; 671 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-06-06T08:12:03.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 119951 10.47 MB/sec execute 485 sec latency 2659.218 ms 2024-06-06T08:12:04.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120128 10.46 MB/sec execute 486 sec latency 307.325 ms 2024-06-06T08:12:04.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:04 smithi181 ceph-mon[36790]: pgmap v763: 129 pgs: 129 active+clean; 668 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.9 MiB/s wr, 420 op/s 2024-06-06T08:12:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:04 smithi087 ceph-mon[32276]: pgmap v763: 129 pgs: 129 active+clean; 668 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.9 MiB/s wr, 420 op/s 2024-06-06T08:12:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:04 smithi195 ceph-mon[37040]: pgmap v763: 129 pgs: 129 active+clean; 668 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.9 MiB/s wr, 420 op/s 2024-06-06T08:12:05.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120236 10.44 MB/sec execute 487 sec latency 1020.672 ms 2024-06-06T08:12:06.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120274 10.42 MB/sec execute 488 sec latency 2020.716 ms 2024-06-06T08:12:06.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:06 smithi181 ceph-mon[36790]: pgmap v764: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-06-06T08:12:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:06 smithi087 ceph-mon[32276]: pgmap v764: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-06-06T08:12:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:06 smithi195 ceph-mon[37040]: pgmap v764: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-06-06T08:12:07.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120312 10.41 MB/sec execute 489 sec latency 3020.761 ms 2024-06-06T08:12:07.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:07 smithi181 ceph-mon[36790]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T08:12:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:07 smithi087 ceph-mon[32276]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T08:12:07.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:07 smithi195 ceph-mon[37040]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T08:12:08.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120397 10.41 MB/sec execute 490 sec latency 3847.480 ms 2024-06-06T08:12:08.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:08 smithi181 ceph-mon[36790]: pgmap v766: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-06-06T08:12:08.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:08 smithi087 ceph-mon[32276]: pgmap v766: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-06-06T08:12:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:08 smithi195 ceph-mon[37040]: pgmap v766: 129 pgs: 129 active+clean; 670 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-06-06T08:12:09.196 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120453 10.39 MB/sec execute 491 sec latency 1262.118 ms 2024-06-06T08:12:10.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120564 10.38 MB/sec execute 492 sec latency 1400.071 ms 2024-06-06T08:12:10.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:10 smithi181 ceph-mon[36790]: pgmap v767: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:10.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:10 smithi087 ceph-mon[32276]: pgmap v767: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:10 smithi195 ceph-mon[37040]: pgmap v767: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:11.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120681 10.37 MB/sec execute 493 sec latency 1447.388 ms 2024-06-06T08:12:12.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120822 10.37 MB/sec execute 494 sec latency 1104.955 ms 2024-06-06T08:12:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:12 smithi087 ceph-mon[32276]: pgmap v768: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:12.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:12 smithi195 ceph-mon[37040]: pgmap v768: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:12.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:12 smithi181 ceph-mon[36790]: pgmap v768: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-06-06T08:12:13.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 120938 10.35 MB/sec execute 495 sec latency 1084.693 ms 2024-06-06T08:12:14.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 121160 10.36 MB/sec execute 496 sec latency 654.601 ms 2024-06-06T08:12:14.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:14 smithi087 ceph-mon[32276]: pgmap v769: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 658 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-06-06T08:12:14.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:14 smithi195 ceph-mon[37040]: pgmap v769: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 658 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-06-06T08:12:14.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:14 smithi181 ceph-mon[36790]: pgmap v769: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 658 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-06-06T08:12:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.063+0000 7fcc28038640 1 -- 172.21.15.87:0/1333407654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 msgr2=0x7fcc20100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.063+0000 7fcc28038640 1 --2- 172.21.15.87:0/1333407654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc20100d50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009930 tx=0x7fcc1002a420 comp rx=0 tx=0).stop 2024-06-06T08:12:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 -- 172.21.15.87:0/1333407654 shutdown_connections 2024-06-06T08:12:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 --2- 172.21.15.87:0/1333407654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20101290 0x7fcc2010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 --2- 172.21.15.87:0/1333407654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc20100d50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 --2- 172.21.15.87:0/1333407654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20106890 0x7fcc20106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 -- 172.21.15.87:0/1333407654 >> 172.21.15.87:0/1333407654 conn(0x7fcc20077a20 msgr2=0x7fcc20077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 -- 172.21.15.87:0/1333407654 shutdown_connections 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 -- 172.21.15.87:0/1333407654 wait complete. 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.064+0000 7fcc28038640 1 Processor -- start 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 -- start start 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20106890 0x7fcc2019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20110ca0 con 0x7fcc201008d0 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20110b20 con 0x7fcc20106890 2024-06-06T08:12:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc28038640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20110e20 con 0x7fcc20101290 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:60774/0 (socket says 172.21.15.87:60774) 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48546/0 (socket says 172.21.15.87:48546) 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc255d3640 1 -- 172.21.15.87:0/3235791300 learned_addr learned my addr 172.21.15.87:0/3235791300 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc265d5640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20106890 0x7fcc2019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 -- 172.21.15.87:0/3235791300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 msgr2=0x7fcc20197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 -- 172.21.15.87:0/3235791300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20106890 msgr2=0x7fcc2019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20106890 0x7fcc2019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc25dd4640 1 -- 172.21.15.87:0/3235791300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2019c0b0 con 0x7fcc201008d0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.065+0000 7fcc255d3640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc25dd4640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009bc0 tx=0x7fcc1c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc16ffd640 1 -- 172.21.15.87:0/3235791300 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c01e070 con 0x7fcc201008d0 2024-06-06T08:12:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2019c3a0 con 0x7fcc201008d0 2024-06-06T08:12:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc16ffd640 1 -- 172.21.15.87:0/3235791300 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc1c0198f0 con 0x7fcc201008d0 2024-06-06T08:12:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc16ffd640 1 -- 172.21.15.87:0/3235791300 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0187f0 con 0x7fcc201008d0 2024-06-06T08:12:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.066+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc20069050 con 0x7fcc201008d0 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.069+0000 7fcc2ae21740 1 Processor -- start 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.069+0000 7fcc2ae21740 1 -- start start 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.069+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69a90 0x55bd6ab73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bd6aacd4f0 con 0x55bd6ab69170 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd6aa6b870 con 0x55bd6ab69a90 2024-06-06T08:12:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd6aac94b0 con 0x55bd6ab74350 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48556/0 (socket says 172.21.15.87:48556) 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 -- 172.21.15.87:0/4110499202 learned_addr learned my addr 172.21.15.87:0/4110499202 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc16ffd640 1 -- 172.21.15.87:0/3235791300 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcc1c021070 con 0x7fcc201008d0 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 -- 172.21.15.87:0/4110499202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 msgr2=0x55bd6ab76740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab76740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 -- 172.21.15.87:0/4110499202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69a90 msgr2=0x55bd6ab73e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69a90 0x55bd6ab73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 -- 172.21.15.87:0/4110499202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6aa879b0 con 0x55bd6ab69170 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.070+0000 7fcc265d5640 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fcc18010370 tx=0x7fcc18010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f96bdc524d7801 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.071+0000 7fcc16ffd640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 0x7fcc0c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.071+0000 7fcc25dd4640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 0x7fcc0c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.071+0000 7fcc16ffd640 1 -- 172.21.15.87:0/3235791300 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9671+0+0 (secure 0 0 0) 0x7fcc1c0d3030 con 0x7fcc201008d0 2024-06-06T08:12:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.072+0000 7fcc25dd4640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 0x7fcc0c0a5260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55bd6ab6a710 tx=0x7fcc1c00bef0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.073+0000 7fcc0bfff640 1 -- 172.21.15.87:0/4110499202 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18016070 con 0x55bd6ab69170 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.073+0000 7fcc0bfff640 1 -- 172.21.15.87:0/4110499202 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc18004d20 con 0x55bd6ab69170 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.073+0000 7fcc0bfff640 1 -- 172.21.15.87:0/4110499202 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18005480 con 0x55bd6ab69170 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- 172.21.15.87:0/4110499202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 msgr2=0x55bd6ab69550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fcc18010370 tx=0x7fcc18010830 comp rx=0 tx=0).stop 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- 172.21.15.87:0/4110499202 shutdown_connections 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69a90 0x55bd6ab73e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/4110499202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69170 0x55bd6ab69550 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- 172.21.15.87:0/4110499202 >> 172.21.15.87:0/4110499202 conn(0x55bd6ab65680 msgr2=0x55bd6ab65a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- 172.21.15.87:0/4110499202 shutdown_connections 2024-06-06T08:12:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- 172.21.15.87:0/4110499202 wait complete. 2024-06-06T08:12:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.074+0000 7fcc2ae21740 1 -- start start 2024-06-06T08:12:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.075+0000 7fcc2ae21740 1 Processor -- start 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.075+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.075+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 0x55bd6ab71850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.075+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab74350 0x55bd6ab71d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.075+0000 7fcc265d5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:34828/0 (socket says 172.21.15.87:34828) 2024-06-06T08:12:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 0x55bd6ab71850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 0x55bd6ab71850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:60778/0 (socket says 172.21.15.87:60778) 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 -- 172.21.15.87:0/1332298246 learned_addr learned my addr 172.21.15.87:0/1332298246 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55bd6ab76f80 con 0x55bd6ab74350 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd6aa56ae0 con 0x55bd6ab69170 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd6ab77100 con 0x55bd6ab69a90 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab74350 0x55bd6ab71d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc2ae21740 10 client.0 _ll_register_callbacks cb 0x55bd6aa8f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 -- 172.21.15.87:0/1332298246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 msgr2=0x55bd6ab71850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 0x55bd6ab71850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 -- 172.21.15.87:0/1332298246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab74350 msgr2=0x55bd6ab71d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab74350 0x55bd6ab71d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc265d5640 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6a7e1210 con 0x55bd6ab69170 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.076+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc18004d20 con 0x55bd6ab69170 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc265d5640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc180057e0 tx=0x7fcc18009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc65e837db6f97b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc097fa640 10 client.27533 ms_handle_connect on v2:172.21.15.181:3300/0 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc097fa640 1 -- 172.21.15.87:0/1332298246 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18016070 con 0x55bd6ab69170 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc097fa640 1 -- 172.21.15.87:0/1332298246 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9671+0+0 (secure 0 0 0) 0x7fcc18003970 con 0x55bd6ab69170 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc097fa640 1 -- 172.21.15.87:0/1332298246 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcc18023040 con 0x55bd6ab69170 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc2ae21740 10 client.27533 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.077+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=55}) -- 0x55bd6a8ad620 con 0x55bd6ab69170 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc097fa640 1 -- 172.21.15.87:0/1332298246 <== mon.2 v2:172.21.15.181:3300/0 4 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcc1800f040 con 0x55bd6ab69170 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 10 client.27533 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 10 client.27533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 20 client.27533 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6ab841f0 0x55bd6ab865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 4 client.27533 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd6aae0f10 con 0x55bd6ab841f0 2024-06-06T08:12:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.078+0000 7fcc255d3640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6ab841f0 0x55bd6ab865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.080+0000 7fcc255d3640 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6ab841f0 0x55bd6ab865e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a09042003bbf7a server_cookie=aecc21f67c65b53e in_seq=0 out_seq=0 2024-06-06T08:12:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.080+0000 7fcc097fa640 10 client.27533 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc097fa640 1 -- 172.21.15.87:0/1332298246 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bd6aae0f10 con 0x55bd6ab841f0 2024-06-06T08:12:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc097fa640 10 client.27533 handle_command_reply: tid=0 2024-06-06T08:12:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 client.27533 shutdown 2024-06-06T08:12:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 msgr2=0x55bd6ab71310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc180057e0 tx=0x7fcc18009bb0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 shutdown_connections 2024-06-06T08:12:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6ab841f0 0x55bd6ab865e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab74350 0x55bd6ab71d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc097fa640 0 client.27533 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69a90 0x55bd6ab71850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/1332298246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab71310 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 >> 172.21.15.87:0/1332298246 conn(0x55bd6ab65680 msgr2=0x55bd6ab6b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 shutdown_connections 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.083+0000 7fcc2ae21740 1 -- 172.21.15.87:0/1332298246 wait complete. 2024-06-06T08:12:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.084+0000 7fcc2ae21740 20 client.27533 trim_cache size 0 max 16384 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.100+0000 7fcc2ae21740 1 Processor -- start 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.100+0000 7fcc2ae21740 1 -- start start 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.100+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69170 0x55bd6ab6bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.100+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab74350 0x55bd6ab8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201af7e0 con 0x55bd6ab69a90 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201ad0b0 con 0x55bd6ab74350 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201aaf20 con 0x55bd6ab69170 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48576/0 (socket says 172.21.15.87:48576) 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc265d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69170 0x55bd6ab6bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 -- 172.21.15.87:0/299510372 learned_addr learned my addr 172.21.15.87:0/299510372 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc265d5640 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab74350 0x55bd6ab8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 -- 172.21.15.87:0/299510372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69170 msgr2=0x55bd6ab6bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69170 0x55bd6ab6bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 -- 172.21.15.87:0/299510372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab74350 msgr2=0x55bd6ab8c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab74350 0x55bd6ab8c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 -- 172.21.15.87:0/299510372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6aa700d0 con 0x55bd6ab69a90 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc255d3640 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009a50 tx=0x7fcc10004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620221036b83c0a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc0b7fe640 1 -- 172.21.15.87:0/299510372 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10035040 con 0x55bd6ab69a90 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc0b7fe640 1 -- 172.21.15.87:0/299510372 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc10005500 con 0x55bd6ab69a90 2024-06-06T08:12:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.101+0000 7fcc0b7fe640 1 -- 172.21.15.87:0/299510372 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10031650 con 0x55bd6ab69a90 2024-06-06T08:12:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 -- 172.21.15.87:0/299510372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 msgr2=0x55bd6ab8bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009a50 tx=0x7fcc10004180 comp rx=0 tx=0).stop 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 -- 172.21.15.87:0/299510372 shutdown_connections 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab74350 0x55bd6ab8c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab8bd00 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/299510372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab69170 0x55bd6ab6bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 -- 172.21.15.87:0/299510372 >> 172.21.15.87:0/299510372 conn(0x55bd6ab65680 msgr2=0x55bd6ab647e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 -- 172.21.15.87:0/299510372 shutdown_connections 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.102+0000 7fcc2ae21740 1 -- 172.21.15.87:0/299510372 wait complete. 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 -- start start 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 Processor -- start 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab7c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab6e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201ab570 con 0x55bd6ab69a90 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201ac9e0 con 0x55bd6ab69170 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201abc40 con 0x55bd6ab74350 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc2ae21740 10 client.0 _ll_register_callbacks cb 0x7fcc18009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc255d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48592/0 (socket says 172.21.15.87:48592) 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc255d3640 1 -- 172.21.15.87:0/3871687403 learned_addr learned my addr 172.21.15.87:0/3871687403 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.103+0000 7fcc265d5640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab6e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc265d5640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab7c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 -- 172.21.15.87:0/3871687403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 msgr2=0x55bd6ab6e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab6e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 -- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 msgr2=0x55bd6ab7c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab7c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 -- 172.21.15.87:0/3871687403 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6ab6e7f0 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc255d3640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fcc10004b00 tx=0x7fcc10004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ba4293a1ec885a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc097fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10056020 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9671+0+0 (secure 0 0 0) 0x7fcc10005020 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.104+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc10005500 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcc10061050 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 10 client.18702 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55bd6abee990 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc10005500 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fcc10031cd0 con 0x55bd6ab69a90 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 10 client.18702 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 20 client.18702 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6abef050 0x55bd6abf1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 4 client.18702 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:12:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd6aadac20 con 0x55bd6abef050 2024-06-06T08:12:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.105+0000 7fcc265d5640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6abef050 0x55bd6abf1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.106+0000 7fcc097fa640 10 client.18702 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.106+0000 7fcc265d5640 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6abef050 0x55bd6abf1470 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf13f971149b74b server_cookie=5806297e99051af4 in_seq=0 out_seq=0 2024-06-06T08:12:15.201 INFO:tasks.workunit.client.0.smithi087.stdout: 10 121423 10.37 MB/sec execute 497 sec latency 372.754 ms 2024-06-06T08:12:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.849+0000 7fcc097fa640 1 -- 172.21.15.87:0/3871687403 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd6aadac20 con 0x55bd6abef050 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.849+0000 7fcc097fa640 10 client.18702 handle_command_reply: tid=0 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.849+0000 7fcc2ae21740 1 client.18702 shutdown 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 msgr2=0x55bd6ab6dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fcc10004b00 tx=0x7fcc10004920 comp rx=0 tx=0).stop 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 shutdown_connections 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55bd6abef050 0x55bd6abf1470 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd6ab74350 0x55bd6ab6e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bd6ab69a90 0x55bd6ab6dd70 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 --2- 172.21.15.87:0/3871687403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd6ab69170 0x55bd6ab7c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 >> 172.21.15.87:0/3871687403 conn(0x55bd6ab65680 msgr2=0x55bd6ab712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc097fa640 0 client.18702 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 shutdown_connections 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 1 -- 172.21.15.87:0/3871687403 wait complete. 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.850+0000 7fcc2ae21740 20 client.18702 trim_cache size 0 max 16384 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:12:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:12:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 msgr2=0x7fcc0c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 0x7fcc0c0a5260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55bd6ab6a710 tx=0x7fcc1c00bef0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 msgr2=0x7fcc201970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009bc0 tx=0x7fcc1c00b690 comp rx=0 tx=0).stop 2024-06-06T08:12:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 shutdown_connections 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcc0c0a2da0 0x7fcc0c0a5260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc20106890 0x7fcc2019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc20101290 0x7fcc20197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 --2- 172.21.15.87:0/3235791300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc201008d0 0x7fcc201970c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 >> 172.21.15.87:0/3235791300 conn(0x7fcc20077a20 msgr2=0x7fcc20103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 shutdown_connections 2024-06-06T08:12:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:15.851+0000 7fcc28038640 1 -- 172.21.15.87:0/3235791300 wait complete. 2024-06-06T08:12:16.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 121503 10.36 MB/sec execute 498 sec latency 732.931 ms 2024-06-06T08:12:16.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:16 smithi087 ceph-mon[32276]: pgmap v770: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-06-06T08:12:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:16 smithi195 ceph-mon[37040]: pgmap v770: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-06-06T08:12:16.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:16 smithi181 ceph-mon[36790]: pgmap v770: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-06-06T08:12:17.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 121799 10.38 MB/sec execute 499 sec latency 395.541 ms 2024-06-06T08:12:18.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 122105 10.38 MB/sec execute 500 sec latency 391.918 ms 2024-06-06T08:12:18.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:18 smithi087 ceph-mon[32276]: pgmap v771: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-06-06T08:12:18.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:18 smithi195 ceph-mon[37040]: pgmap v771: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-06-06T08:12:18.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:18 smithi181 ceph-mon[36790]: pgmap v771: 129 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 116 active+clean; 656 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-06-06T08:12:19.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 122368 10.39 MB/sec execute 501 sec latency 359.757 ms 2024-06-06T08:12:20.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 122709 10.40 MB/sec execute 502 sec latency 405.824 ms 2024-06-06T08:12:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:20 smithi087 ceph-mon[32276]: pgmap v772: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-06T08:12:20.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:20 smithi087 ceph-mon[32276]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T08:12:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:20 smithi195 ceph-mon[37040]: pgmap v772: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-06T08:12:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:20 smithi195 ceph-mon[37040]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T08:12:20.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:20 smithi181 ceph-mon[36790]: pgmap v772: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-06-06T08:12:20.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:20 smithi181 ceph-mon[36790]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T08:12:21.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 122996 10.42 MB/sec execute 503 sec latency 488.370 ms 2024-06-06T08:12:22.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 123245 10.42 MB/sec execute 504 sec latency 409.507 ms 2024-06-06T08:12:22.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:22 smithi087 ceph-mon[32276]: pgmap v774: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-06-06T08:12:22.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:22 smithi195 ceph-mon[37040]: pgmap v774: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-06-06T08:12:22.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:22 smithi181 ceph-mon[36790]: pgmap v774: 129 pgs: 129 active+clean; 651 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-06-06T08:12:23.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 123486 10.42 MB/sec execute 505 sec latency 591.353 ms 2024-06-06T08:12:24.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 123837 10.44 MB/sec execute 506 sec latency 431.040 ms 2024-06-06T08:12:24.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:24 smithi087 ceph-mon[32276]: pgmap v775: 129 pgs: 129 active+clean; 667 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-06-06T08:12:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:24 smithi195 ceph-mon[37040]: pgmap v775: 129 pgs: 129 active+clean; 667 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-06-06T08:12:24.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:24 smithi181 ceph-mon[36790]: pgmap v775: 129 pgs: 129 active+clean; 667 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-06-06T08:12:25.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124137 10.47 MB/sec execute 507 sec latency 395.404 ms 2024-06-06T08:12:26.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124349 10.47 MB/sec execute 508 sec latency 454.933 ms 2024-06-06T08:12:26.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:26 smithi087 ceph-mon[32276]: pgmap v776: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:26 smithi195 ceph-mon[37040]: pgmap v776: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:26.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:26 smithi181 ceph-mon[36790]: pgmap v776: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:27.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124546 10.47 MB/sec execute 509 sec latency 545.699 ms 2024-06-06T08:12:28.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124658 10.46 MB/sec execute 510 sec latency 667.424 ms 2024-06-06T08:12:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:28 smithi087 ceph-mon[32276]: pgmap v777: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:28 smithi195 ceph-mon[37040]: pgmap v777: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:28.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:28 smithi181 ceph-mon[36790]: pgmap v777: 129 pgs: 129 active+clean; 676 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.948+0000 7f4a50ef7640 1 -- 172.21.15.87:0/4128202298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c101290 msgr2=0x7f4a4c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.948+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/4128202298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c101290 0x7f4a4c10d750 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009cd0 tx=0x7f4a4402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 -- 172.21.15.87:0/4128202298 shutdown_connections 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/4128202298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c101290 0x7f4a4c10d750 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/4128202298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/4128202298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 0x7f4a4c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 -- 172.21.15.87:0/4128202298 >> 172.21.15.87:0/4128202298 conn(0x7f4a4c077a20 msgr2=0x7f4a4c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 -- 172.21.15.87:0/4128202298 shutdown_connections 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 -- 172.21.15.87:0/4128202298 wait complete. 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 Processor -- start 2024-06-06T08:12:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.949+0000 7f4a50ef7640 1 -- start start 2024-06-06T08:12:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c101290 0x7f4a4c0fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 0x7f4a4c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110cf0 con 0x7f4a4c1008d0 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110b70 con 0x7f4a4c101290 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110e70 con 0x7f4a4c106890 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33124/0 (socket says 172.21.15.87:33124) 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2546329003 learned_addr learned my addr 172.21.15.87:0/2546329003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c101290 0x7f4a4c0fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 0x7f4a4c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2546329003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 msgr2=0x7f4a4c0fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 0x7f4a4c0fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2546329003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c101290 msgr2=0x7f4a4c0fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c101290 0x7f4a4c0fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2546329003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c1aabc0 con 0x7f4a4c1008d0 2024-06-06T08:12:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9f0 tx=0x7f4a3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a3b7fe640 1 -- 172.21.15.87:0/2546329003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00cd80 con 0x7f4a4c1008d0 2024-06-06T08:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c1aadf0 con 0x7f4a4c1008d0 2024-06-06T08:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a3b7fe640 1 -- 172.21.15.87:0/2546329003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c005500 con 0x7f4a4c1008d0 2024-06-06T08:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a3b7fe640 1 -- 172.21.15.87:0/2546329003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c010690 con 0x7f4a4c1008d0 2024-06-06T08:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.950+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1ab260 con 0x7f4a4c1008d0 2024-06-06T08:12:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.953+0000 7f4a53bfd740 1 Processor -- start 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.953+0000 7f4a3b7fe640 1 -- 172.21.15.87:0/2546329003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4a3c0046c0 con 0x7f4a4c1008d0 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.953+0000 7f4a53bfd740 1 -- start start 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fc9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fd3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fd6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b09f2d4f0 con 0x564b09fd4350 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b09ecb870 con 0x564b09fc9a90 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564b09f294b0 con 0x564b09fc9170 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fc9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fd6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fd3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fc9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:58190/0 (socket says 172.21.15.87:58190) 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/3764329522 learned_addr learned my addr 172.21.15.87:0/3764329522 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a3b7fe640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 0x7f4a340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 -- 172.21.15.87:0/3764329522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 msgr2=0x564b09fc9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fc9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 -- 172.21.15.87:0/3764329522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 msgr2=0x564b09fd3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fd3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 -- 172.21.15.87:0/3764329522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b09ee79b0 con 0x564b09fd4350 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 0x7f4a340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a3b7fe640 1 -- 172.21.15.87:0/2546329003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f4a3c0cfc20 con 0x7f4a4c1008d0 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fd6740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0ff0b0 tx=0x7f4a40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d8f9d8669568cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a38ff9640 1 -- 172.21.15.87:0/3764329522 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40004050 con 0x564b09fd4350 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a38ff9640 1 -- 172.21.15.87:0/3764329522 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a40005500 con 0x564b09fd4350 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.954+0000 7f4a38ff9640 1 -- 172.21.15.87:0/3764329522 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40004de0 con 0x564b09fd4350 2024-06-06T08:12:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 0x7f4a340a5260 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a4402aed0 tx=0x7f4a440095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 -- 172.21.15.87:0/3764329522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 msgr2=0x564b09fd6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fd6740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0ff0b0 tx=0x7f4a40011810 comp rx=0 tx=0).stop 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 -- 172.21.15.87:0/3764329522 shutdown_connections 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fd6740 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/3764329522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fc9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 -- 172.21.15.87:0/3764329522 >> 172.21.15.87:0/3764329522 conn(0x564b09fc5680 msgr2=0x564b09fc5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 -- 172.21.15.87:0/3764329522 shutdown_connections 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.955+0000 7f4a53bfd740 1 -- 172.21.15.87:0/3764329522 wait complete. 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 -- start start 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 Processor -- start 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9a90 0x564b09fd1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fd4350 0x564b09fd1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b09fd6f80 con 0x564b09fc9170 2024-06-06T08:12:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564b09eb6ae0 con 0x564b09fd4350 2024-06-06T08:12:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564b09fd7100 con 0x564b09fc9a90 2024-06-06T08:12:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33150/0 (socket says 172.21.15.87:33150) 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2469303899 learned_addr learned my addr 172.21.15.87:0/2469303899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.956+0000 7f4a53bfd740 10 client.0 _ll_register_callbacks cb 0x564b09eef9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9a90 0x564b09fd1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fd4350 0x564b09fd1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2469303899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9a90 msgr2=0x564b09fd1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9a90 0x564b09fd1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fd4350 msgr2=0x564b09fd1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fd4350 0x564b09fd1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 -- 172.21.15.87:0/2469303899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b09c41210 con 0x564b09fc9170 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c6a0 tx=0x7f4a3c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1fa1c304ef51c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c09b860 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f4a3c0a4700 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a40005500 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4a3c09b680 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a53bfd740 10 client.18714 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.957+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564b09d0d620 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a40005500 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f4a3c0d5030 con 0x564b09fc9170 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 10 client.18714 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 10 client.18714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 20 client.18714 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b09fe4580 0x564b09fe6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 4 client.18714 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:12:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b09f40f10 con 0x564b09fe4580 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b09fe4580 0x564b09fe6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a31ffb640 10 client.18714 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.958+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b09fe4580 0x564b09fe6970 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313579d716cef2ec server_cookie=1e79f2b936757709 in_seq=0 out_seq=0 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.961+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2469303899 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b09f40f10 con 0x564b09fe4580 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.961+0000 7f4a31ffb640 10 client.18714 handle_command_reply: tid=0 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 client.18714 shutdown 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 msgr2=0x564b09fd1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c6a0 tx=0x7f4a3c009310 comp rx=0 tx=0).stop 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 shutdown_connections 2024-06-06T08:12:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b09fe4580 0x564b09fe6970 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fd4350 0x564b09fd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9a90 0x564b09fd1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2469303899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fc9170 0x564b09fd1320 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a31ffb640 0 client.18714 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 >> 172.21.15.87:0/2469303899 conn(0x564b09fc5680 msgr2=0x564b09fcb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 shutdown_connections 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2469303899 wait complete. 2024-06-06T08:12:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.962+0000 7f4a53bfd740 20 client.18714 trim_cache size 0 max 16384 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 Processor -- start 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 -- start start 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b7ee0 con 0x564b09fd4350 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b57b0 con 0x564b09fc9a90 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b3620 con 0x564b09fc9170 2024-06-06T08:12:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33164/0 (socket says 172.21.15.87:33164) 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/1958740304 learned_addr learned my addr 172.21.15.87:0/1958740304 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.978+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 -- 172.21.15.87:0/1958740304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 msgr2=0x564b09fec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 -- 172.21.15.87:0/1958740304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 msgr2=0x564b09fce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 -- 172.21.15.87:0/1958740304 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b09fc98c0 con 0x564b09fc9170 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fec240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004c00 tx=0x7f4a3c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9514694c89cafa13 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a33fff640 1 -- 172.21.15.87:0/1958740304 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0a9070 con 0x564b09fc9170 2024-06-06T08:12:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a33fff640 1 -- 172.21.15.87:0/1958740304 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c0d3460 con 0x564b09fc9170 2024-06-06T08:12:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.979+0000 7f4a33fff640 1 -- 172.21.15.87:0/1958740304 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c09b4a0 con 0x564b09fc9170 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 -- 172.21.15.87:0/1958740304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 msgr2=0x564b09fec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fec240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004c00 tx=0x7f4a3c004a40 comp rx=0 tx=0).stop 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 -- 172.21.15.87:0/1958740304 shutdown_connections 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/1958740304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fec240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 -- 172.21.15.87:0/1958740304 >> 172.21.15.87:0/1958740304 conn(0x564b09fc5680 msgr2=0x564b09fcc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 -- 172.21.15.87:0/1958740304 shutdown_connections 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.980+0000 7f4a53bfd740 1 -- 172.21.15.87:0/1958740304 wait complete. 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 -- start start 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 Processor -- start 2024-06-06T08:12:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fdc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fdcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b3c70 con 0x564b09fd4350 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b50e0 con 0x564b09fc9a90 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b4340 con 0x564b09fc9170 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a53bfd740 10 client.0 _ll_register_callbacks cb 0x7f4a3c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33166/0 (socket says 172.21.15.87:33166) 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/2124090445 learned_addr learned my addr 172.21.15.87:0/2124090445 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fdc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a49d9b640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fdcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/2124090445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 msgr2=0x564b09fdc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fdc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 msgr2=0x564b09fdcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fdcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.981+0000 7f4a4ad9d640 1 -- 172.21.15.87:0/2124090445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b09fce390 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a4ad9d640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a44033040 tx=0x7f4a4402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8de1f3f3a35a3bd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44069060 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f4a440305a0 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a3c0d3460 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4a440307a0 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a53bfd740 10 client.18720 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.982+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564b0a04ed00 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c0d3460 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f4a44045070 con 0x564b09fd4350 2024-06-06T08:12:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 10 client.18720 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 10 client.18720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 20 client.18720 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b0a04f480 0x564b0a0518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 4 client.18720 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b09f3ac20 con 0x564b0a04f480 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.983+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b0a04f480 0x564b0a0518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.984+0000 7f4a31ffb640 10 client.18720 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:28.984+0000 7f4a4a59c640 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b0a04f480 0x564b0a0518a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3224d94cd9d9edfe server_cookie=9e70e93113fa2ae0 in_seq=0 out_seq=0 2024-06-06T08:12:29.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124856 10.45 MB/sec execute 511 sec latency 378.011 ms 2024-06-06T08:12:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a31ffb640 1 -- 172.21.15.87:0/2124090445 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564b09f3ac20 con 0x564b0a04f480 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a31ffb640 10 client.18720 handle_command_reply: tid=0 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 client.18720 shutdown 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 msgr2=0x564b09fcdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a44033040 tx=0x7f4a4402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 shutdown_connections 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x564b0a04f480 0x564b0a0518a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b09fd4350 0x564b09fcdbb0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a31ffb640 0 client.18720 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b09fc9a90 0x564b09fdcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 --2- 172.21.15.87:0/2124090445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564b09fc9170 0x564b09fdc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.816+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 >> 172.21.15.87:0/2124090445 conn(0x564b09fc5680 msgr2=0x564b09fcab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 shutdown_connections 2024-06-06T08:12:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a53bfd740 1 -- 172.21.15.87:0/2124090445 wait complete. 2024-06-06T08:12:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a53bfd740 20 client.18720 trim_cache size 0 max 16384 2024-06-06T08:12:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:12:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:12:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 msgr2=0x7f4a340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 0x7f4a340a5260 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a4402aed0 tx=0x7f4a440095c0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 msgr2=0x7f4a4c0fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9f0 tx=0x7f4a3c00edf0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 shutdown_connections 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f4a340a2da0 0x7f4a340a5260 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4c106890 0x7f4a4c0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a4c101290 0x7f4a4c0fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 --2- 172.21.15.87:0/2546329003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4c1008d0 0x7f4a4c0fd860 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 >> 172.21.15.87:0/2546329003 conn(0x7f4a4c077a20 msgr2=0x7f4a4c069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 shutdown_connections 2024-06-06T08:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:29.817+0000 7f4a50ef7640 1 -- 172.21.15.87:0/2546329003 wait complete. 2024-06-06T08:12:30.197 INFO:tasks.workunit.client.0.smithi087.stdout: 10 124968 10.43 MB/sec execute 512 sec latency 662.474 ms 2024-06-06T08:12:30.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:30 smithi087 ceph-mon[32276]: pgmap v778: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-06-06T08:12:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:30 smithi195 ceph-mon[37040]: pgmap v778: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-06-06T08:12:30.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:30 smithi181 ceph-mon[36790]: pgmap v778: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-06-06T08:12:31.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 125292 10.46 MB/sec execute 513 sec latency 253.427 ms 2024-06-06T08:12:32.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 125551 10.46 MB/sec execute 514 sec latency 204.144 ms 2024-06-06T08:12:32.357 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:32 smithi181 ceph-mon[36790]: pgmap v779: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 259 op/s 2024-06-06T08:12:32.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:32 smithi087 ceph-mon[32276]: pgmap v779: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 259 op/s 2024-06-06T08:12:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:32 smithi195 ceph-mon[37040]: pgmap v779: 129 pgs: 129 active+clean; 697 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 259 op/s 2024-06-06T08:12:33.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 125832 10.46 MB/sec execute 515 sec latency 271.861 ms 2024-06-06T08:12:34.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 126107 10.47 MB/sec execute 516 sec latency 296.261 ms 2024-06-06T08:12:34.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:34 smithi087 ceph-mon[32276]: pgmap v780: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-06-06T08:12:34.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:34 smithi195 ceph-mon[37040]: pgmap v780: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-06-06T08:12:34.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:34 smithi181 ceph-mon[36790]: pgmap v780: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.0 MiB/s wr, 422 op/s 2024-06-06T08:12:35.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 126519 10.50 MB/sec execute 517 sec latency 372.946 ms 2024-06-06T08:12:36.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 126784 10.52 MB/sec execute 518 sec latency 410.798 ms 2024-06-06T08:12:36.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:36 smithi087 ceph-mon[32276]: pgmap v781: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 465 op/s 2024-06-06T08:12:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:36 smithi195 ceph-mon[37040]: pgmap v781: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 465 op/s 2024-06-06T08:12:36.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:36 smithi181 ceph-mon[36790]: pgmap v781: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 465 op/s 2024-06-06T08:12:37.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 127008 10.51 MB/sec execute 519 sec latency 170.240 ms 2024-06-06T08:12:38.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 127256 10.51 MB/sec execute 520 sec latency 269.866 ms 2024-06-06T08:12:38.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:38 smithi087 ceph-mon[32276]: pgmap v782: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 430 op/s 2024-06-06T08:12:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:38 smithi195 ceph-mon[37040]: pgmap v782: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 430 op/s 2024-06-06T08:12:38.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:38 smithi181 ceph-mon[36790]: pgmap v782: 129 pgs: 129 active+clean; 693 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 430 op/s 2024-06-06T08:12:39.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 127573 10.53 MB/sec execute 521 sec latency 395.142 ms 2024-06-06T08:12:40.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 127908 10.54 MB/sec execute 522 sec latency 183.167 ms 2024-06-06T08:12:40.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:40 smithi087 ceph-mon[32276]: pgmap v783: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.5 MiB/s wr, 546 op/s 2024-06-06T08:12:40.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:40 smithi195 ceph-mon[37040]: pgmap v783: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.5 MiB/s wr, 546 op/s 2024-06-06T08:12:40.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:40 smithi181 ceph-mon[36790]: pgmap v783: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.5 MiB/s wr, 546 op/s 2024-06-06T08:12:41.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 128191 10.55 MB/sec execute 523 sec latency 240.933 ms 2024-06-06T08:12:42.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 128475 10.56 MB/sec execute 524 sec latency 197.708 ms 2024-06-06T08:12:42.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:42 smithi087 ceph-mon[32276]: pgmap v784: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-06-06T08:12:42.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:42 smithi195 ceph-mon[37040]: pgmap v784: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-06-06T08:12:42.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:42 smithi181 ceph-mon[36790]: pgmap v784: 129 pgs: 129 active+clean; 708 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-06-06T08:12:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 -- 172.21.15.87:0/1733845684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc1008d0 msgr2=0x7f81bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 --2- 172.21.15.87:0/1733845684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc1008d0 0x7f81bc100d50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009930 tx=0x7f81b002a420 comp rx=0 tx=0).stop 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 -- 172.21.15.87:0/1733845684 shutdown_connections 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 --2- 172.21.15.87:0/1733845684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc101290 0x7f81bc10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 --2- 172.21.15.87:0/1733845684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc1008d0 0x7f81bc100d50 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.921+0000 7f81c27b7640 1 --2- 172.21.15.87:0/1733845684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc106890 0x7f81bc106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.922+0000 7f81c27b7640 1 -- 172.21.15.87:0/1733845684 >> 172.21.15.87:0/1733845684 conn(0x7f81bc077a20 msgr2=0x7f81bc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.922+0000 7f81c27b7640 1 -- 172.21.15.87:0/1733845684 shutdown_connections 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.922+0000 7f81c27b7640 1 -- 172.21.15.87:0/1733845684 wait complete. 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.922+0000 7f81c27b7640 1 Processor -- start 2024-06-06T08:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.922+0000 7f81c27b7640 1 -- start start 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc1008d0 0x7f81bc197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc101290 0x7f81bc197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc110ca0 con 0x7f81bc106890 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc110b20 con 0x7f81bc101290 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc110e20 con 0x7f81bc1008d0 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45546/0 (socket says 172.21.15.87:45546) 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 -- 172.21.15.87:0/775688142 learned_addr learned my addr 172.21.15.87:0/775688142 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc101290 0x7f81bc197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81bbfff640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc1008d0 0x7f81bc197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 -- 172.21.15.87:0/775688142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc1008d0 msgr2=0x7f81bc197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc1008d0 0x7f81bc197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 -- 172.21.15.87:0/775688142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc101290 msgr2=0x7f81bc197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc101290 0x7f81bc197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 -- 172.21.15.87:0/775688142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc19c030 con 0x7f81bc106890 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c0d54640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f81b400cce0 tx=0x7f81b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81b97fa640 1 -- 172.21.15.87:0/775688142 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400f450 con 0x7f81bc106890 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81b97fa640 1 -- 172.21.15.87:0/775688142 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b400fad0 con 0x7f81bc106890 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81b97fa640 1 -- 172.21.15.87:0/775688142 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4018c10 con 0x7f81bc106890 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.923+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc19c320 con 0x7f81bc106890 2024-06-06T08:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.924+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc1a3bb0 con 0x7f81bc106890 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81b97fa640 1 -- 172.21.15.87:0/775688142 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81b400f5f0 con 0x7f81bc106890 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 Processor -- start 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 -- start start 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38078f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e3806f14f0 con 0x55e38079a560 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e38068f870 con 0x55e38078fca0 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81c55a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e3806ed4b0 con 0x55e38078f380 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81b97fa640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 0x7f81a40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45550/0 (socket says 172.21.15.87:45550) 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3729721500 learned_addr learned my addr 172.21.15.87:0/3729721500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.926+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38078f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81b97fa640 1 -- 172.21.15.87:0/775688142 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f81b40cfeb0 con 0x7f81bc106890 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81c0d54640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 0x7f81a40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3729721500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 msgr2=0x55e38078f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38078f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3729721500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 msgr2=0x55e38079a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3729721500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3806ab9b0 con 0x55e38079a560 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005ec0 tx=0x7f81b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc37880ad2e5a9d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81c0d54640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 0x7f81a40a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81b40a46f0 tx=0x7f81b406be10 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81a27fc640 1 -- 172.21.15.87:0/3729721500 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035070 con 0x55e38079a560 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81a27fc640 1 -- 172.21.15.87:0/3729721500 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.927+0000 7f81a27fc640 1 -- 172.21.15.87:0/3729721500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00053a0 con 0x55e38079a560 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- 172.21.15.87:0/3729721500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 msgr2=0x55e38079c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005ec0 tx=0x7f81b002a420 comp rx=0 tx=0).stop 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- 172.21.15.87:0/3729721500 shutdown_connections 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38079c950 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3729721500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38078f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- 172.21.15.87:0/3729721500 >> 172.21.15.87:0/3729721500 conn(0x55e38078b890 msgr2=0x55e38078bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- 172.21.15.87:0/3729721500 shutdown_connections 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- 172.21.15.87:0/3729721500 wait complete. 2024-06-06T08:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.928+0000 7f81c55a2740 1 -- start start 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 Processor -- start 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e380795b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e380796080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e3807891f0 con 0x55e38079a560 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e38067aae0 con 0x55e38078fca0 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e380789370 con 0x55e38078f380 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81c55a2740 10 client.0 _ll_register_callbacks cb 0x55e3806b39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45562/0 (socket says 172.21.15.87:45562) 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.929+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3088997724 learned_addr learned my addr 172.21.15.87:0/3088997724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e380796080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e380795b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3088997724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 msgr2=0x55e380795b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e380795b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 msgr2=0x55e380796080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e380796080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3088997724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e380405210 con 0x55e38079a560 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f81b00316e0 tx=0x7f81b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf197e06c8ded43c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f819b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0053070 con 0x55e38079a560 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f81b0005120 con 0x55e38079a560 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81b005f050 con 0x55e38079a560 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81c55a2740 10 client.18732 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.930+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e3804d1620 con 0x55e38079a560 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f81b003c440 con 0x55e38079a560 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 10 client.18732 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 10 client.18732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 20 client.18732 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e3807a83d0 0x55e3807aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 4 client.18732 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e380704f10 con 0x55e3807a83d0 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e3807a83d0 0x55e3807aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e3807a83d0 0x55e3807aa7c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a593d32281c92b8 server_cookie=770054341130b557 in_seq=0 out_seq=0 2024-06-06T08:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.931+0000 7f819b7fe640 10 client.18732 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f819b7fe640 1 -- 172.21.15.87:0/3088997724 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e380704f10 con 0x55e3807a83d0 2024-06-06T08:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f819b7fe640 10 client.18732 handle_command_reply: tid=0 2024-06-06T08:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 client.18732 shutdown 2024-06-06T08:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 msgr2=0x55e3807a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f81b00316e0 tx=0x7f81b0000bc0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 shutdown_connections 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e3807a83d0 0x55e3807aa7c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e3807a3f90 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e380796080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f819b7fe640 0 client.18732 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3088997724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e380795b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.933+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 >> 172.21.15.87:0/3088997724 conn(0x55e38078b890 msgr2=0x55e3807887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.934+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 shutdown_connections 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.934+0000 7f81c55a2740 1 -- 172.21.15.87:0/3088997724 wait complete. 2024-06-06T08:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.934+0000 7f81c55a2740 20 client.18732 trim_cache size 0 max 16384 2024-06-06T08:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 Processor -- start 2024-06-06T08:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 -- start start 2024-06-06T08:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078f380 0x55e38078a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078fca0 0x55e3807a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1af560 con 0x55e38079a560 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1ace30 con 0x55e38078f380 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81c55a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1aaca0 con 0x55e38078fca0 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45578/0 (socket says 172.21.15.87:45578) 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 -- 172.21.15.87:0/458502938 learned_addr learned my addr 172.21.15.87:0/458502938 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078f380 0x55e38078a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bbfff640 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078fca0 0x55e3807a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 -- 172.21.15.87:0/458502938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078fca0 msgr2=0x55e3807a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078fca0 0x55e3807a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 -- 172.21.15.87:0/458502938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078f380 msgr2=0x55e38078a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078f380 0x55e38078a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 -- 172.21.15.87:0/458502938 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e38078fad0 con 0x55e38079a560 2024-06-06T08:12:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.950+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009930 tx=0x7f81b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44b851b42f45294 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81a1ffb640 1 -- 172.21.15.87:0/458502938 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0053040 con 0x55e38079a560 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81a1ffb640 1 -- 172.21.15.87:0/458502938 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81a1ffb640 1 -- 172.21.15.87:0/458502938 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035030 con 0x55e38079a560 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 -- 172.21.15.87:0/458502938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 msgr2=0x55e380794220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009930 tx=0x7f81b0005fd0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 -- 172.21.15.87:0/458502938 shutdown_connections 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e380794220 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078fca0 0x55e3807a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 --2- 172.21.15.87:0/458502938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078f380 0x55e38078a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 -- 172.21.15.87:0/458502938 >> 172.21.15.87:0/458502938 conn(0x55e38078b890 msgr2=0x55e38079af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.951+0000 7f81c55a2740 1 -- 172.21.15.87:0/458502938 shutdown_connections 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 -- 172.21.15.87:0/458502938 wait complete. 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 -- start start 2024-06-06T08:12:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 Processor -- start 2024-06-06T08:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38079f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1ab2f0 con 0x55e38079a560 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1ac760 con 0x55e38078fca0 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1ab9c0 con 0x55e38078f380 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81c55a2740 10 client.0 _ll_register_callbacks cb 0x7f81ac0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45588/0 (socket says 172.21.15.87:45588) 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81bbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.952+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3615608064 learned_addr learned my addr 172.21.15.87:0/3615608064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38079f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3615608064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 msgr2=0x55e38079f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38079f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 msgr2=0x55e38079f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 -- 172.21.15.87:0/3615608064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e38080b300 con 0x55e38079a560 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81bb7fe640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f81b0031750 tx=0x7f81b003c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f994671a15abeda8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f819b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035070 con 0x55e38079a560 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f81b0005280 con 0x55e38079a560 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81b006a040 con 0x55e38079a560 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81c55a2740 10 client.18738 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e3808126c0 con 0x55e38079a560 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.953+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b0041860 con 0x55e38079a560 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f81b0058440 con 0x55e38079a560 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 10 client.18738 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 10 client.18738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 20 client.18738 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e380812a00 0x55e380814e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 4 client.18738 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3806fec20 con 0x55e380812a00 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e380812a00 0x55e380814e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f819b7fe640 10 client.18738 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:42.954+0000 7f81bbfff640 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e380812a00 0x55e380814e20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb47e3af8c4e003c server_cookie=82329ea189d0d047 in_seq=0 out_seq=0 2024-06-06T08:12:43.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 128675 10.57 MB/sec execute 525 sec latency 350.110 ms 2024-06-06T08:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.058+0000 7f819b7fe640 1 -- 172.21.15.87:0/3615608064 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e3806fec20 con 0x55e380812a00 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.058+0000 7f819b7fe640 10 client.18738 handle_command_reply: tid=0 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.058+0000 7f81c55a2740 1 client.18738 shutdown 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.058+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 msgr2=0x55e38080acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f81b0031750 tx=0x7f81b003c580 comp rx=0 tx=0).stop 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 shutdown_connections 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x55e380812a00 0x55e380814e20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e38079a560 0x55e38080acc0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f819b7fe640 0 client.18738 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e38078fca0 0x55e38079f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 --2- 172.21.15.87:0/3615608064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e38078f380 0x55e38079f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 >> 172.21.15.87:0/3615608064 conn(0x55e38078b890 msgr2=0x55e380795910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 shutdown_connections 2024-06-06T08:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 1 -- 172.21.15.87:0/3615608064 wait complete. 2024-06-06T08:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.059+0000 7f81c55a2740 20 client.18738 trim_cache size 0 max 16384 2024-06-06T08:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.060+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 msgr2=0x7f81a40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.060+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 0x7f81a40a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81b40a46f0 tx=0x7f81b406be10 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.060+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 msgr2=0x7f81bc19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.060+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f81b400cce0 tx=0x7f81b400a620 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 shutdown_connections 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f81a40a2da0 0x7f81a40a5260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81bc106890 0x7f81bc19b900 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc101290 0x7f81bc197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 --2- 172.21.15.87:0/775688142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81bc1008d0 0x7f81bc197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 >> 172.21.15.87:0/775688142 conn(0x7f81bc077a20 msgr2=0x7f81bc1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 shutdown_connections 2024-06-06T08:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:44.061+0000 7f81c27b7640 1 -- 172.21.15.87:0/775688142 wait complete. 2024-06-06T08:12:44.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 128777 10.57 MB/sec execute 526 sec latency 678.186 ms 2024-06-06T08:12:44.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:44 smithi087 ceph-mon[32276]: pgmap v785: 129 pgs: 129 active+clean; 717 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 485 op/s 2024-06-06T08:12:44.511 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:44 smithi195 ceph-mon[37040]: pgmap v785: 129 pgs: 129 active+clean; 717 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 485 op/s 2024-06-06T08:12:44.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:44 smithi181 ceph-mon[36790]: pgmap v785: 129 pgs: 129 active+clean; 717 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 485 op/s 2024-06-06T08:12:45.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 129077 10.57 MB/sec execute 527 sec latency 334.624 ms 2024-06-06T08:12:46.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 129335 10.58 MB/sec execute 528 sec latency 400.335 ms 2024-06-06T08:12:46.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:46 smithi087 ceph-mon[32276]: pgmap v786: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 362 op/s 2024-06-06T08:12:46.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:46 smithi195 ceph-mon[37040]: pgmap v786: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 362 op/s 2024-06-06T08:12:46.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:46 smithi181 ceph-mon[36790]: pgmap v786: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 362 op/s 2024-06-06T08:12:47.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 129576 10.57 MB/sec execute 529 sec latency 413.324 ms 2024-06-06T08:12:48.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 129724 10.56 MB/sec execute 530 sec latency 475.786 ms 2024-06-06T08:12:48.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:48 smithi087 ceph-mon[32276]: pgmap v787: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-06-06T08:12:48.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:48 smithi195 ceph-mon[37040]: pgmap v787: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-06-06T08:12:48.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:48 smithi181 ceph-mon[36790]: pgmap v787: 129 pgs: 129 active+clean; 724 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-06-06T08:12:49.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 129916 10.56 MB/sec execute 531 sec latency 544.163 ms 2024-06-06T08:12:49.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:49 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:12:49.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:49 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:12:49.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:49 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:12:50.165 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:50 smithi181 ceph-mon[36790]: pgmap v788: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.7 MiB/s wr, 518 op/s 2024-06-06T08:12:50.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130263 10.58 MB/sec execute 532 sec latency 470.971 ms 2024-06-06T08:12:50.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:50 smithi087 ceph-mon[32276]: pgmap v788: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.7 MiB/s wr, 518 op/s 2024-06-06T08:12:50.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:50 smithi195 ceph-mon[37040]: pgmap v788: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.7 MiB/s wr, 518 op/s 2024-06-06T08:12:51.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130571 10.61 MB/sec execute 533 sec latency 496.633 ms 2024-06-06T08:12:52.199 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130747 10.60 MB/sec execute 534 sec latency 460.021 ms 2024-06-06T08:12:52.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:52 smithi195 ceph-mon[37040]: pgmap v789: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 402 op/s 2024-06-06T08:12:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:52 smithi181 ceph-mon[36790]: pgmap v789: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 402 op/s 2024-06-06T08:12:52.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:52 smithi087 ceph-mon[32276]: pgmap v789: 129 pgs: 129 active+clean; 711 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 402 op/s 2024-06-06T08:12:53.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130853 10.58 MB/sec execute 535 sec latency 491.489 ms 2024-06-06T08:12:54.199 INFO:tasks.workunit.client.0.smithi087.stdout: 10 130952 10.57 MB/sec execute 536 sec latency 818.332 ms 2024-06-06T08:12:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:54 smithi181 ceph-mon[36790]: pgmap v790: 129 pgs: 129 active+clean; 723 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-06-06T08:12:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:54 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:54.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:54 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:54.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:54 smithi087 ceph-mon[32276]: pgmap v790: 129 pgs: 129 active+clean; 723 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-06-06T08:12:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:54 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:54.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:54 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:54.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:54 smithi195 ceph-mon[37040]: pgmap v790: 129 pgs: 129 active+clean; 723 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-06-06T08:12:54.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:54 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:54.762 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:54 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:12:55.199 INFO:tasks.workunit.client.0.smithi087.stdout: 10 131227 10.57 MB/sec execute 537 sec latency 861.125 ms 2024-06-06T08:12:56.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 131454 10.57 MB/sec execute 538 sec latency 508.146 ms 2024-06-06T08:12:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:56 smithi181 ceph-mon[36790]: pgmap v791: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-06-06T08:12:56.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:56 smithi087 ceph-mon[32276]: pgmap v791: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-06-06T08:12:56.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:56 smithi195 ceph-mon[37040]: pgmap v791: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3121944445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c3810e360 msgr2=0x7f1c38110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3121944445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c3810e360 0x7f1c38110750 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009ca0 tx=0x7f1c3002a8c0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3121944445 shutdown_connections 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3121944445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c3810e360 0x7f1c38110750 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3121944445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381010d0 0x7f1c3810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3121944445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c381007b0 0x7f1c38100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3121944445 >> 172.21.15.87:0/3121944445 conn(0x7f1c38077a20 msgr2=0x7f1c38077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3121944445 shutdown_connections 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3121944445 wait complete. 2024-06-06T08:12:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.171+0000 7f1c3d59e640 1 Processor -- start 2024-06-06T08:12:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 -- start start 2024-06-06T08:12:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381007b0 0x7f1c3819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c3810e360 0x7f1c3819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113f90 con 0x7f1c381010d0 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113e10 con 0x7f1c381007b0 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c3d59e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38114110 con 0x7f1c3810e360 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56114/0 (socket says 172.21.15.87:56114) 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 -- 172.21.15.87:0/3566422056 learned_addr learned my addr 172.21.15.87:0/3566422056 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381007b0 0x7f1c3819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c377fe640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c3810e360 0x7f1c3819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 -- 172.21.15.87:0/3566422056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c3810e360 msgr2=0x7f1c3819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c3810e360 0x7f1c3819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 -- 172.21.15.87:0/3566422056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381007b0 msgr2=0x7f1c3819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381007b0 0x7f1c3819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.172+0000 7f1c367fc640 1 -- 172.21.15.87:0/3566422056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3819f480 con 0x7f1c381010d0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c367fc640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00e9f0 tx=0x7f1c2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c1ffff640 1 -- 172.21.15.87:0/3566422056 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00cd80 con 0x7f1c381010d0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3819f770 con 0x7f1c381010d0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c1ffff640 1 -- 172.21.15.87:0/3566422056 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c010910 con 0x7f1c381010d0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c1ffff640 1 -- 172.21.15.87:0/3566422056 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0188f0 con 0x7f1c381010d0 2024-06-06T08:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.173+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c381a7000 con 0x7f1c381010d0 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.175+0000 7f1c1ffff640 1 -- 172.21.15.87:0/3566422056 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1c2c010ac0 con 0x7f1c381010d0 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.175+0000 7f1c40389740 1 Processor -- start 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.175+0000 7f1c40389740 1 -- start start 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.175+0000 7f1c40389740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebc7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.175+0000 7f1c40389740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c40389740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebd4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c40389740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5644eeb2b4f0 con 0x5644eebc7a90 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c40389740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5644eeac9870 con 0x5644eebc7170 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c40389740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644eeb274b0 con 0x5644eebd2350 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c377fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebc7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56120/0 (socket says 172.21.15.87:56120) 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 -- 172.21.15.87:0/651550683 learned_addr learned my addr 172.21.15.87:0/651550683 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c367fc640 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebd4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c1ffff640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 0x7f1c200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 -- 172.21.15.87:0/651550683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 msgr2=0x5644eebd4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebd4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 -- 172.21.15.87:0/651550683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 msgr2=0x5644eebc7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebc7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 -- 172.21.15.87:0/651550683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644eeae59b0 con 0x5644eebc7a90 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c377fe640 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebc7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c377fe640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 0x7f1c200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c1ffff640 1 -- 172.21.15.87:0/3566422056 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f1c2c0cfbb0 con 0x7f1c381010d0 2024-06-06T08:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.176+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009cf0 tx=0x7f1c28011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208f84f75a877918 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.177+0000 7f1c377fe640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 0x7f1c200a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005e30 tx=0x7f1c300095c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.177+0000 7f1c1d7fa640 1 -- 172.21.15.87:0/651550683 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004230 con 0x5644eebc7a90 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.177+0000 7f1c1d7fa640 1 -- 172.21.15.87:0/651550683 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c280043d0 con 0x5644eebc7a90 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.177+0000 7f1c1d7fa640 1 -- 172.21.15.87:0/651550683 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28019960 con 0x5644eebc7a90 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- 172.21.15.87:0/651550683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 msgr2=0x5644eebd1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009cf0 tx=0x7f1c28011810 comp rx=0 tx=0).stop 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- 172.21.15.87:0/651550683 shutdown_connections 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebd4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd1e10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- 172.21.15.87:0/651550683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebc7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- 172.21.15.87:0/651550683 >> 172.21.15.87:0/651550683 conn(0x5644eebc3680 msgr2=0x5644eebc3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- 172.21.15.87:0/651550683 shutdown_connections 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- 172.21.15.87:0/651550683 wait complete. 2024-06-06T08:12:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- start start 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 Processor -- start 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebcf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebcfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5644eebd4f80 con 0x5644eebc7170 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5644eeab4ae0 con 0x5644eebc7a90 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.178+0000 7f1c40389740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644eebd5100 con 0x5644eebd2350 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c40389740 10 client.0 _ll_register_callbacks cb 0x5644eeaed9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56124/0 (socket says 172.21.15.87:56124) 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 -- 172.21.15.87:0/2525729507 learned_addr learned my addr 172.21.15.87:0/2525729507 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c367fc640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebcf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c377fe640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebcfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 -- 172.21.15.87:0/2525729507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 msgr2=0x5644eebcfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebcfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 -- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 msgr2=0x5644eebcf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebcf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 -- 172.21.15.87:0/2525729507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ee83f210 con 0x5644eebc7170 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c367fc640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebcf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1c280052a0 tx=0x7f1c280045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4c3536cd42f5fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28027070 con 0x5644eebc7170 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f1c280037f0 con 0x5644eebc7170 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c280043d0 con 0x5644eebc7170 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.179+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1c28033050 con 0x5644eebc7170 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 10 client.18750 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5644ee90b620 con 0x5644eebc7170 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c280043d0 con 0x5644eebc7170 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1c28004ba0 con 0x5644eebc7170 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 10 client.18750 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 10 client.18750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 20 client.18750 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eebe2420 0x5644eebe4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 4 client.18750 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644eeb3ef10 con 0x5644eebe2420 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.180+0000 7f1c367fc640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eebe2420 0x5644eebe4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.181+0000 7f1c1aff5640 10 client.18750 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.181+0000 7f1c367fc640 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eebe2420 0x5644eebe4810 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0c2d2976a50ec2 server_cookie=9d3210860ef31b0 in_seq=0 out_seq=0 2024-06-06T08:12:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c1aff5640 1 -- 172.21.15.87:0/2525729507 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644eeb3ef10 con 0x5644eebe2420 2024-06-06T08:12:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c1aff5640 10 client.18750 handle_command_reply: tid=0 2024-06-06T08:12:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 client.18750 shutdown 2024-06-06T08:12:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 msgr2=0x5644eebcf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1c280052a0 tx=0x7f1c280045f0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 shutdown_connections 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eebe2420 0x5644eebe4810 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebcf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 --2- 172.21.15.87:0/2525729507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcf280 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c1aff5640 0 client.18750 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 >> 172.21.15.87:0/2525729507 conn(0x5644eebc3680 msgr2=0x5644eebc9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 shutdown_connections 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 1 -- 172.21.15.87:0/2525729507 wait complete. 2024-06-06T08:12:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.182+0000 7f1c40389740 20 client.18750 trim_cache size 0 max 16384 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 Processor -- start 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 -- start start 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebe9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381b3b00 con 0x5644eebc7170 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381b13d0 con 0x5644eebc7a90 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c40389740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381af240 con 0x5644eebd2350 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56128/0 (socket says 172.21.15.87:56128) 2024-06-06T08:12:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 -- 172.21.15.87:0/936407676 learned_addr learned my addr 172.21.15.87:0/936407676 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:57.200 INFO:tasks.workunit.client.0.smithi087.stdout: 10 131774 10.59 MB/sec execute 539 sec latency 554.894 ms 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c367fc640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c377fe640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebe9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 -- 172.21.15.87:0/936407676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 msgr2=0x5644eebe9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebe9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 -- 172.21.15.87:0/936407676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 msgr2=0x5644eebce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c36ffd640 1 -- 172.21.15.87:0/936407676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644eebcd7d0 con 0x5644eebc7170 2024-06-06T08:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.199+0000 7f1c377fe640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebe9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000c00 tx=0x7f1c28005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91367c47db5b254f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c1cff9640 1 -- 172.21.15.87:0/936407676 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28030020 con 0x5644eebc7170 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c1cff9640 1 -- 172.21.15.87:0/936407676 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c280052d0 con 0x5644eebc7170 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c1cff9640 1 -- 172.21.15.87:0/936407676 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28027070 con 0x5644eebc7170 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 -- 172.21.15.87:0/936407676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 msgr2=0x5644eebcdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000c00 tx=0x7f1c28005850 comp rx=0 tx=0).stop 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 -- 172.21.15.87:0/936407676 shutdown_connections 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eebe9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7a90 0x5644eebce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 --2- 172.21.15.87:0/936407676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7170 0x5644eebcdb50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 -- 172.21.15.87:0/936407676 >> 172.21.15.87:0/936407676 conn(0x5644eebc3680 msgr2=0x5644eebc3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.200+0000 7f1c40389740 1 -- 172.21.15.87:0/936407676 shutdown_connections 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 -- 172.21.15.87:0/936407676 wait complete. 2024-06-06T08:12:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 -- start start 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 Processor -- start 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebd53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eec4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381af890 con 0x5644eebc7a90 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381b0d00 con 0x5644eebc7170 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381aff60 con 0x5644eebd2350 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c40389740 10 client.0 _ll_register_callbacks cb 0x7f1c28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56140/0 (socket says 172.21.15.87:56140) 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 -- 172.21.15.87:0/376514682 learned_addr learned my addr 172.21.15.87:0/376514682 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c377fe640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eec4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebd53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 -- 172.21.15.87:0/376514682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 msgr2=0x5644eec4af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eec4af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 -- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 msgr2=0x5644eebd53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.201+0000 7f1c367fc640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebd53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c367fc640 1 -- 172.21.15.87:0/376514682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644eebd6200 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c367fc640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c09ba60 tx=0x7f1c2c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9f10b075868b26 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0a0060 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9583+0+0 (secure 0 0 0) 0x7f1c2c0983d0 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c280052d0 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c0b9050 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 10 client.18756 fetch_fsmap learned FSMap version 55 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5644eec4c5f0 con 0x5644eebc7a90 2024-06-06T08:12:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c280052d0 con 0x5644eebc7a90 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7f1c2c09b220 con 0x5644eebc7a90 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 10 client.18756 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 10 client.18756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 20 client.18756 populate_metadata read hostname 'smithi087' 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eec4cd20 0x5644eec4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 4 client.18756 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644eeb38c20 con 0x5644eec4cd20 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.202+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eec4cd20 0x5644eec4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.203+0000 7f1c36ffd640 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eec4cd20 0x5644eec4f140 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f338005bf662fa36 server_cookie=33047e840c0ad5c9 in_seq=0 out_seq=0 2024-06-06T08:12:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:57.203+0000 7f1c1aff5640 10 client.18756 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:58.198 INFO:tasks.workunit.client.0.smithi087.stdout: 10 131815 10.59 MB/sec execute 540 sec latency 755.519 ms 2024-06-06T08:12:58.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:12:58 smithi181 ceph-mon[36790]: pgmap v792: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.591+0000 7f1c1aff5640 1 -- 172.21.15.87:0/376514682 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644eeb38c20 con 0x5644eec4cd20 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.591+0000 7f1c1aff5640 10 client.18756 handle_command_reply: tid=0 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.591+0000 7f1c40389740 1 client.18756 shutdown 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 msgr2=0x5644eebd5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c09ba60 tx=0x7f1c2c005f10 comp rx=0 tx=0).stop 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 shutdown_connections 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x5644eec4cd20 0x5644eec4f140 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644eebd2350 0x5644eec4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c1aff5640 0 client.18756 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644eebc7a90 0x5644eebd5930 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 --2- 172.21.15.87:0/376514682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644eebc7170 0x5644eebd53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 >> 172.21.15.87:0/376514682 conn(0x5644eebc3680 msgr2=0x5644eebcdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 shutdown_connections 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 1 -- 172.21.15.87:0/376514682 wait complete. 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.592+0000 7f1c40389740 20 client.18756 trim_cache size 0 max 16384 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:{ 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout: "return_code": 0 2024-06-06T08:12:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stdout:} 2024-06-06T08:12:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 msgr2=0x7f1c200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 0x7f1c200a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005e30 tx=0x7f1c300095c0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 msgr2=0x7f1c3819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00e9f0 tx=0x7f1c2c00eeb0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 shutdown_connections 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f1c200a2da0 0x7f1c200a5260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c3810e360 0x7f1c3819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c381010d0 0x7f1c3819a9d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 --2- 172.21.15.87:0/3566422056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c381007b0 0x7f1c3819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.593+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 >> 172.21.15.87:0/3566422056 conn(0x7f1c38077a20 msgr2=0x7f1c3810f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.594+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 shutdown_connections 2024-06-06T08:12:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi087.front.sepia.ceph.com.smithi087.stderr:2024-06-06T08:12:58.594+0000 7f1c3d59e640 1 -- 172.21.15.87:0/3566422056 wait complete. 2024-06-06T08:12:58.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:12:58 smithi087 ceph-mon[32276]: pgmap v792: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-06-06T08:12:58.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:12:58 smithi195 ceph-mon[37040]: pgmap v792: 129 pgs: 129 active+clean; 726 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-06-06T08:12:59.202 INFO:tasks.workunit.client.0.smithi087.stdout: 10 131986 10.58 MB/sec execute 541 sec latency 871.620 ms 2024-06-06T08:13:00.199 INFO:tasks.workunit.client.0.smithi087.stdout: 10 132154 10.57 MB/sec execute 542 sec latency 241.551 ms 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: pgmap v793: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:13:00.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: pgmap v793: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-06-06T08:13:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:13:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:13:00.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: pgmap v793: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-06-06T08:13:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:13:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:13:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:13:00.625 INFO:tasks.workunit.client.0.smithi087.stdout:[313427] write failed on handle 11316 (Resource temporarily unavailable) 2024-06-06T08:13:00.625 INFO:tasks.workunit.client.0.smithi087.stdout:Child failed with status 1 2024-06-06T08:13:00.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:13:00.664 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-06T08:13:00.664 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-06T08:13:01.001 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi087 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-06-06T08:13:01.221 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bc91305e1d1a4c4ba7d6c9baacf9bed2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi087 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-06-06T08:13:01.224 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-06T08:13:01.234 INFO:tasks.quiescer:joining Quiescers 2024-06-06T08:13:01.234 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-06T08:13:01.234 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:13:01.779 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:02.265 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:02 smithi087 ceph-mon[32276]: pgmap v794: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T08:13:02.265 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:02 smithi087 ceph-mon[32276]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T08:13:02.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:02 smithi181 ceph-mon[36790]: pgmap v794: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T08:13:02.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:02 smithi181 ceph-mon[36790]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T08:13:02.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:02 smithi195 ceph-mon[37040]: pgmap v794: 129 pgs: 129 active+clean; 735 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T08:13:02.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:02 smithi195 ceph-mon[37040]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T08:13:04.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.024+0000 7fe2a8efc700 1 -- 172.21.15.87:0/523794755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c095950 msgr2=0x7fe29c0a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:04.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.024+0000 7fe2a8efc700 1 --2- 172.21.15.87:0/523794755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c095950 0x7fe29c0a1b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4067f10 tx=0x7fe2a4068aa0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 -- 172.21.15.87:0/523794755 shutdown_connections 2024-06-06T08:13:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 --2- 172.21.15.87:0/523794755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c095950 0x7fe29c0a1b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 --2- 172.21.15.87:0/523794755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c09b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 --2- 172.21.15.87:0/523794755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c094f90 0x7fe29c095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 -- 172.21.15.87:0/523794755 >> 172.21.15.87:0/523794755 conn(0x7fe29c00a850 msgr2=0x7fe29c00ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 -- 172.21.15.87:0/523794755 shutdown_connections 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.025+0000 7fe2a8efc700 1 -- 172.21.15.87:0/523794755 wait complete. 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.026+0000 7fe2a8efc700 1 Processor -- start 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.026+0000 7fe2a8efc700 1 -- start start 2024-06-06T08:13:04.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c094f90 0x7fe29c0aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c095950 0x7fe29c0a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c0a3210 con 0x7fe29c09ad50 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c0a3090 con 0x7fe29c094f90 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a8efc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c0a3390 con 0x7fe29c095950 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:56152/0 (socket says 172.21.15.87:56152) 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a3fff700 1 -- 172.21.15.87:0/128398640 learned_addr learned my addr 172.21.15.87:0/128398640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:04.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a2ffd700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c095950 0x7fe29c0a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.027+0000 7fe2a37fe700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c094f90 0x7fe29c0aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.028+0000 7fe2a3fff700 1 -- 172.21.15.87:0/128398640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c095950 msgr2=0x7fe29c0a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.028+0000 7fe2a3fff700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c095950 0x7fe29c0a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.028+0000 7fe2a3fff700 1 -- 172.21.15.87:0/128398640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c094f90 msgr2=0x7fe29c0aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.028+0000 7fe2a3fff700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c094f90 0x7fe29c0aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.028+0000 7fe2a3fff700 1 -- 172.21.15.87:0/128398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c0a54d0 con 0x7fe29c09ad50 2024-06-06T08:13:04.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.029+0000 7fe2a3fff700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4051000 tx=0x7fe2a4078b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:04.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.034+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4078e20 con 0x7fe29c09ad50 2024-06-06T08:13:04.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.034+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4069970 con 0x7fe29c09ad50 2024-06-06T08:13:04.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.034+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a407e890 con 0x7fe29c09ad50 2024-06-06T08:13:04.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.035+0000 7fe2a8efc700 1 -- 172.21.15.87:0/128398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c147a10 con 0x7fe29c09ad50 2024-06-06T08:13:04.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.035+0000 7fe2a8efc700 1 -- 172.21.15.87:0/128398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c147e70 con 0x7fe29c09ad50 2024-06-06T08:13:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.037+0000 7fe2a8efc700 1 -- 172.21.15.87:0/128398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c0963e0 con 0x7fe29c09ad50 2024-06-06T08:13:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.039+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe2a4069490 con 0x7fe29c09ad50 2024-06-06T08:13:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.040+0000 7fe2a0ff9700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 0x7fe2940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.040+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9671+0+0 (secure 0 0 0) 0x7fe2a4134460 con 0x7fe29c09ad50 2024-06-06T08:13:04.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.046+0000 7fe2a37fe700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 0x7fe2940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:04.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.049+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe2a4100c90 con 0x7fe29c09ad50 2024-06-06T08:13:04.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.055+0000 7fe2a37fe700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 0x7fe2940a53e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe290009a10 tx=0x7fe290009190 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:04.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:04 smithi087 ceph-mon[32276]: pgmap v796: 129 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 86 active+clean; 726 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 441 op/s 2024-06-06T08:13:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:04 smithi195 ceph-mon[37040]: pgmap v796: 129 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 86 active+clean; 726 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 441 op/s 2024-06-06T08:13:04.520 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:04 smithi181 ceph-mon[36790]: pgmap v796: 129 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 86 active+clean; 726 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 441 op/s 2024-06-06T08:13:04.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.524+0000 7fe2a8efc700 1 -- 172.21.15.87:0/128398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe29c0a5d00 con 0x7fe29c09ad50 2024-06-06T08:13:04.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.528+0000 7fe2a0ff9700 1 -- 172.21.15.87:0/128398640 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7fe2a4100590 con 0x7fe29c09ad50 2024-06-06T08:13:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:13:04.528 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:13:04.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.536+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 msgr2=0x7fe2940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:04.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.536+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 0x7fe2940a53e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe290009a10 tx=0x7fe290009190 comp rx=0 tx=0).stop 2024-06-06T08:13:04.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.536+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 msgr2=0x7fe29c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:04.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.536+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4051000 tx=0x7fe2a4078b20 comp rx=0 tx=0).stop 2024-06-06T08:13:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.540+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 shutdown_connections 2024-06-06T08:13:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.541+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29c094f90 0x7fe29c0aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.541+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fe2940a2f20 0x7fe2940a53e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.541+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe29c09ad50 0x7fe29c0a4d60 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.541+0000 7fe28e7fc700 1 --2- 172.21.15.87:0/128398640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c095950 0x7fe29c0a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:04.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.541+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 >> 172.21.15.87:0/128398640 conn(0x7fe29c00a850 msgr2=0x7fe29c093c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.543+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 shutdown_connections 2024-06-06T08:13:04.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:04.545+0000 7fe28e7fc700 1 -- 172.21.15.87:0/128398640 wait complete. 2024-06-06T08:13:04.545 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:13:05.326 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24467 quiesce db --cancel --all' 2024-06-06T08:13:05.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24467 quiesce db --cancel --all 2024-06-06T08:13:05.382 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:05 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/128398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:13:05.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:05 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/128398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:13:05.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:05 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/128398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:13:05.809 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:06.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:06 smithi087 ceph-mon[32276]: pgmap v797: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:06 smithi195 ceph-mon[37040]: pgmap v797: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:06.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:06 smithi181 ceph-mon[36790]: pgmap v797: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:08.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.195+0000 7fd86a3d0700 1 -- 172.21.15.87:0/1973195941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86411bc70 msgr2=0x7fd86411e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.195+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/1973195941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86411bc70 0x7fd86411e060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd854009990 tx=0x7fd854009ca0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.197+0000 7fd86a3d0700 1 -- 172.21.15.87:0/1973195941 shutdown_connections 2024-06-06T08:13:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.197+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/1973195941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86411bc70 0x7fd86411e060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.197+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/1973195941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110b80 0x7fd864109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.197+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/1973195941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd864110ff0 0x7fd8641099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.197+0000 7fd86a3d0700 1 -- 172.21.15.87:0/1973195941 >> 172.21.15.87:0/1973195941 conn(0x7fd86406d470 msgr2=0x7fd86406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.202+0000 7fd86a3d0700 1 -- 172.21.15.87:0/1973195941 shutdown_connections 2024-06-06T08:13:08.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.204+0000 7fd86a3d0700 1 -- 172.21.15.87:0/1973195941 wait complete. 2024-06-06T08:13:08.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.205+0000 7fd86a3d0700 1 Processor -- start 2024-06-06T08:13:08.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.205+0000 7fd86a3d0700 1 -- start start 2024-06-06T08:13:08.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd864110b80 0x7fd8641ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86411bc70 0x7fd8641a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641215e0 con 0x7fd864110ff0 2024-06-06T08:13:08.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd864121460 con 0x7fd864110b80 2024-06-06T08:13:08.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.206+0000 7fd86a3d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd864121760 con 0x7fd86411bc70 2024-06-06T08:13:08.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.209+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.209+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50166/0 (socket says 172.21.15.87:50166) 2024-06-06T08:13:08.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.209+0000 7fd868bcd700 1 -- 172.21.15.87:0/2667230811 learned_addr learned my addr 172.21.15.87:0/2667230811 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:08.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.210+0000 7fd869bcf700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86411bc70 0x7fd8641a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.211+0000 7fd868bcd700 1 -- 172.21.15.87:0/2667230811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86411bc70 msgr2=0x7fd8641a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.211+0000 7fd868bcd700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86411bc70 0x7fd8641a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.211+0000 7fd868bcd700 1 -- 172.21.15.87:0/2667230811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd864110b80 msgr2=0x7fd8641ad6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:13:08.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.211+0000 7fd868bcd700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd864110b80 0x7fd8641ad6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.211+0000 7fd868bcd700 1 -- 172.21.15.87:0/2667230811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8641a8000 con 0x7fd864110ff0 2024-06-06T08:13:08.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.212+0000 7fd868bcd700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ef10 tx=0x7fd86000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.212+0000 7fd85a7fc700 1 -- 172.21.15.87:0/2667230811 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86000ca70 con 0x7fd864110ff0 2024-06-06T08:13:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.213+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8641a8290 con 0x7fd864110ff0 2024-06-06T08:13:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.213+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641b4500 con 0x7fd864110ff0 2024-06-06T08:13:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.213+0000 7fd85a7fc700 1 -- 172.21.15.87:0/2667230811 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd860019070 con 0x7fd864110ff0 2024-06-06T08:13:08.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.213+0000 7fd85a7fc700 1 -- 172.21.15.87:0/2667230811 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600146e0 con 0x7fd864110ff0 2024-06-06T08:13:08.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.216+0000 7fd85a7fc700 1 -- 172.21.15.87:0/2667230811 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd860012050 con 0x7fd864110ff0 2024-06-06T08:13:08.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.221+0000 7fd875eb5b80 1 Processor -- start 2024-06-06T08:13:08.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.222+0000 7fd85a7fc700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 0x7fd8500a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.222+0000 7fd85a7fc700 1 -- 172.21.15.87:0/2667230811 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9671+0+0 (secure 0 0 0) 0x7fd8600cf760 con 0x7fd864110ff0 2024-06-06T08:13:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.225+0000 7fd875eb5b80 1 -- start start 2024-06-06T08:13:08.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df43844a0 0x562df43848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562df42d5880 con 0x562df4389d50 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562df427d790 con 0x562df438a670 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.226+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562df42d1840 con 0x562df43844a0 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.227+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.229+0000 7fd869bcf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df43844a0 0x562df43848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.229+0000 7fd869bcf700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 0x7fd8500a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.229+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:46728/0 (socket says 172.21.15.87:46728) 2024-06-06T08:13:08.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.229+0000 7fd868bcd700 1 -- 172.21.15.87:0/3658401227 learned_addr learned my addr 172.21.15.87:0/3658401227 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:08.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.232+0000 7fd868bcd700 1 -- 172.21.15.87:0/3658401227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df43844a0 msgr2=0x562df43848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:13:08.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.232+0000 7fd868bcd700 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df43844a0 0x562df43848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.232+0000 7fd868bcd700 1 -- 172.21.15.87:0/3658401227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 msgr2=0x562df438a130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:13:08.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.233+0000 7fd868bcd700 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.233+0000 7fd868bcd700 1 -- 172.21.15.87:0/3658401227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df4273fc0 con 0x562df438a670 2024-06-06T08:13:08.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.233+0000 7fd869bcf700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 0x7fd8500a53a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd854005cd0 tx=0x7fd854005c00 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.235+0000 7fd868bcd700 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd860097c60 tx=0x7fd86009b160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575de4f34d4e291a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.237+0000 7fd84f7fe700 1 -- 172.21.15.87:0/3658401227 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86009b800 con 0x562df438a670 2024-06-06T08:13:08.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.237+0000 7fd84f7fe700 1 -- 172.21.15.87:0/3658401227 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8600ce040 con 0x562df438a670 2024-06-06T08:13:08.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.237+0000 7fd84f7fe700 1 -- 172.21.15.87:0/3658401227 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86009ba60 con 0x562df438a670 2024-06-06T08:13:08.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.243+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3658401227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 msgr2=0x562df4383f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.243+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd860097c60 tx=0x7fd86009b160 comp rx=0 tx=0).stop 2024-06-06T08:13:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.244+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3658401227 shutdown_connections 2024-06-06T08:13:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.244+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df43844a0 0x562df43848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.244+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.244+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3658401227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df438a670 0x562df4383f60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.244+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3658401227 >> 172.21.15.87:0/3658401227 conn(0x562df42aed30 msgr2=0x562df437f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.248+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3658401227 shutdown_connections 2024-06-06T08:13:08.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.248+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3658401227 wait complete. 2024-06-06T08:13:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.248+0000 7fd875eb5b80 1 -- start start 2024-06-06T08:13:08.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.248+0000 7fd875eb5b80 1 Processor -- start 2024-06-06T08:13:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.249+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4390460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.249+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.249+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df438dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.250+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562df43a20f0 con 0x562df4389d50 2024-06-06T08:13:08.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.250+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562df4275c60 con 0x562df43844a0 2024-06-06T08:13:08.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.250+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562df43a2270 con 0x562df438a670 2024-06-06T08:13:08.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.250+0000 7fd875eb5b80 10 client.0 _ll_register_callbacks cb 0x562df426f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:13:08.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50182/0 (socket says 172.21.15.87:50182) 2024-06-06T08:13:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 -- 172.21.15.87:0/415839962 learned_addr learned my addr 172.21.15.87:0/415839962 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:08.251 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:08 smithi087 ceph-mon[32276]: pgmap v798: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd869bcf700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df438dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 -- 172.21.15.87:0/415839962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 msgr2=0x562df438dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df438dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 -- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 msgr2=0x562df4390460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4390460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.251+0000 7fd868bcd700 1 -- 172.21.15.87:0/415839962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df4224940 con 0x562df4389d50 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.252+0000 7fd868bcd700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd86009b500 tx=0x7fd860012600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b7b7c7784353 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.252+0000 7fd84cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:13:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.252+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600a4470 con 0x562df4389d50 2024-06-06T08:13:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.252+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9671+0+0 (secure 0 0 0) 0x7fd86009bc70 con 0x562df4389d50 2024-06-06T08:13:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.253+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8600ce040 con 0x562df4389d50 2024-06-06T08:13:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.253+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd8600a3fe0 con 0x562df4389d50 2024-06-06T08:13:08.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.253+0000 7fd875eb5b80 10 client.18765 fetch_fsmap learned FSMap version 55 2024-06-06T08:13:08.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.254+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562df43aca70 con 0x562df4389d50 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.254+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8600ce040 con 0x562df4389d50 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd86009cb30 con 0x562df4389d50 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 10 client.18765 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 10 client.18765 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 20 client.18765 populate_metadata read hostname 'smithi087' 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df43af180 0x562df43b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 4 client.18765 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:13:08.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.255+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df42e2d80 con 0x562df43af180 2024-06-06T08:13:08.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.263+0000 7fd8693ce700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df43af180 0x562df43b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.280+0000 7fd84cff9700 10 client.18765 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:13:08.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.280+0000 7fd8693ce700 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df43af180 0x562df43b15e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50428be94109d25 server_cookie=bb702eec9673a4f7 in_seq=0 out_seq=0 2024-06-06T08:13:08.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.282+0000 7fd84cff9700 1 -- 172.21.15.87:0/415839962 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562df42e2d80 con 0x562df43af180 2024-06-06T08:13:08.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.283+0000 7fd84cff9700 10 client.18765 handle_command_reply: tid=0 2024-06-06T08:13:08.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.283+0000 7fd875eb5b80 1 client.18765 shutdown 2024-06-06T08:13:08.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.286+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 msgr2=0x562df438d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.286+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd86009b500 tx=0x7fd860012600 comp rx=0 tx=0).stop 2024-06-06T08:13:08.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 shutdown_connections 2024-06-06T08:13:08.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df43af180 0x562df43b15e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4390460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df438dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/415839962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438d760 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 >> 172.21.15.87:0/415839962 conn(0x562df42aed30 msgr2=0x562df4382f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.287+0000 7fd84cff9700 0 client.18765 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:13:08.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.288+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 shutdown_connections 2024-06-06T08:13:08.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.288+0000 7fd875eb5b80 1 -- 172.21.15.87:0/415839962 wait complete. 2024-06-06T08:13:08.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.289+0000 7fd875eb5b80 20 client.18765 trim_cache size 0 max 16384 2024-06-06T08:13:08.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.334+0000 7fd875eb5b80 1 Processor -- start 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 -- start start 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4383f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480118d0 con 0x562df4389d50 2024-06-06T08:13:08.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800f160 con 0x562df43844a0 2024-06-06T08:13:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.335+0000 7fd875eb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800cfd0 con 0x562df438a670 2024-06-06T08:13:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.336+0000 7fd869bcf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.336+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.336+0000 7fd869bcf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:39112/0 (socket says 172.21.15.87:39112) 2024-06-06T08:13:08.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.336+0000 7fd869bcf700 1 -- 172.21.15.87:0/3630436570 learned_addr learned my addr 172.21.15.87:0/3630436570 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:08.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.336+0000 7fd8693ce700 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4383f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.338+0000 7fd869bcf700 1 -- 172.21.15.87:0/3630436570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 msgr2=0x562df4383f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:13:08.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.338+0000 7fd869bcf700 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4383f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.338+0000 7fd869bcf700 1 -- 172.21.15.87:0/3630436570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 msgr2=0x562df438b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:13:08.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.338+0000 7fd869bcf700 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.338+0000 7fd869bcf700 1 -- 172.21.15.87:0/3630436570 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df43a2270 con 0x562df438a670 2024-06-06T08:13:08.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.342+0000 7fd869bcf700 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd85400bac0 tx=0x7fd85402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05f7d4878baee9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.343+0000 7fd84effd700 1 -- 172.21.15.87:0/3630436570 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540039e0 con 0x562df438a670 2024-06-06T08:13:08.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.343+0000 7fd84effd700 1 -- 172.21.15.87:0/3630436570 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85402f070 con 0x562df438a670 2024-06-06T08:13:08.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.343+0000 7fd84effd700 1 -- 172.21.15.87:0/3630436570 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85400f460 con 0x562df438a670 2024-06-06T08:13:08.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.344+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3630436570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 msgr2=0x562df43b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.344+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd85400bac0 tx=0x7fd85402a000 comp rx=0 tx=0).stop 2024-06-06T08:13:08.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.345+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3630436570 shutdown_connections 2024-06-06T08:13:08.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.345+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df4383f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.345+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43b6eb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.345+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/3630436570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df438b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.345+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3630436570 >> 172.21.15.87:0/3630436570 conn(0x562df42aed30 msgr2=0x562df437f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.348+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3630436570 shutdown_connections 2024-06-06T08:13:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.348+0000 7fd875eb5b80 1 -- 172.21.15.87:0/3630436570 wait complete. 2024-06-06T08:13:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.348+0000 7fd875eb5b80 1 -- start start 2024-06-06T08:13:08.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.348+0000 7fd875eb5b80 1 Processor -- start 2024-06-06T08:13:08.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df43a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd868bcd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50196/0 (socket says 172.21.15.87:50196) 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd868bcd700 1 -- 172.21.15.87:0/1347391539 learned_addr learned my addr 172.21.15.87:0/1347391539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800d620 con 0x562df4389d50 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800ea90 con 0x562df43844a0 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800dcf0 con 0x562df438a670 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.349+0000 7fd875eb5b80 10 client.0 _ll_register_callbacks cb 0x562df426f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.351+0000 7fd868bcd700 1 -- 172.21.15.87:0/1347391539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 msgr2=0x562df43a5ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:13:08.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.351+0000 7fd868bcd700 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43a5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.351+0000 7fd868bcd700 1 -- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 msgr2=0x562df43a5460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:13:08.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.351+0000 7fd868bcd700 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df43a5460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.351+0000 7fd868bcd700 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df4419050 con 0x562df4389d50 2024-06-06T08:13:08.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.355+0000 7fd868bcd700 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd8600981d0 tx=0x7fd8600098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ee15abcbeb7be server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:08.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.357+0000 7fd84cff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-06-06T08:13:08.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.358+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600a4470 con 0x562df4389d50 2024-06-06T08:13:08.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.358+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9671+0+0 (secure 0 0 0) 0x7fd8600a4180 con 0x562df4389d50 2024-06-06T08:13:08.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.358+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd85402f070 con 0x562df4389d50 2024-06-06T08:13:08.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.364+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd86009b800 con 0x562df4389d50 2024-06-06T08:13:08.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.365+0000 7fd875eb5b80 10 client.18771 fetch_fsmap learned FSMap version 55 2024-06-06T08:13:08.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.365+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562df4419050 con 0x562df4389d50 2024-06-06T08:13:08.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.369+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85402f070 con 0x562df4389d50 2024-06-06T08:13:08.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.369+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 55) ==== 8346+0+0 (secure 0 0 0) 0x7fd86009a6c0 con 0x562df4389d50 2024-06-06T08:13:08.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.369+0000 7fd875eb5b80 10 client.18771 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T08:13:08.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.369+0000 7fd875eb5b80 10 client.18771 resolve_mds: validated gid 24467 aka daemon mds.b 2024-06-06T08:13:08.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.369+0000 7fd875eb5b80 20 client.18771 populate_metadata read hostname 'smithi087' 2024-06-06T08:13:08.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.370+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df441a580 0x562df441c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:08.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.370+0000 7fd875eb5b80 4 client.18771 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-06T08:13:08.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.370+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 --> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x562df42dd300 con 0x562df441a580 2024-06-06T08:13:08.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.373+0000 7fd8693ce700 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df441a580 0x562df441c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:08.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.387+0000 7fd84cff9700 10 client.18771 ms_handle_connect on v2:172.21.15.181:6838/851837058 2024-06-06T08:13:08.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.387+0000 7fd8693ce700 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df441a580 0x562df441c9a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54883c46a63add2f server_cookie=3d89d0fccfc8a28e in_seq=0 out_seq=0 2024-06-06T08:13:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd84cff9700 1 -- 172.21.15.87:0/1347391539 <== mds.0 v2:172.21.15.181:6838/851837058 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x562df42dd300 con 0x562df441a580 2024-06-06T08:13:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd84cff9700 10 client.18771 handle_command_reply: tid=0 2024-06-06T08:13:08.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 client.18771 shutdown 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 msgr2=0x562df43a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd8600981d0 tx=0x7fd8600098a0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 shutdown_connections 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:6838/851837058,v1:172.21.15.181:6839/851837058] conn(0x562df441a580 0x562df441c9a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562df43844a0 0x562df43a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562df438a670 0x562df43a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 --2- 172.21.15.87:0/1347391539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562df4389d50 0x562df43a59a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 >> 172.21.15.87:0/1347391539 conn(0x562df42aed30 msgr2=0x562df4388820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd84cff9700 0 client.18771 ms_handle_reset on v2:172.21.15.181:6838/851837058 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.389+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 shutdown_connections 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.390+0000 7fd875eb5b80 1 -- 172.21.15.87:0/1347391539 wait complete. 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.390+0000 7fd875eb5b80 20 client.18771 trim_cache size 0 max 16384 2024-06-06T08:13:08.391 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:08.392 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 55, 2024-06-06T08:13:08.392 INFO:teuthology.orchestra.run.smithi087.stdout: "leader": 24467, 2024-06-06T08:13:08.392 INFO:teuthology.orchestra.run.smithi087.stdout: "set_version": 76, 2024-06-06T08:13:08.392 INFO:teuthology.orchestra.run.smithi087.stdout: "sets": {} 2024-06-06T08:13:08.392 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 msgr2=0x7fd8500a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 0x7fd8500a53a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd854005cd0 tx=0x7fd854005c00 comp rx=0 tx=0).stop 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 msgr2=0x7fd8641adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ef10 tx=0x7fd86000c3a0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 shutdown_connections 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd864110b80 0x7fd8641ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fd8500a2ee0 0x7fd8500a53a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86411bc70 0x7fd8641a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 --2- 172.21.15.87:0/2667230811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd864110ff0 0x7fd8641adbe0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:08.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 >> 172.21.15.87:0/2667230811 conn(0x7fd86406d470 msgr2=0x7fd86410e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:08.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.391+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 shutdown_connections 2024-06-06T08:13:08.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:08.393+0000 7fd86a3d0700 1 -- 172.21.15.87:0/2667230811 wait complete. 2024-06-06T08:13:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:08 smithi195 ceph-mon[37040]: pgmap v798: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:08.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:08 smithi181 ceph-mon[36790]: pgmap v798: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 720 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T08:13:09.432 INFO:tasks.quiescer:done joining Quiescers 2024-06-06T08:13:09.432 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-06T08:13:09.442 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T08:13:09.450 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-06T08:13:09.461 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-06T08:13:09.461 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T08:13:09.461 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T08:13:09.462 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-06T08:13:09.525 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-06T08:13:09.592 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T08:13:09.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 -- 172.21.15.87:0/2839090125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069980 msgr2=0x7f5920100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:09.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 --2- 172.21.15.87:0/2839090125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069980 0x7f5920100280 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f591c009930 tx=0x7f591c02a420 comp rx=0 tx=0).stop 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 -- 172.21.15.87:0/2839090125 shutdown_connections 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 --2- 172.21.15.87:0/2839090125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920104c70 0x7f59201050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 --2- 172.21.15.87:0/2839090125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920069df0 0x7f59201007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 --2- 172.21.15.87:0/2839090125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069980 0x7f5920100280 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 -- 172.21.15.87:0/2839090125 >> 172.21.15.87:0/2839090125 conn(0x7f5920076e30 msgr2=0x7f59200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 -- 172.21.15.87:0/2839090125 shutdown_connections 2024-06-06T08:13:09.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.734+0000 7f5927d29640 1 -- 172.21.15.87:0/2839090125 wait complete. 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 Processor -- start 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 -- start start 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920069980 0x7f59201970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920104c70 0x7f592019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5920110ca0 con 0x7f5920069df0 2024-06-06T08:13:09.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5920110b20 con 0x7f5920104c70 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.735+0000 7f5927d29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5920110e20 con 0x7f5920069980 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50214/0 (socket says 172.21.15.87:50214) 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 -- 172.21.15.87:0/756182488 learned_addr learned my addr 172.21.15.87:0/756182488 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59262c6640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920104c70 0x7f592019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5925ac5640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920069980 0x7f59201970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 -- 172.21.15.87:0/756182488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920069980 msgr2=0x7f59201970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920069980 0x7f59201970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 -- 172.21.15.87:0/756182488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920104c70 msgr2=0x7f592019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920104c70 0x7f592019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 -- 172.21.15.87:0/756182488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592019c090 con 0x7f5920069df0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f59252c4640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f591400d950 tx=0x7f591400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914017030 con 0x7f5920069df0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592019c380 con 0x7f5920069df0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5914004020 con 0x7f5920069df0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140047a0 con 0x7f5920069df0 2024-06-06T08:13:09.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.736+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a3c10 con 0x7f5920069df0 2024-06-06T08:13:09.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.737+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592004f760 con 0x7f5920069df0 2024-06-06T08:13:09.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.738+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f59140041d0 con 0x7f5920069df0 2024-06-06T08:13:09.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.739+0000 7f5912ffd640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 0x7f590c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:09.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.739+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9671+0+0 (secure 0 0 0) 0x7f59140cf2b0 con 0x7f5920069df0 2024-06-06T08:13:09.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.739+0000 7f5925ac5640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 0x7f590c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:09.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.739+0000 7f5925ac5640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 0x7f590c0a5260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f591c0096c0 tx=0x7f591c000bc0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:09.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.743+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f591409ba50 con 0x7f5920069df0 2024-06-06T08:13:09.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.938+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5920102da0 con 0x7f590c0a2da0 2024-06-06T08:13:09.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.948+0000 7f5912ffd640 1 -- 172.21.15.87:0/756182488 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f5920102da0 con 0x7f590c0a2da0 2024-06-06T08:13:09.948 INFO:teuthology.orchestra.run.smithi087.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T07:48:14", "first": "2024-06-06T07:49:00", "last": "2024-06-06T08:13:00", "last_pruned": "2024-06-06T08:13:00", "created_count": 25, "pruned_count": 18, "active": true} 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 msgr2=0x7f590c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 0x7f590c0a5260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f591c0096c0 tx=0x7f591c000bc0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 msgr2=0x7f59201975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f591400d950 tx=0x7f591400de10 comp rx=0 tx=0).stop 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 shutdown_connections 2024-06-06T08:13:09.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f590c0a2da0 0x7f590c0a5260 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5920104c70 0x7f592019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5920069df0 0x7f59201975e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 --2- 172.21.15.87:0/756182488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5920069980 0x7f59201970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 >> 172.21.15.87:0/756182488 conn(0x7f5920076e30 msgr2=0x7f5920102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 shutdown_connections 2024-06-06T08:13:09.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:09.950+0000 7f5927d29640 1 -- 172.21.15.87:0/756182488 wait complete. 2024-06-06T08:13:09.961 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T08:13:10.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.064+0000 7fae9fddc640 1 -- 172.21.15.87:0/1660528134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae98101290 msgr2=0x7fae9810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.064+0000 7fae9fddc640 1 --2- 172.21.15.87:0/1660528134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae98101290 0x7fae9810d750 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fae90009cd0 tx=0x7fae9002a8e0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.064+0000 7fae9fddc640 1 -- 172.21.15.87:0/1660528134 shutdown_connections 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.064+0000 7fae9fddc640 1 --2- 172.21.15.87:0/1660528134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae98101290 0x7fae9810d750 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.064+0000 7fae9fddc640 1 --2- 172.21.15.87:0/1660528134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae981008d0 0x7fae98100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 --2- 172.21.15.87:0/1660528134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 0x7fae98106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- 172.21.15.87:0/1660528134 >> 172.21.15.87:0/1660528134 conn(0x7fae98077a20 msgr2=0x7fae98077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- 172.21.15.87:0/1660528134 shutdown_connections 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- 172.21.15.87:0/1660528134 wait complete. 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 Processor -- start 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- start start 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 0x7fae98198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 0x7fae9819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.068 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111da0 con 0x7fae981008d0 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111c20 con 0x7fae98106890 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9fddc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111f20 con 0x7fae98101290 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9db78640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.065+0000 7fae9db78640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50236/0 (socket says 172.21.15.87:50236) 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 -- 172.21.15.87:0/2419167827 learned_addr learned my addr 172.21.15.87:0/2419167827 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9e379640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 0x7fae9819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9d377640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 0x7fae98198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 -- 172.21.15.87:0/2419167827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 msgr2=0x7fae98198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 0x7fae98198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 -- 172.21.15.87:0/2419167827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 msgr2=0x7fae9819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 0x7fae9819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 -- 172.21.15.87:0/2419167827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819d160 con 0x7fae981008d0 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9d377640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 0x7fae98198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9db78640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94017070 con 0x7fae981008d0 2024-06-06T08:13:10.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae94009c30 con 0x7fae981008d0 2024-06-06T08:13:10.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9819d450 con 0x7fae981008d0 2024-06-06T08:13:10.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9400f6b0 con 0x7fae981008d0 2024-06-06T08:13:10.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.066+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a4ce0 con 0x7fae981008d0 2024-06-06T08:13:10.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.067+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9804f760 con 0x7fae981008d0 2024-06-06T08:13:10.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.068+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fae940109d0 con 0x7fae981008d0 2024-06-06T08:13:10.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.069+0000 7fae8effd640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 0x7fae840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.069+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7fae940cf410 con 0x7fae981008d0 2024-06-06T08:13:10.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.069+0000 7fae9d377640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 0x7fae840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.069+0000 7fae9d377640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 0x7fae840a5260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae98077f40 tx=0x7fae88005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.073+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae94009de0 con 0x7fae981008d0 2024-06-06T08:13:10.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.272+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 --> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fae98103a60 con 0x7fae840a2da0 2024-06-06T08:13:10.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.289+0000 7fae8effd640 1 -- 172.21.15.87:0/2419167827 <== mgr.24800 v2:172.21.15.87:6800/4062973588 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fae98103a60 con 0x7fae840a2da0 2024-06-06T08:13:10.289 INFO:teuthology.orchestra.run.smithi087.stdout:/ 1m 6m3h 2024-06-06T08:13:10.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 msgr2=0x7fae840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 0x7fae840a5260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae98077f40 tx=0x7fae88005f90 comp rx=0 tx=0).stop 2024-06-06T08:13:10.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 msgr2=0x7fae981981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).stop 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 shutdown_connections 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fae840a2da0 0x7fae840a5260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae98106890 0x7fae9819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae98101290 0x7fae98198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 --2- 172.21.15.87:0/2419167827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae981008d0 0x7fae981981c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 >> 172.21.15.87:0/2419167827 conn(0x7fae98077a20 msgr2=0x7fae98103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 shutdown_connections 2024-06-06T08:13:10.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.291+0000 7fae9fddc640 1 -- 172.21.15.87:0/2419167827 wait complete. 2024-06-06T08:13:10.302 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:10 smithi087 ceph-mon[32276]: pgmap v799: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-06-06T08:13:10.302 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:10 smithi087 ceph-mon[32276]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T08:13:10.303 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-06T08:13:10.314 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-06T08:13:10.314 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T08:13:10.319 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-06T08:13:10.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T08:13:10.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 -- 172.21.15.87:0/229043504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 msgr2=0x7f6d3c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/229043504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c10e850 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009cd0 tx=0x7f6d3402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 -- 172.21.15.87:0/229043504 shutdown_connections 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/229043504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c10e850 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/229043504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/229043504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 0x7f6d3c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 -- 172.21.15.87:0/229043504 >> 172.21.15.87:0/229043504 conn(0x7f6d3c0fd430 msgr2=0x7f6d3c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 -- 172.21.15.87:0/229043504 shutdown_connections 2024-06-06T08:13:10.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 -- 172.21.15.87:0/229043504 wait complete. 2024-06-06T08:13:10.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.432+0000 7f6d40aa5640 1 Processor -- start 2024-06-06T08:13:10.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 -- start start 2024-06-06T08:13:10.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 0x7f6d3c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c111da0 con 0x7f6d3c102340 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c111c20 con 0x7f6d3c107990 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c111f20 con 0x7f6d3c101980 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50250/0 (socket says 172.21.15.87:50250) 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 -- 172.21.15.87:0/3991254636 learned_addr learned my addr 172.21.15.87:0/3991254636 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d3a59c640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d3ad9d640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 0x7f6d3c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 -- 172.21.15.87:0/3991254636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 msgr2=0x7f6d3c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 -- 172.21.15.87:0/3991254636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 msgr2=0x7f6d3c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 0x7f6d3c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 -- 172.21.15.87:0/3991254636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c19d230 con 0x7f6d3c102340 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d3a59c640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d39d9b640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d950 tx=0x7f6d2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c017070 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c009c30 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c19d520 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.433+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00f6b0 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.434+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1a4db0 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.434+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c0687c0 con 0x7f6d3c102340 2024-06-06T08:13:10.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.435+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6d2c0109d0 con 0x7f6d3c102340 2024-06-06T08:13:10.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.436+0000 7f6d2b7fe640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 0x7f6d240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.436+0000 7f6d3a59c640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 0x7f6d240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.436+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7f6d2c0cf580 con 0x7f6d3c102340 2024-06-06T08:13:10.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.437+0000 7f6d3a59c640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 0x7f6d240a51c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009ba0 tx=0x7f6d30005f90 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.440+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d2c009de0 con 0x7f6d3c102340 2024-06-06T08:13:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:10 smithi195 ceph-mon[37040]: pgmap v799: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-06-06T08:13:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:10 smithi195 ceph-mon[37040]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T08:13:10.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:10 smithi181 ceph-mon[36790]: pgmap v799: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-06-06T08:13:10.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:10 smithi181 ceph-mon[36790]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T08:13:10.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.621+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d3c109960 con 0x7f6d3c102340 2024-06-06T08:13:10.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.623+0000 7f6d2b7fe640 1 -- 172.21.15.87:0/3991254636 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+25501 (secure 0 0 0) 0x7f6d2c09bd70 con 0x7f6d3c102340 2024-06-06T08:13:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:13:10.623 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":143,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T08:13:09.970261+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/3344462507":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/207935511":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/4065374776":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/2153987725":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":44,"length":1}]},{"pool":4,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 msgr2=0x7f6d240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 0x7f6d240a51c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009ba0 tx=0x7f6d30005f90 comp rx=0 tx=0).stop 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 msgr2=0x7f6d3c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d950 tx=0x7f6d2c00de10 comp rx=0 tx=0).stop 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 shutdown_connections 2024-06-06T08:13:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f6d240a2d00 0x7f6d240a51c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c107990 0x7f6d3c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d3c102340 0x7f6d3c198780 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 --2- 172.21.15.87:0/3991254636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3c101980 0x7f6d3c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 >> 172.21.15.87:0/3991254636 conn(0x7f6d3c0fd430 msgr2=0x7f6d3c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 shutdown_connections 2024-06-06T08:13:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.625+0000 7f6d40aa5640 1 -- 172.21.15.87:0/3991254636 wait complete. 2024-06-06T08:13:10.636 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T07:39:10.366934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T07:43:56.169718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T07:43:59.265590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T07:44:09.602679+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T08:13:10.637 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T08:13:10.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 -- 172.21.15.87:0/1610446774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18102330 msgr2=0x7f7c1810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/1610446774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18102330 0x7f7c1810e840 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009cd0 tx=0x7f7c1002a8e0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 -- 172.21.15.87:0/1610446774 shutdown_connections 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/1610446774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18102330 0x7f7c1810e840 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/1610446774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18101970 0x7f7c18101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/1610446774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c18107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 -- 172.21.15.87:0/1610446774 >> 172.21.15.87:0/1610446774 conn(0x7f7c18078df0 msgr2=0x7f7c18079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 -- 172.21.15.87:0/1610446774 shutdown_connections 2024-06-06T08:13:10.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.742+0000 7f7c1cc00640 1 -- 172.21.15.87:0/1610446774 wait complete. 2024-06-06T08:13:10.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.743+0000 7f7c1cc00640 1 Processor -- start 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.743+0000 7f7c1cc00640 1 -- start start 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18102330 0x7f7c18198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c1819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18111d90 con 0x7f7c18101970 2024-06-06T08:13:10.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18111c10 con 0x7f7c18107980 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18111f10 con 0x7f7c18102330 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50256/0 (socket says 172.21.15.87:50256) 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 -- 172.21.15.87:0/2603055196 learned_addr learned my addr 172.21.15.87:0/2603055196 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c0dd9b640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18102330 0x7f7c18198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c16d9d640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c1819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 -- 172.21.15.87:0/2603055196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18102330 msgr2=0x7f7c18198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18102330 0x7f7c18198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 -- 172.21.15.87:0/2603055196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 msgr2=0x7f7c1819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c1819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 -- 172.21.15.87:0/2603055196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1819d240 con 0x7f7c18101970 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c16d9d640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c1819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1659c640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e9f0 tx=0x7f7c0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800cd80 con 0x7f7c18101970 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1819d530 con 0x7f7c18101970 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c08010910 con 0x7f7c18101970 2024-06-06T08:13:10.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c080188f0 con 0x7f7c18101970 2024-06-06T08:13:10.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.744+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c18073bd0 con 0x7f7c18101970 2024-06-06T08:13:10.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.745+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18067f30 con 0x7f7c18101970 2024-06-06T08:13:10.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.747+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7c08010ac0 con 0x7f7c18101970 2024-06-06T08:13:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.747+0000 7f7c0ffff640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 0x7f7c040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.747+0000 7f7c0dd9b640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 0x7f7c040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.747+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7f7c080cfc40 con 0x7f7c18101970 2024-06-06T08:13:10.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.748+0000 7f7c0dd9b640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 0x7f7c040a51c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c181011f0 tx=0x7f7c0000b410 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:10.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.752+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c0809c430 con 0x7f7c18101970 2024-06-06T08:13:10.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.948+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c1802d2e0 con 0x7f7c18101970 2024-06-06T08:13:10.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.948+0000 7f7c0ffff640 1 -- 172.21.15.87:0/2603055196 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7f7c0809bd30 con 0x7f7c18101970 2024-06-06T08:13:10.948 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 msgr2=0x7f7c040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 0x7f7c040a51c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c181011f0 tx=0x7f7c0000b410 comp rx=0 tx=0).stop 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 msgr2=0x7f7c18198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e9f0 tx=0x7f7c0800eeb0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 shutdown_connections 2024-06-06T08:13:10.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f7c040a2d00 0x7f7c040a51c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c18107980 0x7f7c1819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c18102330 0x7f7c18198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 --2- 172.21.15.87:0/2603055196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c18101970 0x7f7c18198250 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 >> 172.21.15.87:0/2603055196 conn(0x7f7c18078df0 msgr2=0x7f7c180ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 shutdown_connections 2024-06-06T08:13:10.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:10.950+0000 7f7c1cc00640 1 -- 172.21.15.87:0/2603055196 wait complete. 2024-06-06T08:13:10.961 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T08:13:11.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 -- 172.21.15.87:0/893453706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 msgr2=0x7fcddc106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 --2- 172.21.15.87:0/893453706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc106c70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009930 tx=0x7fcdcc02a420 comp rx=0 tx=0).stop 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 -- 172.21.15.87:0/893453706 shutdown_connections 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 --2- 172.21.15.87:0/893453706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc101240 0x7fcddc10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 --2- 172.21.15.87:0/893453706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc100880 0x7fcddc100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 --2- 172.21.15.87:0/893453706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc106c70 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 -- 172.21.15.87:0/893453706 >> 172.21.15.87:0/893453706 conn(0x7fcddc076c20 msgr2=0x7fcddc077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 -- 172.21.15.87:0/893453706 shutdown_connections 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.065+0000 7fcde20e7640 1 -- 172.21.15.87:0/893453706 wait complete. 2024-06-06T08:13:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 Processor -- start 2024-06-06T08:13:11.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 -- start start 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc100880 0x7fcddc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 0x7fcddc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc111da0 con 0x7fcddc106890 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc111c20 con 0x7fcddc101240 2024-06-06T08:13:11.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcde20e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc111f20 con 0x7fcddc100880 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50276/0 (socket says 172.21.15.87:50276) 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.066+0000 7fcddbfff640 1 -- 172.21.15.87:0/3983248369 learned_addr learned my addr 172.21.15.87:0/3983248369 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddb7fe640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc100880 0x7fcddc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddaffd640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 0x7fcddc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 -- 172.21.15.87:0/3983248369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc100880 msgr2=0x7fcddc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc100880 0x7fcddc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 -- 172.21.15.87:0/3983248369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 msgr2=0x7fcddc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 0x7fcddc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 -- 172.21.15.87:0/3983248369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc19d190 con 0x7fcddc106890 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddaffd640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 0x7fcddc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcddbfff640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400cce0 tx=0x7fcdd400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400f450 con 0x7fcddc106890 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd400fad0 con 0x7fcddc106890 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4018c10 con 0x7fcddc106890 2024-06-06T08:13:11.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc19d480 con 0x7fcddc106890 2024-06-06T08:13:11.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.067+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a4d10 con 0x7fcddc106890 2024-06-06T08:13:11.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.068+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc068c10 con 0x7fcddc106890 2024-06-06T08:13:11.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.069+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcdd400f5f0 con 0x7fcddc106890 2024-06-06T08:13:11.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.070+0000 7fcdd8ff9640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 0x7fcdc40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.070+0000 7fcddb7fe640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 0x7fcdc40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.070+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7fcdd40cfef0 con 0x7fcddc106890 2024-06-06T08:13:11.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.070+0000 7fcddb7fe640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 0x7fcdc40a5240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc000c00 tx=0x7fcdcc031490 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.074+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcdd4014030 con 0x7fcddc106890 2024-06-06T08:13:11.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.257+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcddc02d2e0 con 0x7fcddc106890 2024-06-06T08:13:11.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.257+0000 7fcdd8ff9640 1 -- 172.21.15.87:0/3983248369 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) ==== 104+0+11 (secure 0 0 0) 0x7fcdd409c6e0 con 0x7fcddc106890 2024-06-06T08:13:11.257 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 msgr2=0x7fcdc40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 0x7fcdc40a5240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc000c00 tx=0x7fcdcc031490 comp rx=0 tx=0).stop 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 msgr2=0x7fcddc19ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400cce0 tx=0x7fcdd400a620 comp rx=0 tx=0).stop 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 shutdown_connections 2024-06-06T08:13:11.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcdc40a2d80 0x7fcdc40a5240 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcddc106890 0x7fcddc19ca60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc101240 0x7fcddc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 --2- 172.21.15.87:0/3983248369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcddc100880 0x7fcddc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 >> 172.21.15.87:0/3983248369 conn(0x7fcddc076c20 msgr2=0x7fcddc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 shutdown_connections 2024-06-06T08:13:11.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.259+0000 7fcde20e7640 1 -- 172.21.15.87:0/3983248369 wait complete. 2024-06-06T08:13:11.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:11 smithi087 ceph-mon[32276]: from='client.18777 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:11 smithi087 ceph-mon[32276]: from='client.18783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3991254636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:13:11.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2603055196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:13:11.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T08:13:11.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 -- 172.21.15.87:0/3804495033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84069fa0 msgr2=0x7f0a8406a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/3804495033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84069fa0 0x7f0a8406a420 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009930 tx=0x7f0a7802a420 comp rx=0 tx=0).stop 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 -- 172.21.15.87:0/3804495033 shutdown_connections 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/3804495033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a8406a960 0x7f0a8410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/3804495033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84069fa0 0x7f0a8406a420 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/3804495033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84107980 0x7f0a84107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 -- 172.21.15.87:0/3804495033 >> 172.21.15.87:0/3804495033 conn(0x7f0a840766a0 msgr2=0x7f0a84076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 -- 172.21.15.87:0/3804495033 shutdown_connections 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.374+0000 7f0a8a84d640 1 -- 172.21.15.87:0/3804495033 wait complete. 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 Processor -- start 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 -- start start 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a84069fa0 0x7f0a841982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8406a960 0x7f0a84198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111d90 con 0x7f0a84107980 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111c10 con 0x7f0a84069fa0 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a8a84d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111f10 con 0x7f0a8406a960 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a88dea640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a88dea640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50294/0 (socket says 172.21.15.87:50294) 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.375+0000 7f0a88dea640 1 -- 172.21.15.87:0/1237530146 learned_addr learned my addr 172.21.15.87:0/1237530146 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a83fff640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a84069fa0 0x7f0a841982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a837fe640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8406a960 0x7f0a84198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 -- 172.21.15.87:0/1237530146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8406a960 msgr2=0x7f0a84198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8406a960 0x7f0a84198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 -- 172.21.15.87:0/1237530146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a84069fa0 msgr2=0x7f0a841982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a84069fa0 0x7f0a841982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 -- 172.21.15.87:0/1237530146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8419d2e0 con 0x7f0a84107980 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a88dea640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00cad0 tx=0x7f0a7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00ae20 con 0x7f0a84107980 2024-06-06T08:13:11.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c005020 con 0x7f0a84107980 2024-06-06T08:13:11.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8419d5d0 con 0x7f0a84107980 2024-06-06T08:13:11.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00f6f0 con 0x7f0a84107980 2024-06-06T08:13:11.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.376+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a841a4e60 con 0x7f0a84107980 2024-06-06T08:13:11.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.377+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a841030d0 con 0x7f0a84107980 2024-06-06T08:13:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.378+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0a7c0051d0 con 0x7f0a84107980 2024-06-06T08:13:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.379+0000 7f0a817fa640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 0x7f0a6c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.379+0000 7f0a83fff640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 0x7f0a6c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.379+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7f0a7c0cf940 con 0x7f0a84107980 2024-06-06T08:13:11.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.379+0000 7f0a83fff640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 0x7f0a6c0a51c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a7400b390 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.383+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a7c014030 con 0x7f0a84107980 2024-06-06T08:13:11.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:11 smithi195 ceph-mon[37040]: from='client.18777 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:11 smithi195 ceph-mon[37040]: from='client.18783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3991254636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:13:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2603055196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:13:11.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:11 smithi181 ceph-mon[36790]: from='client.18777 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:11 smithi181 ceph-mon[36790]: from='client.18783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:13:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3991254636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:13:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2603055196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:13:11.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.683+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a84199640 con 0x7f0a84107980 2024-06-06T08:13:11.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.685+0000 7f0a817fa640 1 -- 172.21.15.87:0/1237530146 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) ==== 100+0+11 (secure 0 0 0) 0x7f0a7c09c130 con 0x7f0a84107980 2024-06-06T08:13:11.684 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 msgr2=0x7f0a6c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 0x7f0a6c0a51c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a7400b390 comp rx=0 tx=0).stop 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 msgr2=0x7f0a8419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00cad0 tx=0x7f0a7c00cf90 comp rx=0 tx=0).stop 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 shutdown_connections 2024-06-06T08:13:11.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f0a6c0a2d00 0x7f0a6c0a51c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a84107980 0x7f0a8419cbb0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8406a960 0x7f0a84198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 --2- 172.21.15.87:0/1237530146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a84069fa0 0x7f0a841982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 >> 172.21.15.87:0/1237530146 conn(0x7f0a840766a0 msgr2=0x7f0a840fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 shutdown_connections 2024-06-06T08:13:11.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.690+0000 7f0a8a84d640 1 -- 172.21.15.87:0/1237530146 wait complete. 2024-06-06T08:13:11.708 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T08:13:11.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f299f7fe640 1 -- 172.21.15.87:0/3484185812 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900319b0 con 0x7f29a0105d30 2024-06-06T08:13:11.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3484185812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0105d30 msgr2=0x7f29a01061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3484185812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0105d30 0x7f29a01061c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a540 comp rx=0 tx=0).stop 2024-06-06T08:13:11.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3484185812 shutdown_connections 2024-06-06T08:13:11.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3484185812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0105d30 0x7f29a01061c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3484185812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0101330 0x7f29a01057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3484185812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.884+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3484185812 >> 172.21.15.87:0/3484185812 conn(0x7f29a00779a0 msgr2=0x7f29a0077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:11.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.890+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3484185812 shutdown_connections 2024-06-06T08:13:11.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.891+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3484185812 wait complete. 2024-06-06T08:13:11.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 Processor -- start 2024-06-06T08:13:11.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 -- start start 2024-06-06T08:13:11.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0101330 0x7f29a00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 0x7f29a010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01073d0 con 0x7f29a0105d30 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0107250 con 0x7f29a0101330 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.892+0000 7f29a6cd4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0107550 con 0x7f29a0100a10 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a5271640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 0x7f29a010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a5271640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 0x7f29a010ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50318/0 (socket says 172.21.15.87:50318) 2024-06-06T08:13:11.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a5271640 1 -- 172.21.15.87:0/3647301346 learned_addr learned my addr 172.21.15.87:0/3647301346 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f299ffff640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0101330 0x7f29a00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 -- 172.21.15.87:0/3647301346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0101330 msgr2=0x7f29a00728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0101330 0x7f29a00728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 -- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 msgr2=0x7f29a010ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 0x7f29a010ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:11.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.893+0000 7f29a4a70640 1 -- 172.21.15.87:0/3647301346 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a010b390 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.894+0000 7f29a4a70640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0072360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f299800eb00 tx=0x7f299800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.894+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998010430 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.894+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a010b680 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.895+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2998010ab0 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.895+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998017bb0 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.895+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a0112e80 con 0x7f29a0100a10 2024-06-06T08:13:11.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.896+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a00687c0 con 0x7f29a0100a10 2024-06-06T08:13:11.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.897+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2998010c60 con 0x7f29a0100a10 2024-06-06T08:13:11.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.898+0000 7f299dffb640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 0x7f29880a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:11.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.898+0000 7f299ffff640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 0x7f29880a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:11.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.898+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7f29980d0230 con 0x7f29a0100a10 2024-06-06T08:13:11.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.898+0000 7f299ffff640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 0x7f29880a53c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a0078070 tx=0x7f299400a2d0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:11.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:11.903+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f299809c9c0 con 0x7f29a0100a10 2024-06-06T08:13:12.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.079+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f29a01131a0 con 0x7f29a0100a10 2024-06-06T08:13:12.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.080+0000 7f299dffb640 1 -- 172.21.15.87:0/3647301346 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v143) ==== 103+0+11 (secure 0 0 0) 0x7f299809c2c0 con 0x7f29a0100a10 2024-06-06T08:13:12.079 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-06-06T08:13:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 msgr2=0x7f29880a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 0x7f29880a53c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a0078070 tx=0x7f299400a2d0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 msgr2=0x7f29a0072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0072360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f299800eb00 tx=0x7f299800c2d0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 shutdown_connections 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f29880a2f00 0x7f29880a53c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29a0105d30 0x7f29a010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a0101330 0x7f29a00728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 --2- 172.21.15.87:0/3647301346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a0100a10 0x7f29a0072360 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 >> 172.21.15.87:0/3647301346 conn(0x7f29a00779a0 msgr2=0x7f29a01037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 shutdown_connections 2024-06-06T08:13:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.082+0000 7f29a6cd4640 1 -- 172.21.15.87:0/3647301346 wait complete. 2024-06-06T08:13:12.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-06T08:13:12.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 -- 172.21.15.87:0/60898454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c101230 msgr2=0x7f183c10d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:12.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 --2- 172.21.15.87:0/60898454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c101230 0x7f183c10d740 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1834009cd0 tx=0x7f183402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 -- 172.21.15.87:0/60898454 shutdown_connections 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 --2- 172.21.15.87:0/60898454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c101230 0x7f183c10d740 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 --2- 172.21.15.87:0/60898454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c100870 0x7f183c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 --2- 172.21.15.87:0/60898454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 0x7f183c106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 -- 172.21.15.87:0/60898454 >> 172.21.15.87:0/60898454 conn(0x7f183c076c20 msgr2=0x7f183c077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 -- 172.21.15.87:0/60898454 shutdown_connections 2024-06-06T08:13:12.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.193+0000 7f1841163640 1 -- 172.21.15.87:0/60898454 wait complete. 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 Processor -- start 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 -- start start 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c101230 0x7f183c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 0x7f183c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f183c110c90 con 0x7f183c100870 2024-06-06T08:13:12.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f183c110b10 con 0x7f183c106880 2024-06-06T08:13:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1841163640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f183c110e10 con 0x7f183c101230 2024-06-06T08:13:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f183ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f183ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50332/0 (socket says 172.21.15.87:50332) 2024-06-06T08:13:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f183ad9d640 1 -- 172.21.15.87:0/3006039430 learned_addr learned my addr 172.21.15.87:0/3006039430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:12.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f183b59e640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 0x7f183c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.194+0000 7f1833fff640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c101230 0x7f183c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 -- 172.21.15.87:0/3006039430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c101230 msgr2=0x7f183c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c101230 0x7f183c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 -- 172.21.15.87:0/3006039430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 msgr2=0x7f183c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 0x7f183c19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:12.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 -- 172.21.15.87:0/3006039430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c19d250 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f183ad9d640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f182c00d950 tx=0x7f182c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c017030 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182c004020 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c19d540 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0047a0 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.195+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1a4dd0 con 0x7f183c100870 2024-06-06T08:13:12.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.196+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c101fb0 con 0x7f183c100870 2024-06-06T08:13:12.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.197+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f182c0041d0 con 0x7f183c100870 2024-06-06T08:13:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.198+0000 7f1838d99640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 0x7f18280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.198+0000 7f1833fff640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 0x7f18280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.198+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9595+0+0 (secure 0 0 0) 0x7f182c0cf810 con 0x7f183c100870 2024-06-06T08:13:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.198+0000 7f1833fff640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 0x7f18280a5240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1824005e10 tx=0x7f1824005da0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:12.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.201+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f182c09c080 con 0x7f183c100870 2024-06-06T08:13:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:12 smithi087 ceph-mon[32276]: pgmap v801: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-06-06T08:13:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3983248369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:13:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1237530146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:13:12.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:12 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3647301346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:13:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.429+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f183c068c10 con 0x7f183c100870 2024-06-06T08:13:12.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.438+0000 7f1838d99640 1 -- 172.21.15.87:0/3006039430 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740904 (secure 0 0 0) 0x7f182c09b980 con 0x7f183c100870 2024-06-06T08:13:12.448 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:13:12.448 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:12.448 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 26, 2024-06-06T08:13:12.448 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.448 INFO:teuthology.orchestra.run.smithi087.stdout: "active_gid": 24800, 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "active_name": "x", 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "active_addrs": { 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:6800", 2024-06-06T08:13:12.449 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 4062973588 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:6801", 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 4062973588 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.450 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: "active_addr": "172.21.15.87:6801/4062973588", 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: "active_change": "2024-06-06T07:48:00.386150+0000", 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: "active_mgr_features": 4540701547738038271, 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: "available": true, 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: "standbys": [ 2024-06-06T08:13:12.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "gid": 24817, 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "y", 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr_features": 4540701547738038271, 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-06-06T08:13:12.452 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.454 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.455 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.456 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.457 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.458 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.459 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.460 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.461 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.462 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.462 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.462 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.462 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.462 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.463 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.464 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.465 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-06-06T08:13:12.466 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-06-06T08:13:12.466 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.466 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.466 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.466 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.468 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-06-06T08:13:12.469 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-06-06T08:13:12.470 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.471 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.472 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-06-06T08:13:12.472 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.472 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.472 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.473 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.475 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.476 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.477 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.478 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.479 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.480 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.481 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.482 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.483 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.484 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.485 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-06-06T08:13:12.486 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:12.488 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-06-06T08:13:12.489 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-06-06T08:13:12.490 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-06-06T08:13:12.491 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.492 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:12.493 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:13:12.494 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.495 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.496 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:12.497 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.499 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.500 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.501 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.502 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.504 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.505 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.506 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.507 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-06-06T08:13:12.508 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.509 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.510 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-06-06T08:13:12.511 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-06-06T08:13:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.513 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.514 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:12.515 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.516 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.517 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:13:12.518 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.519 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.520 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.521 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:12.522 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.524 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-06-06T08:13:12.525 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.526 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:13:12.527 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.527 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.527 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.527 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.528 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-06-06T08:13:12.529 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.530 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.532 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.533 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:13:12.533 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.533 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.533 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.534 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-06-06T08:13:12.535 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.536 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.537 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.538 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.538 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:13:12.538 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.538 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.538 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.540 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.541 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.542 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-06-06T08:13:12.543 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:12.544 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.545 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.546 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.547 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.548 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.549 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.551 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.552 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.553 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.554 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.555 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.556 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.557 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.558 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:13:12.559 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.561 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-06-06T08:13:12.562 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.563 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-06-06T08:13:12.564 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.565 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.566 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:12.567 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.568 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_samba": { 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_samba", 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.570 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-06-06T08:13:12.571 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.572 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-06-06T08:13:12.573 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.574 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.575 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:13:12.576 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.577 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.578 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.579 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-06-06T08:13:12.581 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.583 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-06-06T08:13:12.584 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.585 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:13:12.586 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:13:12.587 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.588 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.589 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.590 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.591 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.592 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.594 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.595 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.596 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:13:12.597 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.598 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.599 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.600 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.601 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.602 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.603 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.603 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.603 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.603 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.604 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:13:12.605 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.607 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:13:12.608 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:13:12.608 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.608 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.608 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.608 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.609 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.610 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.611 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.611 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.611 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.611 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.612 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.613 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.614 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.615 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.616 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.617 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.618 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-06-06T08:13:12.619 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:13:12.621 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.622 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.623 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.624 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.625 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.626 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.627 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.628 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.629 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:13:12.630 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:13:12.631 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.632 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.633 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.634 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.635 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.636 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.637 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.638 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.638 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-06-06T08:13:12.638 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.639 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:13:12.640 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.640 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.640 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.640 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.641 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.642 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.642 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:13:12.642 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.642 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.642 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.643 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-06-06T08:13:12.643 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:13:12.643 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.643 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.644 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:13:12.645 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.645 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.645 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.645 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-06-06T08:13:12.646 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:13:12.646 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.646 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.646 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.646 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.647 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.647 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.647 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.647 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:13:12.647 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.648 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.649 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.650 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.651 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.652 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.653 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.654 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.655 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.656 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.657 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-06-06T08:13:12.658 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.659 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.660 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.661 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.662 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.663 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.665 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.667 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:13:12.667 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:13:12.667 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.667 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.667 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.668 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.669 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.670 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.671 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.673 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.674 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:13:12.675 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.676 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:13:12.677 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.678 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:13:12.679 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:13:12.680 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.680 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.680 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.680 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.680 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.681 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.682 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:13:12.682 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:13:12.682 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.685 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.686 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.687 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.688 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.689 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.690 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.691 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:13:12.692 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.693 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.694 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.695 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.696 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.697 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.698 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.699 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:12.700 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.702 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.704 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.705 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.706 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.707 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.708 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.709 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.710 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.711 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:13:12.712 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.713 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:13:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.715 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.717 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:13:12.718 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.719 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.720 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.721 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.721 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.721 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.721 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.721 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:13:12.722 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.723 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.724 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.725 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.726 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.727 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.728 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.729 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.730 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.731 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.732 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.733 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.734 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.735 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.736 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.737 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.738 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.740 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.741 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.742 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:13:12.743 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.744 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.745 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.746 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.747 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.748 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.749 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.750 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.751 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.753 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.755 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-06-06T08:13:12.755 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-06-06T08:13:12.755 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.755 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.755 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.756 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.757 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.758 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.759 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.761 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.762 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.764 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.765 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.766 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.767 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.768 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.769 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-06-06T08:13:12.770 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-06-06T08:13:12.771 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.772 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.773 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.774 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.774 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.774 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:12.774 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.775 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:12.776 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:12.777 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.777 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.777 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.777 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-06-06T08:13:12.777 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.779 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.780 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.781 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.782 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.783 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.784 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.785 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.786 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:12.786 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:12.786 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.786 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.787 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.788 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.789 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.790 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.791 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.792 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.793 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.794 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.795 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.796 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.797 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.798 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.799 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.800 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.801 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.802 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.803 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.804 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-06-06T08:13:12.805 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:13:12.806 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.807 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.808 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.809 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.810 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.811 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.812 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.813 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:13:12.814 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.815 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.816 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.817 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.818 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.819 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.821 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.822 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.823 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.824 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.824 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.824 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.824 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.824 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.825 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.826 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.827 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.828 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.829 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.830 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.831 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.832 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-06-06T08:13:12.833 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.834 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.835 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.838 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-06-06T08:13:12.840 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.841 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.842 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.843 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.845 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.846 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.847 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.848 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.849 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.850 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.851 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-06-06T08:13:12.852 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.853 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.853 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.853 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.853 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.853 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.854 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.854 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:13:12.854 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.854 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.855 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-06-06T08:13:12.857 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:12.859 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.860 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.861 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.862 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.865 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.866 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.867 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.868 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.869 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.870 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.871 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.872 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.875 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.876 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.877 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.878 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.879 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.880 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.881 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.882 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.884 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.885 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.886 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:13:12.887 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.888 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-06-06T08:13:12.889 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.890 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.891 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.892 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.893 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.894 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.895 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.896 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.897 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.898 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.899 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-06-06T08:13:12.900 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.901 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.902 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.903 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.904 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.904 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.904 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.904 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.904 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.905 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.906 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.907 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.908 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.909 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.910 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.912 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.913 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:13:12.914 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.915 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:13:12.916 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:12.917 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.918 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.919 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.920 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.921 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.923 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.924 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.925 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.926 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.927 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.928 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.929 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.930 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.932 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.933 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.934 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.935 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.936 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.937 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.938 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.939 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.940 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.941 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.942 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.943 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.944 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.945 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.946 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.947 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.949 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.950 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.951 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.952 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.953 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.954 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.955 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.956 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.957 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.958 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.960 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.961 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.962 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.963 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.964 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-06-06T08:13:12.965 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.967 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.968 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.969 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.970 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.971 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.972 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.973 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.974 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.975 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.976 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-06-06T08:13:12.977 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.978 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.979 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.981 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.982 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.983 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.984 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.985 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.986 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:12.987 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.988 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.989 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:12.992 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:12.993 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.994 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.995 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.996 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.997 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:12.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-06-06T08:13:12.999 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:13:13.001 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:13:13.002 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.003 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.004 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.005 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.006 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.007 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.008 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.009 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.010 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.011 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.012 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-06-06T08:13:13.014 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.015 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.016 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.017 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-06-06T08:13:13.018 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.019 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.020 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:13:13.021 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.022 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.023 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.025 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.026 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.027 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.028 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.029 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.030 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.031 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.032 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.033 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.035 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:13.036 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:13:13.037 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.038 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.039 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.040 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.042 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.043 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:13.044 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.045 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.046 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.047 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.048 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.050 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.051 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.052 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.054 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-06-06T08:13:13.055 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.056 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:13:13.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.058 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.059 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.060 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-06-06T08:13:13.061 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.062 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.064 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.066 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.067 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.068 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.069 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.070 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.071 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.072 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.073 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.074 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.075 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.076 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.077 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.079 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.080 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.081 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.082 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.083 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.084 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.085 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.086 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.087 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.089 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.090 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.094 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.094 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.094 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.094 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.095 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.096 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.097 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.098 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.098 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.098 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.098 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.099 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.100 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.101 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.103 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.104 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.105 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.106 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.107 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.108 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.109 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.110 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.111 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.112 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.113 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.114 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.115 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-06-06T08:13:13.116 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.117 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.118 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.119 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.120 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.122 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.123 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.124 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-06-06T08:13:13.125 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.126 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.127 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.128 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.129 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.130 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-06-06T08:13:13.132 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.133 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.134 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.136 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.137 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-06-06T08:13:13.138 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.139 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.140 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.141 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.142 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.143 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.144 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.145 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.146 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.147 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.148 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.149 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.150 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.151 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.153 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.154 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.155 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.156 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.157 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.158 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-06-06T08:13:13.159 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.160 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.161 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.162 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.163 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.164 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.165 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.166 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.167 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.168 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-06-06T08:13:13.169 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.170 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.171 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.172 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.174 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.175 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.177 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.178 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.179 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.181 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.183 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.185 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-06-06T08:13:13.186 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.187 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.188 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.189 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.190 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-06-06T08:13:13.191 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.192 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.193 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.194 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:13:13.195 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.196 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.197 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.198 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.199 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.200 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.201 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.202 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.203 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.204 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.206 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.207 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.208 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.209 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.211 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.212 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.213 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.215 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.216 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.217 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-06-06T08:13:13.218 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.219 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.220 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.221 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.222 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.223 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.224 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.225 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.226 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.227 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.228 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.229 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-06-06T08:13:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.232 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.233 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.233 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.233 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.233 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.233 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.234 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.235 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.237 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.238 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.240 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-06-06T08:13:13.241 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:13:13.242 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.243 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.244 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.245 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:13:13.246 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.247 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:13.249 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.250 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.251 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.253 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.254 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.255 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.256 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.257 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.258 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.259 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-06-06T08:13:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: "gid": 24823, 2024-06-06T08:13:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "z", 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr_features": 4540701547738038271, 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.272 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.273 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.274 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.275 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.276 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.277 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.278 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.279 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.280 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-06-06T08:13:13.281 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.282 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-06-06T08:13:13.283 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-06-06T08:13:13.284 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.285 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.286 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.287 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.289 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-06-06T08:13:13.290 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.291 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.292 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.293 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.294 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.296 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.297 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.298 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.299 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-06-06T08:13:13.300 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:13:13.301 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.302 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.303 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:13:13.304 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-06-06T08:13:13.306 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.307 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.308 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-06-06T08:13:13.309 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:13.310 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.311 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.314 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.315 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.316 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.317 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.318 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.319 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.320 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.321 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.322 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.323 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.325 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-06-06T08:13:13.326 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.328 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:13.329 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.330 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.331 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:13:13.332 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:13:13.333 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.334 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:13.335 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.336 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.337 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-06-06T08:13:13.338 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:13:13.339 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.340 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:13:13.341 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.342 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-06-06T08:13:13.344 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.345 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.346 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-06-06T08:13:13.348 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.349 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-06-06T08:13:13.350 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:13:13.351 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.352 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.353 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.354 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.356 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:13:13.357 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-06-06T08:13:13.358 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.359 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.360 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:13:13.361 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.362 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.363 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-06-06T08:13:13.364 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.365 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.366 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-06-06T08:13:13.367 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.368 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.369 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:13:13.370 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-06-06T08:13:13.373 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.374 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-06-06T08:13:13.375 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:13.376 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.377 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.378 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_samba": { 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_samba", 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.379 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.380 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.381 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-06-06T08:13:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.383 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.384 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:13:13.385 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.387 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.388 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-06-06T08:13:13.391 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.392 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.394 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.395 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.396 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:13.397 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.398 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.399 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-06-06T08:13:13.400 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:13:13.401 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.403 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.405 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.406 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.407 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.408 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.409 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.410 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.411 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.412 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.413 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.414 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-06-06T08:13:13.415 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.417 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-06-06T08:13:13.418 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:13:13.419 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.420 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.421 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-06-06T08:13:13.422 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.423 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.424 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.426 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-06-06T08:13:13.427 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:13:13.428 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-06-06T08:13:13.429 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:13:13.430 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.431 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.432 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.433 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.434 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.435 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-06-06T08:13:13.436 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.437 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.440 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-06-06T08:13:13.441 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.442 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.443 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.444 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.445 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.447 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.450 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.451 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.452 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.453 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.454 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.455 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.456 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.457 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.458 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:13:13.459 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.460 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.461 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.462 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.463 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.464 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.465 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.466 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.467 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.468 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.469 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.470 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.471 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.473 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.477 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.478 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.479 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.480 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.481 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:13:13.482 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.483 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.484 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.488 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:13:13.489 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:13:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.492 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.493 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.494 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.495 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.496 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.497 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.498 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.499 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:13:13.499 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.502 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:13:13.503 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.504 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.505 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.506 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.507 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.509 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:13:13.510 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.511 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.512 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.513 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.514 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.516 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.517 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.518 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.519 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.520 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.521 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.522 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.523 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.524 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.525 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.526 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.527 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:13:13.527 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.527 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.527 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.527 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:13:13.528 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.529 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:13:13.530 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.531 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.532 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.533 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.534 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.535 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.536 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:13:13.537 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.538 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.540 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:13.541 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.543 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:13:13.544 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.545 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.546 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.548 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.550 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.551 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.552 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.554 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.555 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.556 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.557 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.558 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.559 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.560 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.561 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.562 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.563 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-06-06T08:13:13.564 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-06-06T08:13:13.565 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.566 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.567 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.568 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.569 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.570 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:13.571 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.572 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.573 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.574 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.575 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.576 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.577 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.578 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.579 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.580 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.581 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.582 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.583 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.584 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.585 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.586 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.587 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.588 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.589 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.590 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.591 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.592 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.593 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.596 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-06-06T08:13:13.597 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.598 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.599 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:13:13.600 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.601 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.602 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-06-06T08:13:13.603 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.604 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.605 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.606 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.607 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.608 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.609 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.610 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.611 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.612 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.613 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.615 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.616 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.617 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.618 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.619 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.621 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.622 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.623 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.624 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.625 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.627 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.628 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.629 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.630 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.631 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.633 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.634 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.635 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.636 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.637 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.638 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-06-06T08:13:13.639 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.640 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.641 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:13.643 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.644 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.645 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-06-06T08:13:13.646 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:13.647 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-06-06T08:13:13.648 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.649 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.650 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.651 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.652 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.653 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.654 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.655 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.656 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.657 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.659 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.660 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.662 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.663 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.664 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.665 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.666 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.667 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-06-06T08:13:13.668 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-06-06T08:13:13.669 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.671 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.673 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.674 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.675 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.676 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.677 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.678 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.679 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.680 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-06-06T08:13:13.681 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.682 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.684 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.685 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.686 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.687 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.688 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.689 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.691 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-06-06T08:13:13.692 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.693 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-06-06T08:13:13.694 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.695 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.696 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.697 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-06-06T08:13:13.698 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:13:13.699 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.700 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.701 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.702 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.703 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.704 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.705 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.706 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.710 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.710 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.710 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.710 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.711 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.712 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.713 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.714 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.715 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.716 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.717 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.718 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.719 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.720 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.721 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.722 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.724 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.726 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.727 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.728 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.729 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.730 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.731 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.732 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.733 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.734 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.735 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.736 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.737 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:13:13.738 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.739 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.740 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.741 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.742 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.743 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.744 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.745 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.746 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.747 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.748 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-06-06T08:13:13.749 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-06-06T08:13:13.750 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.751 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.752 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.753 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.754 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.755 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.756 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.757 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.758 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.759 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.760 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.761 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.762 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.763 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.764 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.765 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.766 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.767 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.768 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.769 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.770 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.771 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.772 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.773 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.774 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.775 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.776 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.778 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.779 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.780 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.781 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:13:13.782 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.783 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.784 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.785 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.786 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.787 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.788 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.789 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.790 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.792 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.793 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.794 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.795 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.796 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.798 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-06-06T08:13:13.800 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.801 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.802 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.804 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.805 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.806 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.807 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.809 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.810 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.811 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.812 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.813 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.814 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.815 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.816 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.817 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.818 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.820 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-06-06T08:13:13.821 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.822 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.823 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-06-06T08:13:13.824 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:13.825 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.826 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.827 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.828 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.829 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.831 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.832 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.833 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.834 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.835 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.837 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:13:13.838 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.839 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.840 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.841 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.842 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.843 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:13.844 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.845 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.846 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.847 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.848 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.849 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.850 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.851 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.852 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.853 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.855 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.857 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.859 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.860 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.861 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.862 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.863 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.864 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.866 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.867 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.868 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.869 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.870 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.871 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.872 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.873 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.874 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.875 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.876 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.877 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.879 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.880 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.881 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.883 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.885 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.887 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-06-06T08:13:13.888 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.889 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.890 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.891 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.892 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.894 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.896 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-06-06T08:13:13.896 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-06-06T08:13:13.896 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:13.896 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.896 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.897 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.898 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.899 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.900 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-06-06T08:13:13.901 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.902 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.903 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.905 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-06-06T08:13:13.906 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.907 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-06-06T08:13:13.908 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.909 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-06-06T08:13:13.910 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.911 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:13:13.912 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-06-06T08:13:13.913 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.914 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.915 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.916 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.918 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.919 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.921 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.922 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.923 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.924 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-06-06T08:13:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.927 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.928 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.929 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.930 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.931 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.932 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.933 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.934 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.935 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.936 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.937 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.938 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.939 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.940 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.941 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.942 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.943 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.944 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.946 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.947 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-06-06T08:13:13.949 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.950 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.951 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.952 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.954 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.955 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.956 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.957 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.958 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.959 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:13.960 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.961 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.963 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:13:13.964 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.966 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.967 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.969 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-06-06T08:13:13.972 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.973 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-06-06T08:13:13.974 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.975 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.976 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.977 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.980 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-06-06T08:13:13.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.982 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.983 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.984 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.985 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.986 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.988 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.989 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:13.990 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:13.991 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.992 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.993 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.994 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.995 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.996 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-06-06T08:13:13.997 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:13.998 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:13.999 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:13:14.000 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.001 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.002 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.003 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.004 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.005 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.006 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.007 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.008 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.010 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.012 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.013 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.014 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.015 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.016 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.017 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.018 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.019 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.020 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-06-06T08:13:14.021 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:13:14.022 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.023 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.024 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.025 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.026 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.027 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-06-06T08:13:14.029 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.030 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.032 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.033 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.034 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.035 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.036 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.037 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.038 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.042 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.043 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.044 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.045 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-06-06T08:13:14.046 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.047 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.048 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.049 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.050 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.050 INFO:teuthology.orchestra.run.smithi087.stdout: "modules": [ 2024-06-06T08:13:14.050 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-06-06T08:13:14.050 INFO:teuthology.orchestra.run.smithi087.stdout: "dashboard", 2024-06-06T08:13:14.050 INFO:teuthology.orchestra.run.smithi087.stdout: "iostat", 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "nfs", 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "restful", 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "snap_schedule" 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-06-06T08:13:14.051 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.052 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:13:14.053 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.054 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.055 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.057 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.058 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.059 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.060 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.061 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.062 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.063 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:13:14.064 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.066 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.067 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.068 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.069 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-06-06T08:13:14.070 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.071 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-06-06T08:13:14.072 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.074 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:13:14.075 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.076 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.078 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.079 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-06-06T08:13:14.080 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.081 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-06-06T08:13:14.082 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.083 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.084 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.085 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.086 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-06-06T08:13:14.088 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-06-06T08:13:14.089 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.090 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-06-06T08:13:14.091 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-06-06T08:13:14.092 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.093 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.095 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.097 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.098 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.099 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.100 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.101 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.102 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-06-06T08:13:14.103 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.105 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-06-06T08:13:14.106 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.108 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:14.109 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.110 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-06-06T08:13:14.111 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:13:14.112 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.113 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.114 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-06-06T08:13:14.118 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.119 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.120 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.121 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:13:14.122 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.123 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:13:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-06-06T08:13:14.125 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:13:14.126 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.127 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.128 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.129 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-06-06T08:13:14.130 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.130 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.130 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.130 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:13:14.130 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.131 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-06-06T08:13:14.132 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.133 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.134 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:13:14.135 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-06-06T08:13:14.136 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:13:14.137 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.138 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.139 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.140 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.141 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.142 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.143 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.144 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-06-06T08:13:14.145 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.146 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.147 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.149 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.150 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.151 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.153 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.154 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.156 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-06-06T08:13:14.156 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-06-06T08:13:14.156 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.156 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.156 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.157 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.158 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-06-06T08:13:14.158 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-06-06T08:13:14.158 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.158 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.159 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.160 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-06-06T08:13:14.161 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-06-06T08:13:14.162 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-06-06T08:13:14.164 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.165 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.166 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_samba": { 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_samba", 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:13:14.167 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.168 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.169 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.170 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.171 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:13:14.172 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:13:14.173 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.174 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.175 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-06-06T08:13:14.176 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.177 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.178 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-06-06T08:13:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.180 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.181 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.182 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:13:14.183 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.184 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.185 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-06-06T08:13:14.186 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:13:14.187 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.189 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.190 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.191 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-06-06T08:13:14.192 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:13:14.193 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.194 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.195 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.196 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.197 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.198 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.199 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:13:14.200 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:13:14.201 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.202 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.203 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.204 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.205 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.206 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-06-06T08:13:14.207 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.208 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.209 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-06-06T08:13:14.210 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-06-06T08:13:14.211 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:13:14.212 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.213 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.214 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.215 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-06-06T08:13:14.216 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.217 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.218 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:13:14.219 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.220 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.222 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.223 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:13:14.225 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.226 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:13:14.227 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.228 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:13:14.229 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.230 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.231 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:13:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:13:14.233 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.234 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.236 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.238 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.239 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.240 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.241 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.242 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.243 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.243 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.243 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.243 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.244 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.245 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.246 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.247 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-06-06T08:13:14.248 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.249 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.250 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-06-06T08:13:14.251 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.252 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.253 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.253 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.253 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:13:14.254 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.255 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.255 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.255 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.255 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.255 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:13:14.256 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.257 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:13:14.258 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.259 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.260 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.261 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:13:14.262 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.263 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.264 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.265 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.266 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.268 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:13:14.269 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.270 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.271 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.272 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.273 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.275 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:13:14.276 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.277 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:13:14.278 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.280 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.281 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.282 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.283 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.284 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:14.286 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.288 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.290 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:13:14.290 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:13:14.290 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.290 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.291 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.292 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.292 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.292 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.292 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.293 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.294 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.294 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.294 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.294 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.294 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.295 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.295 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:13:14.295 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:13:14.295 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.296 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:13:14.297 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.298 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:13:14.299 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.300 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:13:14.301 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.302 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:13:14.303 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.304 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.305 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.306 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.308 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.308 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:13:14.308 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.309 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:13:14.310 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.311 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:13:14.312 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.313 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:13:14.314 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.315 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.316 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.317 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.319 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:13:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:13:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:13:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:13:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-06-06T08:13:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-06-06T08:13:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-06-06T08:13:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-06-06T08:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-06-06T08:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-06-06T08:13:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-06-06T08:13:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-06-06T08:13:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-06-06T08:13:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.381 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-06-06T08:13:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-06-06T08:13:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:13:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-06-06T08:13:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-06-06T08:13:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-06-06T08:13:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.460 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.461 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.461 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.461 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.461 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.462 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.463 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.464 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.465 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.467 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.468 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.469 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.470 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.471 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.472 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.473 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-06-06T08:13:14.474 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.475 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.477 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:13:14.478 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.479 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.481 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.482 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.483 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.484 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.485 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.486 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.486 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.486 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.486 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.486 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.487 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.488 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-06-06T08:13:14.490 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.491 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.492 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.493 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.494 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.494 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.494 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.494 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.494 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.495 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.496 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.497 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.498 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.499 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.500 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.502 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.504 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-06-06T08:13:14.505 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.506 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-06-06T08:13:14.507 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.508 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.509 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.510 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.512 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.513 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.514 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.514 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.514 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.514 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.514 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.516 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.517 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.519 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.520 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.521 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.522 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.523 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.523 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.523 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.527 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.528 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.529 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.530 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.531 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.532 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.533 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.534 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.535 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.536 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-06-06T08:13:14.537 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.538 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.539 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.540 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.541 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.542 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.543 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.544 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.545 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.546 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.549 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.550 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.551 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.552 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.553 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.555 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.556 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.557 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.558 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.559 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.560 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.561 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.563 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.565 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.566 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.567 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.568 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.569 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.570 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.571 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.572 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.573 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.574 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.575 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.576 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.577 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.579 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.580 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.582 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.583 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.584 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.585 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.586 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.587 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.588 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.589 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:14.590 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.591 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.592 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:13:14.593 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.594 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.595 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.598 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.599 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.600 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.601 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.602 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.603 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.604 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.605 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.606 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-06-06T08:13:14.607 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.608 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-06-06T08:13:14.609 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:13:14.610 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-06-06T08:13:14.611 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.612 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.613 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:13:14.615 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.616 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.617 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.618 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.619 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.621 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.622 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.623 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.624 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.625 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.626 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.629 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.631 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-06-06T08:13:14.633 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.634 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.635 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-06-06T08:13:14.636 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-06-06T08:13:14.637 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-06-06T08:13:14.638 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.639 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.640 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.641 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.642 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.643 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.644 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.645 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.646 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.647 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.648 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.649 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.650 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.651 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.652 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.653 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.654 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-06-06T08:13:14.656 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.657 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-06-06T08:13:14.658 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.659 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.660 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.661 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.662 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.663 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.664 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.665 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.666 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.667 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.668 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.669 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.671 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.673 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.674 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.675 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.676 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.678 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.679 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.680 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.681 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.682 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.683 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.684 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.686 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.687 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.688 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.689 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.690 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.691 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.692 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.693 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.694 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.695 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.696 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.697 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.698 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.699 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.700 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.701 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.702 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-06-06T08:13:14.703 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.704 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.705 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.706 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.707 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.708 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.709 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-06-06T08:13:14.710 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.711 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.712 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.713 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.716 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.717 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-06-06T08:13:14.718 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.719 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.720 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.721 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.723 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.725 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.726 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.727 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.729 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.730 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.734 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.736 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.737 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.738 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.739 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.740 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.741 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.742 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.744 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.745 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.746 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.747 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.748 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.749 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.750 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.751 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.752 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.753 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.754 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.755 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.756 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.757 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.758 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.759 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.760 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.762 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.763 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.764 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.765 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.766 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.767 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.768 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.769 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.770 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.772 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.773 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.774 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.775 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.776 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.777 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.777 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.777 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:13:14.777 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.777 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.778 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:13:14.779 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.780 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.781 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.782 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.784 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-06-06T08:13:14.785 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.786 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-06-06T08:13:14.787 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.789 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.790 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-06-06T08:13:14.791 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.793 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-06-06T08:13:14.794 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.795 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.796 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.797 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.798 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.799 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.800 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.801 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.803 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.804 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.805 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.807 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-06-06T08:13:14.808 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.809 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.810 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.811 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.812 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.813 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.814 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.815 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.816 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.817 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.818 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.820 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.821 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.822 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.823 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.824 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.825 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.826 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.827 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.828 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.829 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.830 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.831 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.832 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.833 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.835 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-06-06T08:13:14.836 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.837 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:13:14.838 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:13:14.838 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.838 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.838 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.838 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.839 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-06-06T08:13:14.840 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.841 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.842 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.843 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.844 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.846 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.847 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-06-06T08:13:14.848 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.849 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.850 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-06-06T08:13:14.851 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.852 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-06-06T08:13:14.853 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.855 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-06-06T08:13:14.856 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.857 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-06-06T08:13:14.858 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-06-06T08:13:14.859 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.860 INFO:teuthology.orchestra.run.smithi087.stdout: "services": { 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "dashboard": "https://172.21.15.87:8443/" 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "always_on_modules": { 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "octopus": [ 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-06-06T08:13:14.861 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-06-06T08:13:14.862 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "pacific": [ 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-06-06T08:13:14.863 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "quincy": [ 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-06-06T08:13:14.864 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-06-06T08:13:14.865 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "reef": [ 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-06-06T08:13:14.866 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "squid": [ 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-06-06T08:13:14.867 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-06-06T08:13:14.868 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: "last_failure_osd_epoch": 100, 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: "active_clients": [ 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-06-06T08:13:14.869 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3538421074 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "libcephsqlite", 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2938375011 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2686428460 2024-06-06T08:13:14.872 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.873 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2571634230 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1534174603 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.875 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1884175516 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:14.877 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-06-06T08:13:14.877 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:14.879 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:12 smithi181 ceph-mon[36790]: pgmap v801: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-06-06T08:13:14.879 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:12 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3983248369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.879 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:12 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1237530146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.879 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:12 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3647301346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.879 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:13 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/3006039430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:14 smithi181 ceph-mon[36790]: pgmap v802: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 714 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:12 smithi195 ceph-mon[37040]: pgmap v801: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 716 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:12 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3983248369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:12 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1237530146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:12 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3647301346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:13 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/3006039430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:13:14.880 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:14 smithi195 ceph-mon[37040]: pgmap v802: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 714 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-06-06T08:13:14.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 msgr2=0x7f18280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:14.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 0x7f18280a5240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1824005e10 tx=0x7f1824005da0 comp rx=0 tx=0).stop 2024-06-06T08:13:14.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 msgr2=0x7f183c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:14.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f182c00d950 tx=0x7f182c00de10 comp rx=0 tx=0).stop 2024-06-06T08:13:14.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 shutdown_connections 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f18280a2d80 0x7f18280a5240 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c106880 0x7f183c19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f183c101230 0x7f183c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 --2- 172.21.15.87:0/3006039430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183c100870 0x7f183c198260 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 >> 172.21.15.87:0/3006039430 conn(0x7f183c076c20 msgr2=0x7f183c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 shutdown_connections 2024-06-06T08:13:14.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:12.443+0000 7f1841163640 1 -- 172.21.15.87:0/3006039430 wait complete. 2024-06-06T08:13:14.889 DEBUG:tasks.check_counter:Getting stats from a 2024-06-06T08:13:14.889 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-06T08:13:14.890 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:13 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/3006039430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:13:14.890 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:14 smithi087 ceph-mon[32276]: pgmap v802: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 714 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-06-06T08:13:15.126 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:16 smithi195 ceph-mon[37040]: pgmap v803: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:16.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:16 smithi181 ceph-mon[36790]: pgmap v803: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:16 smithi087 ceph-mon[32276]: pgmap v803: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:17.881 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:17.882 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.001145723, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000206538, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:17.883 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.884 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 485, 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 472, 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1658128, 2024-06-06T08:13:17.885 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 291440, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.070699677, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.027553118, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.040512476, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.008888536, 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:17.886 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 472, 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.020582499, 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000043606 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:17.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1658128, 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 291440 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2, 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 360, 2024-06-06T08:13:17.888 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 320, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 219280, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 5, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.025534939, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.019818773, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002021002, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:17.889 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 357, 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.016794167, 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000047042 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:17.890 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 219280 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-06-06T08:13:17.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:17.892 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:17.893 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:17.894 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148794, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 32, 2024-06-06T08:13:17.896 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:17.897 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-06-06T08:13:17.898 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-06-06T08:13:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 483, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1587080, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 483, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 483, 2024-06-06T08:13:17.900 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1587080, 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:17.901 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:18.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:18 smithi195 ceph-mon[37040]: pgmap v804: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:18.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:18 smithi181 ceph-mon[36790]: pgmap v804: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:18.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:18 smithi087 ceph-mon[32276]: pgmap v804: 129 pgs: 129 active+clean; 711 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 134 op/s 2024-06-06T08:13:18.672 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-06T08:13:18.672 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-06T08:13:18.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-06-06T08:13:18.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-06-06T08:13:18.673 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-06-06T08:13:18.673 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-06-06T08:13:18.673 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-06-06T08:13:18.673 DEBUG:tasks.check_counter:Getting stats from d 2024-06-06T08:13:18.674 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-06T08:13:18.910 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:20.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:20 smithi087 ceph-mon[32276]: pgmap v805: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:20.320 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:20 smithi087 ceph-mon[32276]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T08:13:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:20 smithi195 ceph-mon[37040]: pgmap v805: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:20.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:20 smithi195 ceph-mon[37040]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T08:13:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:20 smithi181 ceph-mon[36790]: pgmap v805: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:20.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:20 smithi181 ceph-mon[36790]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T08:13:20.721 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:20.721 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:20.721 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-06-06T08:13:20.721 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-06-06T08:13:20.721 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.001114363, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000240497, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:20.722 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:20.723 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 3, 2024-06-06T08:13:20.724 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 365, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 480, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 298384, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.025593113, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.019560228, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002224648, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:20.725 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 362, 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.028678574, 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000079222 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:20.726 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 298384 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 491, 2024-06-06T08:13:20.727 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 478, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1659760, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 295280, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.071938116, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.029075553, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.039079099, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.009089648, 2024-06-06T08:13:20.728 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 478, 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.023310111, 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000048765 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:20.729 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1659760, 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 295280 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-06-06T08:13:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:20.731 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:20.732 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:20.733 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:20.734 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148748, 2024-06-06T08:13:20.735 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 32, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:20.736 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-06-06T08:13:20.737 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-06-06T08:13:20.738 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 490, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1587887, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 490, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 490, 2024-06-06T08:13:20.739 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1587887, 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:20.740 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:21.806 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-06-06T08:13:21.807 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-06-06T08:13:21.807 DEBUG:tasks.check_counter:Getting stats from g 2024-06-06T08:13:21.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-06T08:13:22.046 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:22.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:22 smithi087 ceph-mon[32276]: pgmap v807: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:22.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:22 smithi195 ceph-mon[37040]: pgmap v807: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:22.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:22 smithi181 ceph-mon[36790]: pgmap v807: 129 pgs: 129 active+clean; 710 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-06-06T08:13:24.414 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:24 smithi087 ceph-mon[32276]: pgmap v808: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-06-06T08:13:24.414 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 484, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 473, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1655872, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 292912, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.072466633, 2024-06-06T08:13:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.027808752, 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.039295695, 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.009149181, 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 472, 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.026321036, 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000055764 2024-06-06T08:13:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1655872, 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 292912 2024-06-06T08:13:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 7, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 364, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 3552, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 298304, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.023888456, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.018076169, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002423479, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 362, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.024004210, 2024-06-06T08:13:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000066309 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 298304 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-06-06T08:13:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.000441961, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000348906, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-06-06T08:13:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148714, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 30, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-06-06T08:13:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 487, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1587497, 2024-06-06T08:13:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 487, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 487, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1587497, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:24.435 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:24.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:24 smithi195 ceph-mon[37040]: pgmap v808: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-06-06T08:13:24.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:24 smithi181 ceph-mon[36790]: pgmap v808: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-06-06T08:13:25.227 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-06-06T08:13:25.228 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-06-06T08:13:25.228 DEBUG:tasks.check_counter:Getting stats from j 2024-06-06T08:13:25.229 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-06T08:13:25.475 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:26.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:26 smithi195 ceph-mon[37040]: pgmap v809: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:26.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:26 smithi181 ceph-mon[36790]: pgmap v809: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:26.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:26 smithi087 ceph-mon[32276]: pgmap v809: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:27.329 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-06-06T08:13:27.329 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:27.329 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-06-06T08:13:27.329 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-06-06T08:13:27.329 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.001216125, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000256547, 2024-06-06T08:13:27.330 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:27.331 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 7, 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 370, 2024-06-06T08:13:27.332 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 3552, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 299360, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.025307300, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.019272204, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002466287, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 368, 2024-06-06T08:13:27.333 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.023609447, 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000064156 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.334 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 299360 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 491, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 480, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1658400, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 297088, 2024-06-06T08:13:27.335 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.073912742, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.028930018, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.040577237, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.009264950, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 479, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.019128623, 2024-06-06T08:13:27.336 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000039934 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1658400, 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 297088 2024-06-06T08:13:27.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:27.338 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:27.339 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:27.340 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:27.341 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148771, 2024-06-06T08:13:27.342 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 32, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:27.343 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:27.344 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-06-06T08:13:27.345 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 494, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1589044, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 494, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-06-06T08:13:27.346 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 494, 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1589044, 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-06-06T08:13:27.347 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-06-06T08:13:28.028 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-06T08:13:28.028 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-06T08:13:28.028 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-06-06T08:13:28.028 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-06-06T08:13:28.028 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-06T08:13:28.029 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-06T08:13:28.029 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-06T08:13:28.029 DEBUG:tasks.check_counter:Getting stats from b 2024-06-06T08:13:28.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-06T08:13:28.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:28 smithi087 ceph-mon[32276]: pgmap v810: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:28.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:28 smithi195 ceph-mon[37040]: pgmap v810: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:28.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:28 smithi181 ceph-mon[36790]: pgmap v810: 129 pgs: 129 active+clean; 710 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-06-06T08:13:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:30 smithi181 ceph-mon[36790]: pgmap v811: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-06-06T08:13:30.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:30 smithi195 ceph-mon[37040]: pgmap v811: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-06-06T08:13:30.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:30 smithi087 ceph-mon[32276]: pgmap v811: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-06-06T08:13:31.045 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 362427, 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 552204, 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 77842360, 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 833991594, 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 25, 2024-06-06T08:13:31.046 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 27.501321070, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 11.271029779, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 12.420261643, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 4.540426762, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 552205, 2024-06-06T08:13:31.047 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 111.230143707, 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000201429 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 702291, 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.135143903, 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000192 2024-06-06T08:13:31.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 838368, 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 275280 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 694142, 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1006371, 2024-06-06T08:13:31.049 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 249884859, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1158938558, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 326, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 45.259306038, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 19.615353279, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 28.170012719, 2024-06-06T08:13:31.050 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 4.083497067, 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1006369, 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 149.003422743, 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000148060 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.051 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1373333, 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.222087048, 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000161 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 223159776, 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 754623440 2024-06-06T08:13:31.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 337058, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 542030, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 72215474, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 537551476, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 13, 2024-06-06T08:13:31.053 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 32.138345107, 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 16.566599849, 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 13.466765414, 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 3.387976443, 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:31.054 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 542026, 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 111.313616657, 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000205365 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 703566, 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.126753299, 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000180 2024-06-06T08:13:31.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1914480, 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 328800 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-06-06T08:13:31.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 147259, 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 247.339995727, 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001679625 2024-06-06T08:13:31.057 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 93917, 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 24.317665662, 2024-06-06T08:13:31.058 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000258927 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 422560, 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 414696, 2024-06-06T08:13:31.059 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 414696, 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5405.170196453, 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.013034054 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 7864, 2024-06-06T08:13:31.060 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 4, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 1, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 9428, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 49, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 39, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 9112, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 5147, 2024-06-06T08:13:31.061 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 3954, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 11, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1756, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 621, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 560, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 560, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 34, 2024-06-06T08:13:31.062 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 607994, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 377872, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 7841, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 150, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 3496, 2024-06-06T08:13:31.063 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 24, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 280, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 9, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 135, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:31.064 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 114, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 716, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 214607440, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 7, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 114, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:31.065 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:31.066 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:31.066 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 44, 2024-06-06T08:13:31.066 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 4, 2024-06-06T08:13:31.066 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 40, 2024-06-06T08:13:31.066 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 70, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 13, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 73556, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 255529, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 158793, 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 46590, 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 16951, 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 285193 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-06-06T08:13:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 231, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 196, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 101, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 342, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 1631, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 61, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 45, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 54204, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 54143, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-06-06T08:13:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 19, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 7055, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 4, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 26, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 91, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 282162, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 279496, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 279496, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 2666, 2024-06-06T08:13:31.076 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 3301, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 479, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 475, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 475, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 4, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 1, 2024-06-06T08:13:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 5, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 1171478677, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 1182182193, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 80293, 2024-06-06T08:13:31.078 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 960.453987554, 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.011961864 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 6865, 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 61427, 2024-06-06T08:13:31.079 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 54562, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 314, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 707, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 393, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 9112, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 143563, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 134451, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 560, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 61804, 2024-06-06T08:13:31.080 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 61244, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 129620, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 231804 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 634727, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 11357, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 422560, 2024-06-06T08:13:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 7403, 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 21963, 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 54909, 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1302.406863192, 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.023719369 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-06-06T08:13:31.082 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10903, 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 209.009328908, 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.019169891 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.083 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-06-06T08:13:31.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 124639, 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 870.240217061, 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006982086 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.086 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.087 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.088 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:31.088 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.088 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.088 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 25, 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.018308373, 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000732334 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-06-06T08:13:31.089 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 119, 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.931395933, 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007826856 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 25, 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5.930858596, 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.237234343 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10, 2024-06-06T08:13:31.091 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.032347871, 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003234787 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 156773, 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1760.650366250, 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.011230571 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.092 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10974, 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 323.981150736, 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.029522612 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.093 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-06-06T08:13:31.094 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.004158535, 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004158535 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 18, 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 14.457726075, 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.803207004 2024-06-06T08:13:31.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 451, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 221.617931149, 2024-06-06T08:13:31.096 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.491392308 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.097 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1636, 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 8.025803134, 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004905747 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 9, 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.023757231, 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002639692 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-06-06T08:13:31.099 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 54204, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 687.839983409, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.012689838 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.100 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.101 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 7, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 5, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 34745, 2024-06-06T08:13:31.102 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 17372, 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 1501, 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:31.103 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:31.104 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:31.105 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:31.106 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:31.107 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 2380498, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 2836, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 352, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 4, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:31.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 187, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 23618344, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 189320, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 258251, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 1257545684, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 37, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 258251, 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-06-06T08:13:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 258251, 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 14764.454879889, 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.057170949 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 258251, 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 363133 2024-06-06T08:13:31.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 258251, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 1743, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 256508, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 8881, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 53086, 2024-06-06T08:13:31.113 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1620, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 81909, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 1225, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 47, 2024-06-06T08:13:31.114 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 109592, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 123, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 80903, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:31.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-06-06T08:13:31.116 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-06-06T08:13:31.117 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 144, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-06-06T08:13:31.118 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 56, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 19806, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 10, 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 5931 2024-06-06T08:13:31.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 7151 2024-06-06T08:13:31.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 16, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 8, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 108286, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 54143, 2024-06-06T08:13:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1393623, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 246725597, 2024-06-06T08:13:31.122 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1393623, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1393623, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 246725597, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:31.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 258251, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1263461115, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 258251, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1263461115, 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:31.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-06-06T08:13:31.126 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 258251, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 258251, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 258251, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 258251, 2024-06-06T08:13:31.127 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:31.128 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 54143, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 5915499, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 54143, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1616, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 5915499, 2024-06-06T08:13:31.129 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55f99611e500": { 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 189, 2024-06-06T08:13:31.130 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 282162, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1177987889, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 282162, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 80293, 2024-06-06T08:13:31.131 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1177987700, 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:31.132 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-06-06T08:13:31.953 INFO:tasks.check_counter:Daemon mds.b mds.exported=24 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=231 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=196 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=25 2024-06-06T08:13:31.954 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=18 2024-06-06T08:13:31.955 DEBUG:tasks.check_counter:Getting stats from e 2024-06-06T08:13:31.955 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-06T08:13:32.201 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:13:32.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:32 smithi181 ceph-mon[36790]: pgmap v812: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T08:13:32.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:32 smithi195 ceph-mon[37040]: pgmap v812: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T08:13:32.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:32 smithi087 ceph-mon[32276]: pgmap v812: 129 pgs: 129 active+clean; 712 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T08:13:34.284 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:34 smithi181 ceph-mon[36790]: pgmap v813: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-06-06T08:13:34.484 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:34 smithi087 ceph-mon[32276]: pgmap v813: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-06-06T08:13:34.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:34 smithi195 ceph-mon[37040]: pgmap v813: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2024-06-06T08:13:34.709 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4541, 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4543, 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 117783902, 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1444045, 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 8, 2024-06-06T08:13:34.710 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.389399948, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.102334443, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.148563840, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.113362335, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4541, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.169901418, 2024-06-06T08:13:34.711 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000037414 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:34.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1182, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1141, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 90072486, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 499916, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-06-06T08:13:34.713 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.175555147, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.040030072, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.112957476, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.026587228, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1140, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.062824360, 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000055109 2024-06-06T08:13:34.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1836432, 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 306192 2024-06-06T08:13:34.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 623, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 985, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 100823862, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1073968, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 10, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-06-06T08:13:34.716 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.121263484, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.024290057, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.074785734, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.015294903, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 983, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.054262046, 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000055200 2024-06-06T08:13:34.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-06-06T08:13:34.718 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 878080 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:34.719 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 9577, 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.213086003, 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000022249 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:34.720 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-06-06T08:13:34.721 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-06-06T08:13:34.722 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 10795, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 2, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 10793, 2024-06-06T08:13:34.723 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 10793, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 5, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-06-06T08:13:34.724 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-06-06T08:13:34.725 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-06-06T08:13:34.726 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:13:34.727 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:34.728 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-06-06T08:13:34.729 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-06-06T08:13:34.730 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:13:34.731 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.732 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 52429, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-06-06T08:13:34.733 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 88, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 6, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-06-06T08:13:34.734 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 309407501, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 309396422, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.735 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 52429 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 803, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 10725, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 9922, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 40, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 41, 2024-06-06T08:13:34.736 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 10795, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 10795, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 67152, 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 215420 2024-06-06T08:13:34.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-06-06T08:13:34.738 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.739 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-06-06T08:13:34.740 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.741 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-06-06T08:13:34.742 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.744 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-06-06T08:13:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-06-06T08:13:34.747 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.748 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.750 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.752 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.753 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-06-06T08:13:34.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.758 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 4, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 3, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 3, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-06-06T08:13:34.759 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 5744, 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:34.760 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:34.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:34.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:34.764 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 275409, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 1012, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-06-06T08:13:34.765 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 187, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:34.766 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 9755673, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 70461, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.767 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 5766, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 5766, 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-06-06T08:13:34.768 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5766, 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 10.291814057, 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001784913 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5766, 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5767 2024-06-06T08:13:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 5766, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 5766, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 3504, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-06-06T08:13:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 2260, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-06-06T08:13:34.771 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:34.772 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-06-06T08:13:34.773 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-06-06T08:13:34.774 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 144, 2024-06-06T08:13:34.775 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 47, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 11, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 13, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 2, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-06-06T08:13:34.776 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.777 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-06-06T08:13:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:34.779 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 6342, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 308077228, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 6342, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 6342, 2024-06-06T08:13:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 308077228, 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:34.781 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 5766, 2024-06-06T08:13:34.782 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 305231764, 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5766, 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 305231764, 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:34.784 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 5766, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 5766, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5766, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 5766, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:34.786 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:34.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5575f2e846e0": { 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:34.788 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-06-06T08:13:34.789 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:34.790 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-06-06T08:13:35.531 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:13:35.532 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:13:35.532 DEBUG:tasks.check_counter:Getting stats from h 2024-06-06T08:13:35.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-06T08:13:35.781 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:13:36.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:36 smithi195 ceph-mon[37040]: pgmap v814: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-06T08:13:36.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:36 smithi181 ceph-mon[36790]: pgmap v814: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-06T08:13:36.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:36 smithi087 ceph-mon[32276]: pgmap v814: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-06-06T08:13:37.566 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 469228, 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 264080, 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 220314259, 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 69785666, 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 13, 2024-06-06T08:13:37.567 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 20.566430491, 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 8.962884629, 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 15.682912880, 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.242683352, 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:37.568 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 264082, 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 6.309512149, 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000023892 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 711533, 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.103837444, 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000145 2024-06-06T08:13:37.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1852176, 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 332816 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 58252, 2024-06-06T08:13:37.570 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 42126, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 16596209, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 40650100, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 15, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 2.845225587, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 1.017966680, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 1.923162762, 2024-06-06T08:13:37.571 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.276072292, 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 42123, 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.837360314, 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000043618 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.572 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 81040, 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.013441766, 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000165 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 794224 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.573 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 112047, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 148282, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 39250379, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 120021399, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 16, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 7.602246727, 2024-06-06T08:13:37.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 3.289217195, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 4.963324260, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.306922992, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 148278, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 6.170978197, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000041617 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 225578, 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.037841181, 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000167 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 36597136, 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 91871088 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.576 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 20717, 2024-06-06T08:13:37.577 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 14.796743524, 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000714231 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-06-06T08:13:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10860, 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.818205894, 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000075341 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-06-06T08:13:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 78616, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 38561, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 38561, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 297.381823187, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007711984 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 40055, 2024-06-06T08:13:37.580 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 28, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 2, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 926, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 203, 2024-06-06T08:13:37.581 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 711, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 12, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 628, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 256, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 69, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 69, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 24, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 126481, 2024-06-06T08:13:37.582 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 46531, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 40048, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 170, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 1679, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 9, 2024-06-06T08:13:37.583 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 135, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 14, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 192, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 11, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 716, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 212644217, 2024-06-06T08:13:37.584 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 7, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 11, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:13:37.585 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 11, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 9289, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 31851, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 19133, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 4456, 2024-06-06T08:13:37.586 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 2129, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 38252 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 1136, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 445, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 280, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 380, 2024-06-06T08:13:37.587 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 1320, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 1, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 5460, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 5460, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-06-06T08:13:37.588 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 19, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 1886, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:13:37.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 10, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 2, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-06-06T08:13:37.590 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 43392, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 43392, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 53, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-06-06T08:13:37.591 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 53, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 5, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 77624926, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-06-06T08:13:37.592 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10207, 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 51.193325293, 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005015511 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-06-06T08:13:37.593 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 881, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 6543, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 5662, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 310, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 553, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 243, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 938, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 7945, 2024-06-06T08:13:37.594 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 7007, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 55, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 6358, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 6303, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 74820, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 223612 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 113513, 2024-06-06T08:13:37.595 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 896, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 78616, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 1092, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 13715, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5467, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 152.013494346, 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.027805651 2024-06-06T08:13:37.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 59, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.167617593, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002840976 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.597 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-06-06T08:13:37.598 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 8034, 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 32.840467736, 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004087685 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.599 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-06-06T08:13:37.600 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.602 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:37.602 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.602 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.602 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-06-06T08:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.605 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17204, 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 12.656315628, 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000735661 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.606 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1703, 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.990607748, 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.009389669 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 22, 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 38.542921218, 2024-06-06T08:13:37.610 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 1.751950964 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:37.611 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 612, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.483601868, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000790199 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.612 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.613 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5460, 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 44.686797050, 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.008184395 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:37.614 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-06-06T08:13:37.615 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 4, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 3528, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 1764, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 1504, 2024-06-06T08:13:37.616 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:37.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:37.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:37.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:37.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 475580, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 630, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:37.621 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 187, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:37.622 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 3245930, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 24086, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-06-06T08:13:37.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 23908, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 74953341, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 23908, 2024-06-06T08:13:37.624 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 23908, 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 239.492656515, 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.010017260 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 23908, 2024-06-06T08:13:37.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 25109 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 23908, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 710, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 23198, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-06-06T08:13:37.626 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 699, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 10906, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 475, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 1, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 2, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 10947, 2024-06-06T08:13:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 11, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-06-06T08:13:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-06-06T08:13:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-06-06T08:13:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 144, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 47, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-06-06T08:13:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 1575, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 463 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-06-06T08:13:37.633 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 1914 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 16, 2024-06-06T08:13:37.634 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 8, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 10919, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 5460, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:37.635 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 639523, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 213946543, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 639523, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 639523, 2024-06-06T08:13:37.636 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 213946543, 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:37.637 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 23908, 2024-06-06T08:13:37.638 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 75550553, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 23908, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 75550553, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.639 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:37.640 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 23908, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 23908, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 23908, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 23908, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 5460, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 597212, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 5460, 2024-06-06T08:13:37.642 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 699, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 597212, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55d97805c500": { 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 45211, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 43392, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 73430622, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 43392, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:37.644 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 10207, 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 73385411, 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:37.645 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-06-06T08:13:38.305 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:38 smithi181 ceph-mon[36790]: pgmap v815: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-06-06T08:13:38.415 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds.imported=14 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1136 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=445 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:13:38.416 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:13:38.416 DEBUG:tasks.check_counter:Getting stats from k 2024-06-06T08:13:38.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-06T08:13:38.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:38 smithi195 ceph-mon[37040]: pgmap v815: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-06-06T08:13:38.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:38 smithi087 ceph-mon[32276]: pgmap v815: 129 pgs: 129 active+clean; 716 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-06-06T08:13:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:13:40.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:40 smithi181 ceph-mon[36790]: pgmap v816: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-06-06T08:13:40.496 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-06-06T08:13:40.496 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:40.496 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 496, 2024-06-06T08:13:40.496 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 485, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1659760, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 300288, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.078075094, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.025880177, 2024-06-06T08:13:40.497 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.046357051, 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.011612121, 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 484, 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.025314465, 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000052302 2024-06-06T08:13:40.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1659760, 2024-06-06T08:13:40.499 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 300288 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 7, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 373, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3552, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 299888, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.020207414, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.014241392, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.002015335, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 371, 2024-06-06T08:13:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.027111143, 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000073075 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 299888 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 0, 2024-06-06T08:13:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 0, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.000339180, 2024-06-06T08:13:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000255314, 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:40.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:40.506 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-06-06T08:13:40.507 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:40.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:40.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:40.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:40.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 149735, 2024-06-06T08:13:40.512 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 30, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:40.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:40.514 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-06-06T08:13:40.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 499, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1589694, 2024-06-06T08:13:40.516 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 499, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 499, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1589694, 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-06-06T08:13:40.517 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-06-06T08:13:40.518 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-06-06T08:13:40.518 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-06-06T08:13:40.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:40.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-06-06T08:13:40.518 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-06-06T08:13:40.519 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:40 smithi195 ceph-mon[37040]: pgmap v816: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-06-06T08:13:40.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:40 smithi087 ceph-mon[32276]: pgmap v816: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-06-06T08:13:41.253 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-06-06T08:13:41.254 DEBUG:tasks.check_counter:Getting stats from c 2024-06-06T08:13:41.254 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-06T08:13:41.616 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:13:42.234 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:42 smithi195 ceph-mon[37040]: pgmap v817: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-06-06T08:13:42.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:42 smithi181 ceph-mon[36790]: pgmap v817: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-06-06T08:13:42.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:42 smithi087 ceph-mon[32276]: pgmap v817: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-06-06T08:13:43.842 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-06-06T08:13:43.842 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:43.842 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 954, 2024-06-06T08:13:43.842 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 949, 2024-06-06T08:13:43.842 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 222971001, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 340371, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.180020054, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.022930017, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.130111449, 2024-06-06T08:13:43.843 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.021962316, 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 948, 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.036810677, 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000038829 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:43.844 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 223968, 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 91376 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.845 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1367, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1474, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 214324744, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 710298, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 6, 2024-06-06T08:13:43.846 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.214878886, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.032715976, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.135470831, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.037220039, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1472, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.070282794, 2024-06-06T08:13:43.847 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000047746 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T08:13:43.848 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 275840 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 420, 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 420, 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 121570562, 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 132742, 2024-06-06T08:13:43.849 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.086343492, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.008037372, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.066355662, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.009384058, 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:43.850 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 419, 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.023277631, 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000055555 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.851 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-06-06T08:13:43.852 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3951, 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.397189875, 2024-06-06T08:13:43.853 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000100528 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.854 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-06-06T08:13:43.855 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-06-06T08:13:43.856 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 27063, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 17734, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 9329, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-06-06T08:13:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 1729, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 22, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-06-06T08:13:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-06-06T08:13:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 716, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 221136869, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 7, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:13:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-06-06T08:13:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-06-06T08:13:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-06-06T08:13:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 117717, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-06-06T08:13:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 5, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 35, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 1171478677, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 1183033522, 2024-06-06T08:13:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 1183017937, 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 117717 2024-06-06T08:13:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 4802, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 26470, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 21668, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 279, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 398, 2024-06-06T08:13:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 119, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 27063, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 27063, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 0, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 89960, 2024-06-06T08:13:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215396 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-06-06T08:13:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-06-06T08:13:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-06-06T08:13:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-06-06T08:13:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 2, 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-06-06T08:13:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 0, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 2957, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 442547, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 5606, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 187, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 31281457, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 236104, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-06-06T08:13:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2565, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 26, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 4, 2024-06-06T08:13:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2565, 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2565, 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 9.821828916, 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003829173 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-06-06T08:13:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2565, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2566 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2565, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2565, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-06-06T08:13:43.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1517, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 1045, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-06-06T08:13:43.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:43.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-06-06T08:13:43.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-06-06T08:13:43.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-06-06T08:13:43.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-06-06T08:13:43.914 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 144, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 17, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 11, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 11, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-06-06T08:13:43.915 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-06-06T08:13:43.916 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-06-06T08:13:43.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-06-06T08:13:43.918 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:43.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2737, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 558608234, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2737, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:43.920 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2737, 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 558608234, 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:43.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2565, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 557934085, 2024-06-06T08:13:43.923 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2565, 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 557934085, 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.924 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:43.925 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2565, 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2565, 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2565, 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2565, 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.926 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:43.927 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-06-06T08:13:43.928 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x559c732da6e0": { 2024-06-06T08:13:43.929 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:43.930 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:43.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.932 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:43.932 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-06-06T08:13:44.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:44 smithi195 ceph-mon[37040]: pgmap v818: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2024-06-06T08:13:44.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:44 smithi181 ceph-mon[36790]: pgmap v818: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2024-06-06T08:13:44.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:44 smithi087 ceph-mon[32276]: pgmap v818: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-06-06T08:13:44.640 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:13:44.641 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:13:44.641 DEBUG:tasks.check_counter:Getting stats from f 2024-06-06T08:13:44.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-06T08:13:44.884 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:13:46.383 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:46 smithi195 ceph-mon[37040]: pgmap v819: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 679 KiB/s wr, 31 op/s 2024-06-06T08:13:46.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:46 smithi181 ceph-mon[36790]: pgmap v819: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 679 KiB/s wr, 31 op/s 2024-06-06T08:13:46.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:46 smithi087 ceph-mon[32276]: pgmap v819: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 679 KiB/s wr, 31 op/s 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1197, 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1199, 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 17090957, 2024-06-06T08:13:46.647 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 380710, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 8, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.106747387, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.030187357, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.035843797, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.031331357, 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:46.648 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1197, 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.042374937, 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000035400 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1905, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2270, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 31984015, 2024-06-06T08:13:46.650 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1399663, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 8, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 6, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.180704108, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.056081898, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.057608715, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.051971766, 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:46.651 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2268, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.094403339, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000041624 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.652 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 795776 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3320, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 3279, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 27850411, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1180382, 2024-06-06T08:13:46.653 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.334999088, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.099230394, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.125336867, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.096692891, 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:46.654 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3278, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.142793850, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000043561 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.655 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1836976, 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 307472 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-06-06T08:13:46.656 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9939, 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.240301920, 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000024177 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:46.657 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-06-06T08:13:46.658 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-06-06T08:13:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 6266, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 71, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 6195, 2024-06-06T08:13:46.660 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 6165, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 21, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-06-06T08:13:46.661 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-06-06T08:13:46.662 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-06-06T08:13:46.663 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:13:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-06-06T08:13:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-06-06T08:13:46.666 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:46.667 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:13:46.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-06-06T08:13:46.669 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 43394, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-06-06T08:13:46.670 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 53, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 5, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 77638940, 2024-06-06T08:13:46.671 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 77638366, 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 43394 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-06-06T08:13:46.672 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 773, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 6415, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 5642, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 50, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 71, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 21, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 6266, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 6407, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 141, 2024-06-06T08:13:46.673 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 62156, 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215420 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-06-06T08:13:46.674 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-06-06T08:13:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.680 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.682 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-06-06T08:13:46.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.684 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-06-06T08:13:46.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.687 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:46.688 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.689 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.690 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-06-06T08:13:46.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:46.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 4, 2024-06-06T08:13:46.694 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 2, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 3, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 3664, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.695 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:46.696 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:46.697 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:46.698 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:46.699 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:46.699 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:46.699 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:46.699 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:46.699 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 253736, 2024-06-06T08:13:46.700 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 837, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:46.701 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 187, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 6375030, 2024-06-06T08:13:46.702 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 43257, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-06-06T08:13:46.703 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 5841, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 21, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 3, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 5841, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5841, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 10.580813752, 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001811472 2024-06-06T08:13:46.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5841, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 5842 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 5841, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 5841, 2024-06-06T08:13:46.705 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 3316, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 2523, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-06-06T08:13:46.706 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:46.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 1, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:46.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-06-06T08:13:46.709 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-06-06T08:13:46.710 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 144, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 47, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 9, 2024-06-06T08:13:46.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 10, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 1, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-06-06T08:13:46.712 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-06-06T08:13:46.713 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.714 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 6418, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 76315460, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 6418, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:46.715 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 6418, 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 76315460, 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:46.717 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 5841, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 73463743, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 5841, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 73463743, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.718 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:46.719 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 5841, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 5841, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 5841, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 5841, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.720 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:46.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-06-06T08:13:46.722 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55d9dc5cc6e0": { 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:46.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-06-06T08:13:46.724 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:46.725 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-06-06T08:13:47.581 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:13:47.582 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:13:47.582 DEBUG:tasks.check_counter:Getting stats from i 2024-06-06T08:13:47.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-06T08:13:47.824 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:13:48.197 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:48 smithi195 ceph-mon[37040]: pgmap v820: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 439 KiB/s wr, 18 op/s 2024-06-06T08:13:48.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:48 smithi181 ceph-mon[36790]: pgmap v820: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 439 KiB/s wr, 18 op/s 2024-06-06T08:13:48.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:48 smithi087 ceph-mon[32276]: pgmap v820: 129 pgs: 129 active+clean; 717 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 439 KiB/s wr, 18 op/s 2024-06-06T08:13:49.559 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-06-06T08:13:49.559 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:49.559 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 616203, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 466117, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 267203411, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 273803233, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 10, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 26.316131711, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 10.679011457, 2024-06-06T08:13:49.560 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 21.124952398, 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.403597594, 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 466115, 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 15.867278937, 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000034041 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.561 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1029891, 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.136796292, 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000132 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 52548160, 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 141986256 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.562 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 43399, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 60258, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 9542439, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 94020529, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 28, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 2.995648177, 2024-06-06T08:13:49.563 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 1.246283390, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 1.463941109, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.320605203, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 60256, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.748525305, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000045614 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.564 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 81030, 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.012511890, 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000154 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 8272, 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 802448 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.565 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 21271, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 21225, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 8136196, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 159410581, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 12, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 7, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 1.766449794, 2024-06-06T08:13:49.566 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.635319414, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.384863133, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.652706516, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 21224, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.290804528, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000060818 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.567 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 10, 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000001670, 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000167 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1854512, 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 313248 2024-06-06T08:13:49.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:49.569 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 38865, 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 30.721506570, 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000790467 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-06-06T08:13:49.570 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 18912, 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.684418520, 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000089066 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 116491, 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 80151, 2024-06-06T08:13:49.571 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 80151, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 433.967107011, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.005414369 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 36340, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-06-06T08:13:49.572 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 20, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 1, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 996, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 156, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 840, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-06-06T08:13:49.573 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 773, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 1, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 52, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 52, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 6, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 172986, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 86941, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 36338, 2024-06-06T08:13:49.574 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 152, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 1880, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-06-06T08:13:49.575 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 10, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 88, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 716, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 218475220, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 7, 2024-06-06T08:13:49.576 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:13:49.577 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 9, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 4501, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 32624, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:13:49.578 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 22037, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 9159, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 4009, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 63757 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 50, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 2416, 2024-06-06T08:13:49.579 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 1259, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 2403, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 174, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 9972, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 9972, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-06-06T08:13:49.580 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 19, 2024-06-06T08:13:49.581 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 1912, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 1, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.582 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 52517, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 52517, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-06-06T08:13:49.583 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 88, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 88, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 6, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-06-06T08:13:49.584 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 309815966, 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 19615, 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 99.601146648, 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.005077805 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-06-06T08:13:49.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 870, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 10842, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 9972, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 282, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 1422, 2024-06-06T08:13:49.586 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 1140, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 996, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 17699, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 16703, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 52, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 11558, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 11506, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 81424, 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223612 2024-06-06T08:13:49.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 152646, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 3114, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 116491, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 1184, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 7141, 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-06-06T08:13:49.588 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9976, 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 164.055410511, 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.016445009 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 10720, 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 77.513763350, 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.007230761 2024-06-06T08:13:49.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.590 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 18167, 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 18.663826440, 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001027347 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.591 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2, 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000232295, 2024-06-06T08:13:49.592 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000116147 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-06-06T08:13:49.593 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.594 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.595 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.595 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-06-06T08:13:49.595 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.595 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.597 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.599 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-06-06T08:13:49.599 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 29272, 2024-06-06T08:13:49.602 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 28.381736461, 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000969586 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1980, 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 54.391663263, 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.027470537 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.603 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.604 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.605 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.606 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 62, 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.479706110, 2024-06-06T08:13:49.607 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.007737195 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-06-06T08:13:49.608 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-06-06T08:13:49.610 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9972, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 90.480768581, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.009073482 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 4, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 3, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-06-06T08:13:49.613 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 11398, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 11398, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1506, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:49.614 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:49.615 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:49.616 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:49.617 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:49.618 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 339234, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 468, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 88, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 1, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-06-06T08:13:49.619 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9784, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 187, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:49.620 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 3302754, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 23893, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-06-06T08:13:49.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 41452, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 307936528, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 33, 2024-06-06T08:13:49.622 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 41452, 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 41452, 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 437.238348831, 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.010548063 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-06-06T08:13:49.623 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 41452, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 42267 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 41452, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 486, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 40966, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-06-06T08:13:49.624 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 486, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 20101, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 293, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-06-06T08:13:49.625 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 1, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 20028, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 0, 2024-06-06T08:13:49.626 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:13:49.627 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-06-06T08:13:49.628 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-06-06T08:13:49.629 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-06-06T08:13:49.629 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-06-06T08:13:49.629 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-06-06T08:13:49.629 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-06-06T08:13:49.630 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 144, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 47, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 18, 2024-06-06T08:13:49.631 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 6, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 993, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 0, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 335 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:13:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 1937 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-06-06T08:13:49.633 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 16, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 8, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 19944, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 9972, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.634 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 680869, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 216860768, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 680869, 2024-06-06T08:13:49.635 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 680869, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 216860768, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:49.637 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 41452, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 309024764, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 41452, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 309024764, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.638 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:49.639 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 41452, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 41452, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 41452, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 41452, 2024-06-06T08:13:49.640 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:49.641 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 9972, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1088236, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 9972, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:49.642 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 486, 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1088236, 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:49.643 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x557096190500": { 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 52517, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 305621662, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 52517, 2024-06-06T08:13:49.644 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 19615, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 305621662, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:49.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.646 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:49.646 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-06-06T08:13:50.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:50 smithi195 ceph-mon[37040]: pgmap v821: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 456 KiB/s wr, 23 op/s 2024-06-06T08:13:50.381 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=50 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2416 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:13:50.382 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:13:50.383 DEBUG:tasks.check_counter:Getting stats from l 2024-06-06T08:13:50.383 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-06T08:13:50.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:50 smithi181 ceph-mon[36790]: pgmap v821: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 456 KiB/s wr, 23 op/s 2024-06-06T08:13:50.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:50 smithi087 ceph-mon[32276]: pgmap v821: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 456 KiB/s wr, 23 op/s 2024-06-06T08:13:50.619 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:13:52.259 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:52 smithi195 ceph-mon[37040]: pgmap v822: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:52.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:52 smithi181 ceph-mon[36790]: pgmap v822: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:52.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:52 smithi087 ceph-mon[32276]: pgmap v822: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:52.669 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-06-06T08:13:52.669 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:13:52.669 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 499, 2024-06-06T08:13:52.669 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 488, 2024-06-06T08:13:52.669 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1660576, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 302208, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 0, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.079994498, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.027757587, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.045483344, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.011301451, 2024-06-06T08:13:52.670 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 487, 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.024737909, 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000050796 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:52.671 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1660576, 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 302208 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3, 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 374, 2024-06-06T08:13:52.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 480, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 299968, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.020186387, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.014469076, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.001963099, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:52.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 371, 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.032816851, 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000088455 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:52.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 299968 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:13:52.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 4, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 4, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3072, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 448, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 2, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 1, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.000614928, 2024-06-06T08:13:52.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.000030976, 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.000540315, 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000724329, 2024-06-06T08:13:52.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000241443 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.678 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 0, 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.679 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:13:52.680 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:13:52.681 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:13:52.682 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:13:52.682 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:13:52.682 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-06-06T08:13:52.682 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:13:52.682 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-06-06T08:13:52.683 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:13:52.683 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:13:52.683 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:13:52.683 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:13:52.683 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:13:52.684 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:13:52.685 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:13:52.686 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 148714, 2024-06-06T08:13:52.686 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 30, 2024-06-06T08:13:52.686 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:13:52.686 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-06-06T08:13:52.686 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-06-06T08:13:52.687 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 928, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 2, 2024-06-06T08:13:52.688 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-06-06T08:13:52.689 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 502, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1590084, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 502, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-06-06T08:13:52.690 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 502, 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1590084, 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:52.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-06-06T08:13:52.692 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-06-06T08:13:53.375 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-06-06T08:13:53.376 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-06-06T08:13:53.377 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-06T08:13:53.392 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-06T08:13:53.392 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T08:13:53.392 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:13:53.419 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T08:13:53.419 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:13:53.476 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T08:13:53.760 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:54.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:54 smithi195 ceph-mon[37040]: pgmap v823: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-06-06T08:13:54.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:54 smithi181 ceph-mon[36790]: pgmap v823: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-06-06T08:13:54.606 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:54 smithi087 ceph-mon[32276]: pgmap v823: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-06-06T08:13:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- 172.21.15.87:0/880139935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 msgr2=0x7f5a4c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 --2- 172.21.15.87:0/880139935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c101a00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099c0 tx=0x7f5a3c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- 172.21.15.87:0/880139935 shutdown_connections 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 --2- 172.21.15.87:0/880139935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 --2- 172.21.15.87:0/880139935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c101a00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 --2- 172.21.15.87:0/880139935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c101fd0 0x7f5a4c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- 172.21.15.87:0/880139935 >> 172.21.15.87:0/880139935 conn(0x7f5a4c078340 msgr2=0x7f5a4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- 172.21.15.87:0/880139935 shutdown_connections 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- 172.21.15.87:0/880139935 wait complete. 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 Processor -- start 2024-06-06T08:13:55.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.583+0000 7f5a524bd700 1 -- start start 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c101fd0 0x7f5a4c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c111710 con 0x7f5a4c101620 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c111590 con 0x7f5a4c1066b0 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a524bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c111890 con 0x7f5a4c101fd0 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:55.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39546/0 (socket says 172.21.15.87:39546) 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 -- 172.21.15.87:0/1398398321 learned_addr learned my addr 172.21.15.87:0/1398398321 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4b7fe700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c101fd0 0x7f5a4c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 -- 172.21.15.87:0/1398398321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c101fd0 msgr2=0x7f5a4c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c101fd0 0x7f5a4c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 -- 172.21.15.87:0/1398398321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 msgr2=0x7f5a4c1a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a50a5a700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c1a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 -- 172.21.15.87:0/1398398321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1a5070 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a50a5a700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.584+0000 7f5a4bfff700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.585+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c027070 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.585+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c1a5240 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.585+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c0040d0 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.585+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00f460 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.585+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a56f0 con 0x7f5a4c101620 2024-06-06T08:13:55.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.586+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c069180 con 0x7f5a4c101620 2024-06-06T08:13:55.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.587+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5a3c02a070 con 0x7f5a4c101620 2024-06-06T08:13:55.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.588+0000 7f5a497fa700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 0x7f5a340a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:55.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.588+0000 7f5a4b7fe700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 0x7f5a340a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:55.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.588+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9567+0+0 (secure 0 0 0) 0x7f5a3c0d0eb0 con 0x7f5a4c101620 2024-06-06T08:13:55.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.588+0000 7f5a4b7fe700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 0x7f5a340a5250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a40006fd0 tx=0x7f5a40008040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:55.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.594+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a3c031020 con 0x7f5a4c101620 2024-06-06T08:13:55.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.810+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a4c02de60 con 0x7f5a4c101620 2024-06-06T08:13:55.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.810+0000 7f5a497fa700 1 -- 172.21.15.87:0/1398398321 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v144) ==== 81+0+945 (secure 0 0 0) 0x7f5a3c013070 con 0x7f5a4c101620 2024-06-06T08:13:55.810 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/2460003046 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/2460003046 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3344462507 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/207935511 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3853061787 2024-06-07T07:36:46.051849+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3813763622 2024-06-07T07:36:20.511068+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/4065374776 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.811 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/4176793720 2024-06-07T07:36:20.511068+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2153987725 2024-06-07T07:48:00.385891+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/1264886975 2024-06-07T07:36:46.051849+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2314391466 2024-06-07T07:36:20.511068+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3870018525 2024-06-07T07:36:20.511068+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/1605115594 2024-06-07T07:36:46.051849+0000 2024-06-06T08:13:55.812 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/2887993384 2024-06-07T07:36:46.051849+0000 2024-06-06T08:13:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 msgr2=0x7f5a340a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 0x7f5a340a5250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a40006fd0 tx=0x7f5a40008040 comp rx=0 tx=0).stop 2024-06-06T08:13:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 msgr2=0x7f5a4c199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:55.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c005640 comp rx=0 tx=0).stop 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 shutdown_connections 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4c1066b0 0x7f5a4c1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f5a340a2d90 0x7f5a340a5250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a4c101620 0x7f5a4c199ab0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 --2- 172.21.15.87:0/1398398321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c101fd0 0x7f5a4c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 >> 172.21.15.87:0/1398398321 conn(0x7f5a4c078340 msgr2=0x7f5a4c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:55.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 shutdown_connections 2024-06-06T08:13:55.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:55.813+0000 7f5a524bd700 1 -- 172.21.15.87:0/1398398321 wait complete. 2024-06-06T08:13:55.815 INFO:teuthology.orchestra.run.smithi087.stderr:listed 16 entries 2024-06-06T08:13:56.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:56 smithi195 ceph-mon[37040]: pgmap v824: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:56.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:56 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1398398321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:13:56.515 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:56 smithi087 ceph-mon[32276]: pgmap v824: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:56.515 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:56 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1398398321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:13:56.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-06T08:13:56.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:56.543 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-06T08:13:56.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:56 smithi181 ceph-mon[36790]: pgmap v824: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-06-06T08:13:56.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:56 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1398398321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:13:56.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T08:13:56.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:56.798 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-06T08:13:56.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:56.825 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-06-06T08:13:56.883 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-06T08:13:56.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:56.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T08:13:56.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:56.950 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-06-06T08:13:57.052 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-06T08:13:57.053 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-06-06T08:13:57.053 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:13:57.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056316,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=200 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/diff,workdir=/var/lib/containers/storage/overlay/4390f1e699e725995daba1fc2e97c249d7aee45805184e9719919aaa227d0e32/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/diff,workdir=/var/lib/containers/storage/overlay/73231cf7f31b4d4407b482fe5df645f3692160f196044182084bc48e2fc0dfb9/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/diff,workdir=/var/lib/containers/storage/overlay/c3af3dd83485225de98ce7e05b1778bca7669fd0aef3e90441896f4b074c78b6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/diff,workdir=/var/lib/containers/storage/overlay/472e5ea227be26315ff651e551e170be25884e46bff34c4e2264ed5a4c903b9e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/diff,workdir=/var/lib/containers/storage/overlay/24e96a50383952d379b7fbe1fbc8a8b61b25fd1be43aceb4e92b83a19d79b69e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/diff,workdir=/var/lib/containers/storage/overlay/c025dd78366daa90f2bb22622560f72d444e1e26749d4582a06af2f5df64ca15/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/diff,workdir=/var/lib/containers/storage/overlay/cb3b8a2dd0a1ef429f0567f46da658be6c72fcdee7bb512386315479e37e0480/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/diff,workdir=/var/lib/containers/storage/overlay/9953194a445df208b164f6a586dc9a818ae51f70ee7c1533c0ac490462a7bf64/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/diff,workdir=/var/lib/containers/storage/overlay/e58985e329d8d49ca3085c19ca9be9e7b9f861fc9fa8c274d2db20af2daa7aef/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7CT4PMO3NKRDVIPIT5IMBE7MPY:/var/lib/containers/storage/overlay/l/SZ5DH3LZLFLZDTX4M2B5XR6OPS,upperdir=/var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/diff,workdir=/var/lib/containers/storage/overlay/59b87618c5b30839ee33d0d2ee21eda497d0f8627977b00e7ffc82a216acc129/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 2024-06-06T08:13:57.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:57.070 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-06-06T08:13:57.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:13:57.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-06-06T08:13:57.193 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-06-06T08:13:57.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:13:57.195 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-06T08:13:57.209 INFO:tasks.cephadm:Teardown begin 2024-06-06T08:13:57.209 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-06T08:13:57.209 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:13:57.479 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:13:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:13:58 smithi195 ceph-mon[37040]: pgmap v825: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2024-06-06T08:13:58.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:13:58 smithi181 ceph-mon[36790]: pgmap v825: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2024-06-06T08:13:58.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:13:58 smithi087 ceph-mon[32276]: pgmap v825: 129 pgs: 129 active+clean; 718 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2024-06-06T08:13:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 -- 172.21.15.87:0/1536822570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 msgr2=0x7ff47410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/1536822570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47410f320 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00b0d0 tx=0x7ff46c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 -- 172.21.15.87:0/1536822570 shutdown_connections 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/1536822570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47410f320 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/1536822570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 0x7ff4741088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/1536822570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 0x7ff47406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 -- 172.21.15.87:0/1536822570 >> 172.21.15.87:0/1536822570 conn(0x7ff4740782c0 msgr2=0x7ff4740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 -- 172.21.15.87:0/1536822570 shutdown_connections 2024-06-06T08:13:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 -- 172.21.15.87:0/1536822570 wait complete. 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.628+0000 7ff47c1c5700 1 Processor -- start 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 -- start start 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 0x7ff47419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 0x7ff4741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff474112750 con 0x7ff47406ab10 2024-06-06T08:13:59.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741125d0 con 0x7ff47406a170 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47c1c5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741128d0 con 0x7ff4741084e0 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53796/0 (socket says 172.21.15.87:53796) 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 -- 172.21.15.87:0/2190565378 learned_addr learned my addr 172.21.15.87:0/2190565378 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479f61700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 0x7ff47419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff47a762700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 0x7ff4741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:59.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 -- 172.21.15.87:0/2190565378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 msgr2=0x7ff4741989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 0x7ff4741989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 -- 172.21.15.87:0/2190565378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 msgr2=0x7ff47419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 0x7ff47419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 -- 172.21.15.87:0/2190565378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff474199130 con 0x7ff47406ab10 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff479760700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e9a0 tx=0x7ff46400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.629+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400ca10 con 0x7ff47406ab10 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.630+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46400cbb0 con 0x7ff47406ab10 2024-06-06T08:13:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.630+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff474199420 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.630+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464018680 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.630+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4741a56e0 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.631+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff474069180 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.632+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff464010430 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.632+0000 7ff46affd700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 0x7ff4600a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.632+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9567+0+0 (secure 0 0 0) 0x7ff4640cfa80 con 0x7ff47406ab10 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.632+0000 7ff479f61700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 0x7ff4600a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:13:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.633+0000 7ff479f61700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 0x7ff4600a5250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff474199b10 tx=0x7ff470009260 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:13:59.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.637+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff46409c290 con 0x7ff47406ab10 2024-06-06T08:13:59.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.902+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff47402dca0 con 0x7ff47406ab10 2024-06-06T08:13:59.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.903+0000 7ff46affd700 1 -- 172.21.15.87:0/2190565378 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7ff46409bb90 con 0x7ff47406ab10 2024-06-06T08:13:59.903 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:13:59.904 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:13:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 msgr2=0x7ff4600a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 0x7ff4600a5250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff474199b10 tx=0x7ff470009260 comp rx=0 tx=0).stop 2024-06-06T08:13:59.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 msgr2=0x7ff47419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e9a0 tx=0x7ff46400ecb0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 shutdown_connections 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff47406a170 0x7ff47419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7ff4600a2d90 0x7ff4600a5250 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4741084e0 0x7ff4741989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 --2- 172.21.15.87:0/2190565378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff47406ab10 0x7ff47419ed50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:13:59.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.906+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 >> 172.21.15.87:0/2190565378 conn(0x7ff4740782c0 msgr2=0x7ff4740ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:13:59.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.907+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 shutdown_connections 2024-06-06T08:13:59.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:13:59.907+0000 7ff47c1c5700 1 -- 172.21.15.87:0/2190565378 wait complete. 2024-06-06T08:13:59.908 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:14:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:00 smithi195 ceph-mon[37040]: pgmap v826: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 146 KiB/s wr, 12 op/s 2024-06-06T08:14:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:00 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:00 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2190565378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:00 smithi181 ceph-mon[36790]: pgmap v826: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 146 KiB/s wr, 12 op/s 2024-06-06T08:14:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:00 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:00 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2190565378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.599 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:00 smithi087 ceph-mon[32276]: pgmap v826: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 146 KiB/s wr, 12 op/s 2024-06-06T08:14:00.600 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:00 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.600 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:00 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2190565378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:00.628 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:14:00.867 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:02.353 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:02 smithi195 ceph-mon[37040]: pgmap v827: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 129 KiB/s wr, 8 op/s 2024-06-06T08:14:02.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:02 smithi087 ceph-mon[32276]: pgmap v827: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 129 KiB/s wr, 8 op/s 2024-06-06T08:14:02.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:02 smithi181 ceph-mon[36790]: pgmap v827: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 129 KiB/s wr, 8 op/s 2024-06-06T08:14:02.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.612+0000 7fcaf2110700 1 -- 172.21.15.87:0/3206470703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec101d50 msgr2=0x7fcaec10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:02.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.612+0000 7fcaf2110700 1 --2- 172.21.15.87:0/3206470703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec101d50 0x7fcaec10e240 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b0a0 tx=0x7fcae400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.612+0000 7fcaf2110700 1 -- 172.21.15.87:0/3206470703 shutdown_connections 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.612+0000 7fcaf2110700 1 --2- 172.21.15.87:0/3206470703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec107400 0x7fcaec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 --2- 172.21.15.87:0/3206470703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec101d50 0x7fcaec10e240 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 --2- 172.21.15.87:0/3206470703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101390 0x7fcaec101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- 172.21.15.87:0/3206470703 >> 172.21.15.87:0/3206470703 conn(0x7fcaec078340 msgr2=0x7fcaec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- 172.21.15.87:0/3206470703 shutdown_connections 2024-06-06T08:14:02.613 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- 172.21.15.87:0/3206470703 wait complete. 2024-06-06T08:14:02.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 Processor -- start 2024-06-06T08:14:02.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- start start 2024-06-06T08:14:02.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:02.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 0x7fcaec1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1116a0 con 0x7fcaec107400 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec111520 con 0x7fcaec101390 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.613+0000 7fcaf2110700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec111820 con 0x7fcaec101d50 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaeb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcae3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 0x7fcaec1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:02.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaeb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.87:34070/0 (socket says 172.21.15.87:34070) 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53822/0 (socket says 172.21.15.87:53822) 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcae3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 0x7fcaec1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.87:55564/0 (socket says 172.21.15.87:55564) 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaeb7fe700 1 -- 172.21.15.87:0/1249805060 learned_addr learned my addr 172.21.15.87:0/1249805060 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 -- 172.21.15.87:0/1249805060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 msgr2=0x7fcaec1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 0x7fcaec1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 -- 172.21.15.87:0/1249805060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 msgr2=0x7fcaec103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 -- 172.21.15.87:0/1249805060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec1a70f0 con 0x7fcaec107400 2024-06-06T08:14:02.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaeb7fe700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaebfff700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b0a0 tx=0x7fcae4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4021070 con 0x7fcaec107400 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae4007b80 con 0x7fcaec107400 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec1a7380 con 0x7fcaec107400 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40052a0 con 0x7fcaec107400 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.614+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec1a7830 con 0x7fcaec107400 2024-06-06T08:14:02.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.615+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec069180 con 0x7fcaec107400 2024-06-06T08:14:02.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.616+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcae4024070 con 0x7fcaec107400 2024-06-06T08:14:02.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.617+0000 7fcae97fa700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 0x7fcad80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:02.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.617+0000 7fcaeb7fe700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 0x7fcad80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:02.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.617+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9567+0+0 (secure 0 0 0) 0x7fcae400f020 con 0x7fcaec107400 2024-06-06T08:14:02.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.617+0000 7fcaeb7fe700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 0x7fcad80a5250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcadc005950 tx=0x7fcadc00b500 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:02.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.621+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcae402b020 con 0x7fcaec107400 2024-06-06T08:14:02.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.865+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaec1a7b50 con 0x7fcaec107400 2024-06-06T08:14:02.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.866+0000 7fcae97fa700 1 -- 172.21.15.87:0/1249805060 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11877 (secure 0 0 0) 0x7fcae409da30 con 0x7fcaec107400 2024-06-06T08:14:02.866 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:14:02.866 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2593068869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2593068869},{"type":"v1","addr":"172.21.15.87:6835","nonce":2593068869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2477022406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2477022406},{"type":"v1","addr":"172.21.15.87:6837","nonce":2477022406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/200596410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":200596410},{"type":"v1","addr":"172.21.15.87:6839","nonce":200596410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/3223638025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":3223638025},{"type":"v1","addr":"172.21.15.87:6841","nonce":3223638025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2222795089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2222795089},{"type":"v1","addr":"172.21.15.195:6833","nonce":2222795089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2618702621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2618702621},{"type":"v1","addr":"172.21.15.181:6833","nonce":2618702621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T07:44:02.139989+0000","modified":"2024-06-06T08:06:52.843595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24467,"mds_1":24448,"mds_2":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/922595027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":922595027},{"type":"v1","addr":"172.21.15.195:6835","nonce":922595027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/1343084853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1343084853},{"type":"v1","addr":"172.21.15.181:6835","nonce":1343084853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":100,"addr":"172.21.15.181:6837/3077028235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3077028235},{"type":"v1","addr":"172.21.15.181:6837","nonce":3077028235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.195:6839/203076359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":203076359},{"type":"v1","addr":"172.21.15.195:6839","nonce":203076359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/851837058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":851837058},{"type":"v1","addr":"172.21.15.181:6839","nonce":851837058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26980":{"gid":26980,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/4097924488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4097924488},{"type":"v1","addr":"172.21.15.195:6837","nonce":4097924488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24467,"qdb_cluster":[24448,24467,24446]},"id":1}]} 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 msgr2=0x7fcad80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 0x7fcad80a5250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcadc005950 tx=0x7fcadc00b500 comp rx=0 tx=0).stop 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 msgr2=0x7fcaec103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b0a0 tx=0x7fcae4004d00 comp rx=0 tx=0).stop 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 shutdown_connections 2024-06-06T08:14:02.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaec101390 0x7fcaec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7fcad80a2d90 0x7fcad80a5250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcaec107400 0x7fcaec103af0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 --2- 172.21.15.87:0/1249805060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaec101d50 0x7fcaec1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 >> 172.21.15.87:0/1249805060 conn(0x7fcaec078340 msgr2=0x7fcaec1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 shutdown_connections 2024-06-06T08:14:02.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:02.868+0000 7fcaf2110700 1 -- 172.21.15.87:0/1249805060 wait complete. 2024-06-06T08:14:02.870 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 55 2024-06-06T08:14:03.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:03 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/1249805060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:03.548 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:03 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/1249805060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:03.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:03 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/1249805060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:14:03.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:14:03.820 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:04.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: pgmap v828: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 152 KiB/s wr, 13 op/s 2024-06-06T08:14:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:14:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:14:04.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:04 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: pgmap v828: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 152 KiB/s wr, 13 op/s 2024-06-06T08:14:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:14:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:14:04.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:04 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: pgmap v828: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 152 KiB/s wr, 13 op/s 2024-06-06T08:14:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:14:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:14:04.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:04 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- 172.21.15.87:0/141218150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c105f30 msgr2=0x7f013c10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 --2- 172.21.15.87:0/141218150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c105f30 0x7f013c10da70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f013400c210 tx=0x7f013400c520 comp rx=0 tx=0).stop 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- 172.21.15.87:0/141218150 shutdown_connections 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 --2- 172.21.15.87:0/141218150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 --2- 172.21.15.87:0/141218150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c105f30 0x7f013c10da70 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 --2- 172.21.15.87:0/141218150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c101a40 0x7f013c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- 172.21.15.87:0/141218150 >> 172.21.15.87:0/141218150 conn(0x7f013c0766b0 msgr2=0x7f013c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:05.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- 172.21.15.87:0/141218150 shutdown_connections 2024-06-06T08:14:05.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- 172.21.15.87:0/141218150 wait complete. 2024-06-06T08:14:05.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 Processor -- start 2024-06-06T08:14:05.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.686+0000 7f014341c700 1 -- start start 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 0x7f013c1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f013c110640 con 0x7f013c101a40 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1104c0 con 0x7f013c101090 2024-06-06T08:14:05.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1107c0 con 0x7f013c105f30 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53850/0 (socket says 172.21.15.87:53850) 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 -- 172.21.15.87:0/4192983141 learned_addr learned my addr 172.21.15.87:0/4192983141 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01419b9700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 0x7f013c1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01411b8700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 -- 172.21.15.87:0/4192983141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 msgr2=0x7f013c1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 0x7f013c1967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 -- 172.21.15.87:0/4192983141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 msgr2=0x7f013c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 -- 172.21.15.87:0/4192983141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c196fc0 con 0x7f013c101a40 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01411b8700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01419b9700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 0x7f013c1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01409b7700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d980 tx=0x7f012c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c017070 con 0x7f013c101a40 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.687+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1972b0 con 0x7f013c101a40 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.688+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c00be20 con 0x7f013c101a40 2024-06-06T08:14:05.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.688+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a3470 con 0x7f013c101a40 2024-06-06T08:14:05.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.688+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c005410 con 0x7f013c101a40 2024-06-06T08:14:05.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.688+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c102270 con 0x7f013c101a40 2024-06-06T08:14:05.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.690+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f012c009b40 con 0x7f013c101a40 2024-06-06T08:14:05.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.690+0000 7f01327fc700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 0x7f01280a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:05.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.690+0000 7f01411b8700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 0x7f01280a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:05.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.690+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9567+0+0 (secure 0 0 0) 0x7f012c0cf5e0 con 0x7f013c101a40 2024-06-06T08:14:05.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.691+0000 7f01411b8700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 0x7f01280a5250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138005ee0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:05.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.694+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f012c09bdf0 con 0x7f013c101a40 2024-06-06T08:14:05.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.898+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f013c02dca0 con 0x7f013c101a40 2024-06-06T08:14:05.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.900+0000 7f01327fc700 1 -- 172.21.15.87:0/4192983141 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+25457 (secure 0 0 0) 0x7f012c09b6f0 con 0x7f013c101a40 2024-06-06T08:14:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-06-06T08:14:05.901 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":144,"fsid":"3f55109c-23d7-11ef-bc9b-c7b262605968","created":"2024-06-06T07:35:49.599649+0000","modified":"2024-06-06T08:13:19.973129+0000","last_up_change":"2024-06-06T07:42:56.749533+0000","last_in_change":"2024-06-06T07:42:36.528121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T07:39:10.366934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T07:43:56.169718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T07:43:59.265590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T07:44:09.602679+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1cb21c7c-fbf5-4e45-9644-1e793e30f06d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6803","nonce":3682387176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6805","nonce":3682387176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6809","nonce":3682387176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3682387176},{"type":"v1","addr":"172.21.15.87:6807","nonce":3682387176}]},"public_addr":"172.21.15.87:6803/3682387176","cluster_addr":"172.21.15.87:6805/3682387176","heartbeat_back_addr":"172.21.15.87:6809/3682387176","heartbeat_front_addr":"172.21.15.87:6807/3682387176","state":["exists","up"]},{"osd":1,"uuid":"144fe682-6bd8-4452-8e0c-cb40af20028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6801","nonce":1976649118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6803","nonce":1976649118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6807","nonce":1976649118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1976649118},{"type":"v1","addr":"172.21.15.181:6805","nonce":1976649118}]},"public_addr":"172.21.15.181:6801/1976649118","cluster_addr":"172.21.15.181:6803/1976649118","heartbeat_back_addr":"172.21.15.181:6807/1976649118","heartbeat_front_addr":"172.21.15.181:6805/1976649118","state":["exists","up"]},{"osd":2,"uuid":"cabc61d6-3e07-4e07-96bf-474a45e3c752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6801","nonce":2910401977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6803","nonce":2910401977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6807","nonce":2910401977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2910401977},{"type":"v1","addr":"172.21.15.195:6805","nonce":2910401977}]},"public_addr":"172.21.15.195:6801/2910401977","cluster_addr":"172.21.15.195:6803/2910401977","heartbeat_back_addr":"172.21.15.195:6807/2910401977","heartbeat_front_addr":"172.21.15.195:6805/2910401977","state":["exists","up"]},{"osd":3,"uuid":"1da26cda-ee26-4b26-b7c8-fed45cdac09a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6811","nonce":4203203594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6813","nonce":4203203594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6817","nonce":4203203594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":4203203594},{"type":"v1","addr":"172.21.15.87:6815","nonce":4203203594}]},"public_addr":"172.21.15.87:6811/4203203594","cluster_addr":"172.21.15.87:6813/4203203594","heartbeat_back_addr":"172.21.15.87:6817/4203203594","heartbeat_front_addr":"172.21.15.87:6815/4203203594","state":["exists","up"]},{"osd":4,"uuid":"3da9aeb5-12e1-46c8-8369-e1ee920ca615","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6809","nonce":3265557015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6811","nonce":3265557015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6815","nonce":3265557015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3265557015},{"type":"v1","addr":"172.21.15.181:6813","nonce":3265557015}]},"public_addr":"172.21.15.181:6809/3265557015","cluster_addr":"172.21.15.181:6811/3265557015","heartbeat_back_addr":"172.21.15.181:6815/3265557015","heartbeat_front_addr":"172.21.15.181:6813/3265557015","state":["exists","up"]},{"osd":5,"uuid":"3fd8d26c-d2e3-4542-9624-1d5c7cbeb684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6809","nonce":1086493667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6811","nonce":1086493667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6815","nonce":1086493667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1086493667},{"type":"v1","addr":"172.21.15.195:6813","nonce":1086493667}]},"public_addr":"172.21.15.195:6809/1086493667","cluster_addr":"172.21.15.195:6811/1086493667","heartbeat_back_addr":"172.21.15.195:6815/1086493667","heartbeat_front_addr":"172.21.15.195:6813/1086493667","state":["exists","up"]},{"osd":6,"uuid":"12b3728b-ea86-47f0-8f77-13631e3eded7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6819","nonce":2277253418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6821","nonce":2277253418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6825","nonce":2277253418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2277253418},{"type":"v1","addr":"172.21.15.87:6823","nonce":2277253418}]},"public_addr":"172.21.15.87:6819/2277253418","cluster_addr":"172.21.15.87:6821/2277253418","heartbeat_back_addr":"172.21.15.87:6825/2277253418","heartbeat_front_addr":"172.21.15.87:6823/2277253418","state":["exists","up"]},{"osd":7,"uuid":"2403cabf-c28a-4c74-b355-6c695506c49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6817","nonce":1036398045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6819","nonce":1036398045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6823","nonce":1036398045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1036398045},{"type":"v1","addr":"172.21.15.181:6821","nonce":1036398045}]},"public_addr":"172.21.15.181:6817/1036398045","cluster_addr":"172.21.15.181:6819/1036398045","heartbeat_back_addr":"172.21.15.181:6823/1036398045","heartbeat_front_addr":"172.21.15.181:6821/1036398045","state":["exists","up"]},{"osd":8,"uuid":"3720aa4b-ec90-4411-b392-93bdbe855dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6817","nonce":2643069924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6819","nonce":2643069924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6823","nonce":2643069924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2643069924},{"type":"v1","addr":"172.21.15.195:6821","nonce":2643069924}]},"public_addr":"172.21.15.195:6817/2643069924","cluster_addr":"172.21.15.195:6819/2643069924","heartbeat_back_addr":"172.21.15.195:6823/2643069924","heartbeat_front_addr":"172.21.15.195:6821/2643069924","state":["exists","up"]},{"osd":9,"uuid":"e7b1b810-b726-42ff-9bea-814b83c404f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6827","nonce":2016619295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6829","nonce":2016619295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6833","nonce":2016619295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2016619295},{"type":"v1","addr":"172.21.15.87:6831","nonce":2016619295}]},"public_addr":"172.21.15.87:6827/2016619295","cluster_addr":"172.21.15.87:6829/2016619295","heartbeat_back_addr":"172.21.15.87:6833/2016619295","heartbeat_front_addr":"172.21.15.87:6831/2016619295","state":["exists","up"]},{"osd":10,"uuid":"1caf6f4e-a4b1-4c4d-b20a-39f378a9aa0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6825","nonce":649796316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6827","nonce":649796316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6831","nonce":649796316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":649796316},{"type":"v1","addr":"172.21.15.181:6829","nonce":649796316}]},"public_addr":"172.21.15.181:6825/649796316","cluster_addr":"172.21.15.181:6827/649796316","heartbeat_back_addr":"172.21.15.181:6831/649796316","heartbeat_front_addr":"172.21.15.181:6829/649796316","state":["exists","up"]},{"osd":11,"uuid":"53318824-a6a1-4e13-8840-030854cc881d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6825","nonce":1273097292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6827","nonce":1273097292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6831","nonce":1273097292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1273097292},{"type":"v1","addr":"172.21.15.195:6829","nonce":1273097292}]},"public_addr":"172.21.15.195:6825/1273097292","cluster_addr":"172.21.15.195:6827/1273097292","heartbeat_back_addr":"172.21.15.195:6831/1273097292","heartbeat_front_addr":"172.21.15.195:6829/1273097292","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:19.472369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:38:43.646649+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:07.606558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:32.640102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:39:57.899515+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:22.168215+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:40:47.343713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:12.625932+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:41:37.664783+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:03.517789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:29.083085+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T07:42:54.920179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/2460003046":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/3344462507":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/207935511":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3853061787":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/3813763622":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/4065374776":"2024-06-07T07:48:00.385891+0000","172.21.15.87:6800/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:6801/4176793720":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/2153987725":"2024-06-07T07:48:00.385891+0000","172.21.15.87:0/1264886975":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2314391466":"2024-06-07T07:36:20.511068+0000","172.21.15.87:0/3870018525":"2024-06-07T07:36:20.511068+0000","172.21.15.87:6801/1605115594":"2024-06-07T07:36:46.051849+0000","172.21.15.87:0/2887993384":"2024-06-07T07:36:46.051849+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":44,"length":1}]},{"pool":4,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:14:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 msgr2=0x7f01280a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 0x7f01280a5250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138005ee0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 msgr2=0x7f013c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d980 tx=0x7f012c00dd40 comp rx=0 tx=0).stop 2024-06-06T08:14:05.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 shutdown_connections 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c101090 0x7f013c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f01280a2d90 0x7f01280a5250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f013c105f30 0x7f013c1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 --2- 172.21.15.87:0/4192983141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f013c101a40 0x7f013c19cb50 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 >> 172.21.15.87:0/4192983141 conn(0x7f013c0766b0 msgr2=0x7f013c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 shutdown_connections 2024-06-06T08:14:05.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:05.902+0000 7f014341c700 1 -- 172.21.15.87:0/4192983141 wait complete. 2024-06-06T08:14:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:06 smithi195 ceph-mon[37040]: pgmap v829: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 26 op/s 2024-06-06T08:14:06.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:06 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/4192983141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:14:06.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:06 smithi181 ceph-mon[36790]: pgmap v829: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 26 op/s 2024-06-06T08:14:06.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:06 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/4192983141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:14:06.608 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:06 smithi087 ceph-mon[32276]: pgmap v829: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 26 op/s 2024-06-06T08:14:06.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:06 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/4192983141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:14:06.747 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-06T08:14:07.274 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:07.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:07 smithi087 ceph-mon[32276]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T08:14:07.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:07 smithi195 ceph-mon[37040]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T08:14:07.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:07 smithi181 ceph-mon[36790]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T08:14:08.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:08 smithi195 ceph-mon[37040]: pgmap v831: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 193 KiB/s wr, 30 op/s 2024-06-06T08:14:08.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:08 smithi181 ceph-mon[36790]: pgmap v831: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 193 KiB/s wr, 30 op/s 2024-06-06T08:14:08.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:08 smithi087 ceph-mon[32276]: pgmap v831: 129 pgs: 129 active+clean; 719 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 193 KiB/s wr, 30 op/s 2024-06-06T08:14:09.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.790+0000 7f8418995700 1 -- 172.21.15.87:0/2950907540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f841411bc70 msgr2=0x7f841411e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:09.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.790+0000 7f8418995700 1 --2- 172.21.15.87:0/2950907540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f841411bc70 0x7f841411e060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b0a0 tx=0x7f840c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.792+0000 7f8418995700 1 -- 172.21.15.87:0/2950907540 shutdown_connections 2024-06-06T08:14:09.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.792+0000 7f8418995700 1 --2- 172.21.15.87:0/2950907540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f841411bc70 0x7f841411e060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.792+0000 7f8418995700 1 --2- 172.21.15.87:0/2950907540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414110bf0 0x7f8414109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.792+0000 7f8418995700 1 --2- 172.21.15.87:0/2950907540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8414111060 0x7f84141099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.792+0000 7f8418995700 1 -- 172.21.15.87:0/2950907540 >> 172.21.15.87:0/2950907540 conn(0x7f841406d600 msgr2=0x7f841406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:09.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.794+0000 7f8418995700 1 -- 172.21.15.87:0/2950907540 shutdown_connections 2024-06-06T08:14:09.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.795+0000 7f8418995700 1 -- 172.21.15.87:0/2950907540 wait complete. 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.796+0000 7f8418995700 1 Processor -- start 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 -- start start 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414110bf0 0x7f84141151b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f841411bc70 0x7f84141156f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111a60 con 0x7f8414119a50 2024-06-06T08:14:09.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84141118e0 con 0x7f8414110bf0 2024-06-06T08:14:09.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.797+0000 7f8418995700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111be0 con 0x7f841411bc70 2024-06-06T08:14:09.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:09.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51602/0 (socket says 172.21.15.87:51602) 2024-06-06T08:14:09.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 -- 172.21.15.87:0/2090359899 learned_addr learned my addr 172.21.15.87:0/2090359899 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 -- 172.21.15.87:0/2090359899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f841411bc70 msgr2=0x7f84141156f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f841411bc70 0x7f84141156f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 -- 172.21.15.87:0/2090359899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414110bf0 msgr2=0x7f84141151b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414110bf0 0x7f84141151b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 -- 172.21.15.87:0/2090359899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841411a600 con 0x7f8414119a50 2024-06-06T08:14:09.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.798+0000 7f8412d9d700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b070 tx=0x7f840c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:09.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.802+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c022030 con 0x7f8414119a50 2024-06-06T08:14:09.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.802+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841411a890 con 0x7f8414119a50 2024-06-06T08:14:09.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.802+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141273f0 con 0x7f8414119a50 2024-06-06T08:14:09.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.803+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840c013c60 con 0x7f8414119a50 2024-06-06T08:14:09.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.803+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841406eb60 con 0x7f8414119a50 2024-06-06T08:14:09.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.807+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c004820 con 0x7f8414119a50 2024-06-06T08:14:09.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.810+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f840c025070 con 0x7f8414119a50 2024-06-06T08:14:09.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.811+0000 7f84037fe700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 0x7f83fc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:09.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.811+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 9671+0+0 (secure 0 0 0) 0x7f840c0d0af0 con 0x7f8414119a50 2024-06-06T08:14:09.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.813+0000 7f841259c700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 0x7f83fc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:09.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.817+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f840c09d460 con 0x7f8414119a50 2024-06-06T08:14:09.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:09.828+0000 7f841259c700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 0x7f83fc0a5460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8404009710 tx=0x7f8404006c60 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:10.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.287+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84141111e0 con 0x7f8414119a50 2024-06-06T08:14:10.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.299+0000 7f84037fe700 1 -- 172.21.15.87:0/2090359899 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7f840c09cd60 con 0x7f8414119a50 2024-06-06T08:14:10.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 msgr2=0x7f83fc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:10.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 0x7f83fc0a5460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8404009710 tx=0x7f8404006c60 comp rx=0 tx=0).stop 2024-06-06T08:14:10.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 msgr2=0x7f8414119ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b070 tx=0x7f840c004e70 comp rx=0 tx=0).stop 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 shutdown_connections 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414110bf0 0x7f84141151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f83fc0a2fa0 0x7f83fc0a5460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8414119a50 0x7f8414119ed0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.304+0000 7f8418995700 1 --2- 172.21.15.87:0/2090359899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f841411bc70 0x7f84141156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.305+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 >> 172.21.15.87:0/2090359899 conn(0x7f841406d600 msgr2=0x7f841411c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:10.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.305+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 shutdown_connections 2024-06-06T08:14:10.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:10.305+0000 7f8418995700 1 -- 172.21.15.87:0/2090359899 wait complete. 2024-06-06T08:14:10.306 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-06T08:14:10.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-mon[37040]: pgmap v832: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 88 KiB/s wr, 30 op/s 2024-06-06T08:14:10.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-mds[67979]: -- 172.21.15.195:0/842086428 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa1bf97a40 con 0x55aa1bf6ac00 2024-06-06T08:14:10.510 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: starting mds.c at 2024-06-06T08:14:10.511 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.511 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.511 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-mds[67666]: -- 172.21.15.195:0/2101790653 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef43047a40 con 0x55ef4301a800 2024-06-06T08:14:10.511 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: starting mds.f at 2024-06-06T08:14:10.511 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.512 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.512 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-mds[68279]: -- 172.21.15.195:0/1688854200 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae73d69a40 con 0x55ae73d3c800 2024-06-06T08:14:10.512 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:14:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: starting mds.i at 2024-06-06T08:14:10.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-mon[36790]: pgmap v832: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 88 KiB/s wr, 30 op/s 2024-06-06T08:14:10.556 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.556 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.556 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-mds[67353]: -- 172.21.15.181:0/1763416023 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56245aa05a40 con 0x56245a9d8800 2024-06-06T08:14:10.556 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: starting mds.b at 2024-06-06T08:14:10.557 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.557 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.557 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-mds[66733]: -- 172.21.15.181:0/2249301307 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d67e7fa40 con 0x559d67e53000 2024-06-06T08:14:10.557 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: starting mds.e at 2024-06-06T08:14:10.557 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: ignoring --setuser ceph since I am not root 2024-06-06T08:14:10.558 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: ignoring --setgroup ceph since I am not root 2024-06-06T08:14:10.558 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-mds[67049]: -- 172.21.15.181:0/4235222529 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1be8c5a40 con 0x55f1be898c00 2024-06-06T08:14:10.558 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:14:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: starting mds.h at 2024-06-06T08:14:10.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:10 smithi087 ceph-mon[32276]: pgmap v832: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 88 KiB/s wr, 30 op/s 2024-06-06T08:14:10.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:10 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T08:14:10.289+0000 7f6106edb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:14:11.070 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-06T08:14:11.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:14:11.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:14:11.359 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:14:11.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-06-06T08:14:11.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:14:11.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T08:14:11.360 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:11 smithi087 ceph-mon[32276]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:14:11.490 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:11.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:14:11.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:14:11.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:14:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-06-06T08:14:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:14:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T08:14:11.510 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:11 smithi195 ceph-mon[37040]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:14:11.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/2090359899' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T08:14:11.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:11 smithi181 ceph-mon[36790]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:14:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: pgmap v834: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 74 KiB/s wr, 30 op/s 2024-06-06T08:14:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6834/1128383670,v1:172.21.15.181:6835/1128383670] up:boot 2024-06-06T08:14:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6836/75942834,v1:172.21.15.195:6839/75942834] up:boot 2024-06-06T08:14:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6834/4224053176,v1:172.21.15.195:6838/4224053176] up:boot 2024-06-06T08:14:12.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.195:6835/3574742339,v1:172.21.15.195:6837/3574742339] up:boot 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6836/3173520848,v1:172.21.15.181:6837/3173520848] up:boot 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: mds.? [v2:172.21.15.181:6838/1431825935,v1:172.21.15.181:6839/1431825935] up:boot 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:14:12.610 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:14:12.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:12 smithi087 ceph-mon[32276]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:14:12.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: pgmap v834: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 74 KiB/s wr, 30 op/s 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6834/1128383670,v1:172.21.15.181:6835/1128383670] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6836/75942834,v1:172.21.15.195:6839/75942834] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6834/4224053176,v1:172.21.15.195:6838/4224053176] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.195:6835/3574742339,v1:172.21.15.195:6837/3574742339] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6836/3173520848,v1:172.21.15.181:6837/3173520848] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: mds.? [v2:172.21.15.181:6838/1431825935,v1:172.21.15.181:6839/1431825935] up:boot 2024-06-06T08:14:12.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:14:12.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:12 smithi195 ceph-mon[37040]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: pgmap v834: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 715 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 74 KiB/s wr, 30 op/s 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6834/1128383670,v1:172.21.15.181:6835/1128383670] up:boot 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6836/75942834,v1:172.21.15.195:6839/75942834] up:boot 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6834/4224053176,v1:172.21.15.195:6838/4224053176] up:boot 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.195:6835/3574742339,v1:172.21.15.195:6837/3574742339] up:boot 2024-06-06T08:14:12.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6836/3173520848,v1:172.21.15.181:6837/3173520848] up:boot 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: mds.? [v2:172.21.15.181:6838/1431825935,v1:172.21.15.181:6839/1431825935] up:boot 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:14:12.807 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:12 smithi181 ceph-mon[36790]: from='mgr.24800 172.21.15.87:0/1295611828' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:14:13.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 -- 172.21.15.87:0/3733388373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 msgr2=0x7f84dc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:13.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 --2- 172.21.15.87:0/3733388373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc107130 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f84c40099c0 tx=0x7f84c4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 -- 172.21.15.87:0/3733388373 shutdown_connections 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 --2- 172.21.15.87:0/3733388373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 0x7f84dc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 --2- 172.21.15.87:0/3733388373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 --2- 172.21.15.87:0/3733388373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc107130 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 -- 172.21.15.87:0/3733388373 >> 172.21.15.87:0/3733388373 conn(0x7f84dc079600 msgr2=0x7f84dc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 -- 172.21.15.87:0/3733388373 shutdown_connections 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.625+0000 7f84e1b54700 1 -- 172.21.15.87:0/3733388373 wait complete. 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 Processor -- start 2024-06-06T08:14:13.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 -- start start 2024-06-06T08:14:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 0x7f84dc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc112790 con 0x7f84dc103120 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc112610 con 0x7f84dc107860 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84e1b54700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc112910 con 0x7f84dc102770 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51630/0 (socket says 172.21.15.87:51630) 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 -- 172.21.15.87:0/99376809 learned_addr learned my addr 172.21.15.87:0/99376809 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:13.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84dbfff700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 0x7f84dc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84db7fe700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 -- 172.21.15.87:0/99376809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 msgr2=0x7f84dc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 -- 172.21.15.87:0/99376809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 msgr2=0x7f84dc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 0x7f84dc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84d2ffd700 1 -- 172.21.15.87:0/99376809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc199190 con 0x7f84dc103120 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.626+0000 7f84db7fe700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:14:13.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84d2ffd700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f84c40059d0 tx=0x7f84c40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4027070 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc199420 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4004050 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4022db0 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.627+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a56f0 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.628+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc10aa30 con 0x7f84dc103120 2024-06-06T08:14:13.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.629+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f84c402a070 con 0x7f84dc103120 2024-06-06T08:14:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.630+0000 7f84d97fa700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 0x7f84c80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.630+0000 7f84db7fe700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 0x7f84c80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.630+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9841+0+0 (secure 0 0 0) 0x7f84c40d0b00 con 0x7f84dc103120 2024-06-06T08:14:13.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.631+0000 7f84db7fe700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 0x7f84c80a5250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84dc107360 tx=0x7f84cc00b2c0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:13.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.634+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84c405e850 con 0x7f84dc103120 2024-06-06T08:14:13.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:13.878+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84dc199f10 con 0x7f84dc103120 2024-06-06T08:14:14.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.319+0000 7f84d97fa700 1 -- 172.21.15.87:0/99376809 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7f84c4013070 con 0x7f84dc103120 2024-06-06T08:14:14.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.321+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 msgr2=0x7f84c80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:14.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.321+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 0x7f84c80a5250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84dc107360 tx=0x7f84cc00b2c0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.321+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 msgr2=0x7f84dc19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:14.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.321+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f84c40059d0 tx=0x7f84c40038b0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 shutdown_connections 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc107860 0x7f84dc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f84c80a2d90 0x7f84c80a5250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc102770 0x7f84dc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 --2- 172.21.15.87:0/99376809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84dc103120 0x7f84dc19ed60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 >> 172.21.15.87:0/99376809 conn(0x7f84dc079600 msgr2=0x7f84dc1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 shutdown_connections 2024-06-06T08:14:14.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:14.322+0000 7f84e1b54700 1 -- 172.21.15.87:0/99376809 wait complete. 2024-06-06T08:14:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:14 smithi087 ceph-mon[32276]: pgmap v835: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 704 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 9 op/s 2024-06-06T08:14:14.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:14 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:14:14.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:14 smithi195 ceph-mon[37040]: pgmap v835: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 704 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 9 op/s 2024-06-06T08:14:14.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:14 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:14:14.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:14 smithi181 ceph-mon[36790]: pgmap v835: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 704 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 9 op/s 2024-06-06T08:14:14.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:14 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:14:15.068 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-06T08:14:15.310 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:15.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:15 smithi087 ceph-mon[32276]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:14:15.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:15 smithi087 ceph-mon[32276]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:14:15.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:15 smithi087 ceph-mon[32276]: Cluster is now healthy 2024-06-06T08:14:15.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:15 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:14:15.565 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:15 smithi087 ceph-mon[32276]: fsmap 12 up:standby 2024-06-06T08:14:15.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:15 smithi195 ceph-mon[37040]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:14:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:15 smithi195 ceph-mon[37040]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:14:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:15 smithi195 ceph-mon[37040]: Cluster is now healthy 2024-06-06T08:14:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:15 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:14:15.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:15 smithi195 ceph-mon[37040]: fsmap 12 up:standby 2024-06-06T08:14:15.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:15 smithi181 ceph-mon[36790]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:14:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:15 smithi181 ceph-mon[36790]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:14:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:15 smithi181 ceph-mon[36790]: Cluster is now healthy 2024-06-06T08:14:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:15 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/99376809' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:14:15.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:15 smithi181 ceph-mon[36790]: fsmap 12 up:standby 2024-06-06T08:14:16.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:16 smithi087 ceph-mon[32276]: pgmap v836: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 8 op/s 2024-06-06T08:14:16.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:16 smithi195 ceph-mon[37040]: pgmap v836: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 8 op/s 2024-06-06T08:14:16.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:16 smithi181 ceph-mon[36790]: pgmap v836: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 8 op/s 2024-06-06T08:14:17.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 -- 172.21.15.87:0/1309122477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0101680 msgr2=0x7f53c0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:17.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/1309122477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0101680 0x7f53c0101a60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0099c0 tx=0x7f53bc009cd0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 -- 172.21.15.87:0/1309122477 shutdown_connections 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/1309122477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c01066d0 0x7f53c010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/1309122477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0101680 0x7f53c0101a60 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/1309122477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c0102030 0x7f53c0105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 -- 172.21.15.87:0/1309122477 >> 172.21.15.87:0/1309122477 conn(0x7f53c00773e0 msgr2=0x7f53c00777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 -- 172.21.15.87:0/1309122477 shutdown_connections 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.214+0000 7f53c7c3b700 1 -- 172.21.15.87:0/1309122477 wait complete. 2024-06-06T08:14:17.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 Processor -- start 2024-06-06T08:14:17.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 -- start start 2024-06-06T08:14:17.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c0101680 0x7f53c0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:17.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 0x7f53c019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53c0111730 con 0x7f53c0102030 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53c01115b0 con 0x7f53c01066d0 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c7c3b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53c01118b0 con 0x7f53c0101680 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37360/0 (socket says 172.21.15.87:37360) 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 -- 172.21.15.87:0/581408979 learned_addr learned my addr 172.21.15.87:0/581408979 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:17.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c59d7700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c0101680 0x7f53c0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c61d8700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 0x7f53c019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 -- 172.21.15.87:0/581408979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c0101680 msgr2=0x7f53c0198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c0101680 0x7f53c0198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 -- 172.21.15.87:0/581408979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 msgr2=0x7f53c019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 0x7f53c019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 -- 172.21.15.87:0/581408979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c019ddc0 con 0x7f53c0102030 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c61d8700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 0x7f53c019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:14:17.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.215+0000 7f53c51d6700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f53b000b500 tx=0x7f53b000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.216+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0015020 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.216+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c019e0b0 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.216+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b0011d10 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.216+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b000f2e0 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.216+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c01a5930 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.217+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c0069180 con 0x7f53c0102030 2024-06-06T08:14:17.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.218+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f53b0018020 con 0x7f53c0102030 2024-06-06T08:14:17.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.218+0000 7f53b6ffd700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 0x7f53ac0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:17.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.219+0000 7f53c59d7700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 0x7f53ac0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:17.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.219+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9841+0+0 (secure 0 0 0) 0x7f53b00cf020 con 0x7f53c0102030 2024-06-06T08:14:17.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.219+0000 7f53c59d7700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 0x7f53ac0a5250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009f50 tx=0x7f53bc005fb0 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:17.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.223+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53b001f020 con 0x7f53c0102030 2024-06-06T08:14:17.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:17.418+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53c019a110 con 0x7f53c0102030 2024-06-06T08:14:18.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.333+0000 7f53b6ffd700 1 -- 172.21.15.87:0/581408979 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f53b009b820 con 0x7f53c0102030 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 msgr2=0x7f53ac0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 0x7f53ac0a5250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009f50 tx=0x7f53bc005fb0 comp rx=0 tx=0).stop 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 msgr2=0x7f53c01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f53b000b500 tx=0x7f53b000b810 comp rx=0 tx=0).stop 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 shutdown_connections 2024-06-06T08:14:18.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c01066d0 0x7f53c019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:18.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f53ac0a2d90 0x7f53ac0a5250 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:18.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53c0101680 0x7f53c0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:18.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 --2- 172.21.15.87:0/581408979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53c0102030 0x7f53c01992f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:18.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.336+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 >> 172.21.15.87:0/581408979 conn(0x7f53c00773e0 msgr2=0x7f53c0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:18.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.337+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 shutdown_connections 2024-06-06T08:14:18.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:18.337+0000 7f53c7c3b700 1 -- 172.21.15.87:0/581408979 wait complete. 2024-06-06T08:14:18.339 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-06-06T08:14:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:18 smithi087 ceph-mon[32276]: pgmap v837: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 7 op/s 2024-06-06T08:14:18.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:18 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:18.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:18 smithi195 ceph-mon[37040]: pgmap v837: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 7 op/s 2024-06-06T08:14:18.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:18 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:18.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:18 smithi181 ceph-mon[36790]: pgmap v837: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 695 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 7 op/s 2024-06-06T08:14:18.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:18 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:19.040 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-06T08:14:19.277 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:19.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:19 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:19.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:19 smithi087 ceph-mon[32276]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T08:14:19.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:19 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:19.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:19 smithi195 ceph-mon[37040]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T08:14:19.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:19 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/581408979' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:19.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:19 smithi181 ceph-mon[36790]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T08:14:20.436 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:20 smithi087 ceph-mon[32276]: pgmap v839: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:20.436 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:20 smithi087 ceph-mon[32276]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T08:14:20.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:20 smithi195 ceph-mon[37040]: pgmap v839: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:20.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:20 smithi195 ceph-mon[37040]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T08:14:20.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:20 smithi181 ceph-mon[36790]: pgmap v839: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:20.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:20 smithi181 ceph-mon[36790]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T08:14:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 -- 172.21.15.87:0/217783495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 msgr2=0x7f3ca01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/217783495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca01012d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009990 tx=0x7f3c94009ca0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 -- 172.21.15.87:0/217783495 shutdown_connections 2024-06-06T08:14:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/217783495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0107570 0x7f3ca0107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/217783495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0101810 0x7f3ca010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/217783495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca01012d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 -- 172.21.15.87:0/217783495 >> 172.21.15.87:0/217783495 conn(0x7f3ca00766b0 msgr2=0x7f3ca0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 -- 172.21.15.87:0/217783495 shutdown_connections 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 -- 172.21.15.87:0/217783495 wait complete. 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.005+0000 7f3ca5dde700 1 Processor -- start 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 -- start start 2024-06-06T08:14:21.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0101810 0x7f3ca019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0107570 0x7f3ca0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01127b0 con 0x7f3ca0100e50 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca0112630 con 0x7f3ca0101810 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3ca5dde700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca0112930 con 0x7f3ca0107570 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37378/0 (socket says 172.21.15.87:37378) 2024-06-06T08:14:21.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 -- 172.21.15.87:0/279621267 learned_addr learned my addr 172.21.15.87:0/279621267 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9effd700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0101810 0x7f3ca019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9ffff700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0107570 0x7f3ca0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 -- 172.21.15.87:0/279621267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0107570 msgr2=0x7f3ca0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0107570 0x7f3ca0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 -- 172.21.15.87:0/279621267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0101810 msgr2=0x7f3ca019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0101810 0x7f3ca019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.006+0000 7f3c9f7fe700 1 -- 172.21.15.87:0/279621267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca01991d0 con 0x7f3ca0100e50 2024-06-06T08:14:21.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3c9f7fe700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007f10 tx=0x7f3c9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:21.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000dbd0 con 0x7f3ca0100e50 2024-06-06T08:14:21.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca01994c0 con 0x7f3ca0100e50 2024-06-06T08:14:21.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9000dd70 con 0x7f3ca0100e50 2024-06-06T08:14:21.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c900133f0 con 0x7f3ca0100e50 2024-06-06T08:14:21.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.007+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca01a5780 con 0x7f3ca0100e50 2024-06-06T08:14:21.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.008+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca0069180 con 0x7f3ca0100e50 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.009+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3c9001a020 con 0x7f3ca0100e50 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.009+0000 7f3c9cff9700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 0x7f3c880a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.010+0000 7f3c9effd700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 0x7f3c880a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.010+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9287+0+0 (secure 0 0 0) 0x7f3c900cf4c0 con 0x7f3ca0100e50 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.010+0000 7f3c9effd700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 0x7f3c880a5250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b5c0 tx=0x7f3c94005960 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:21.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.013+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c90010070 con 0x7f3ca0100e50 2024-06-06T08:14:21.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.208+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ca002dca0 con 0x7f3ca0100e50 2024-06-06T08:14:21.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.349+0000 7f3c9cff9700 1 -- 172.21.15.87:0/279621267 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f3c9009bde0 con 0x7f3ca0100e50 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 msgr2=0x7f3c880a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 0x7f3c880a5250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b5c0 tx=0x7f3c94005960 comp rx=0 tx=0).stop 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 msgr2=0x7f3ca019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007f10 tx=0x7f3c9000d3b0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 shutdown_connections 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f3c880a2d90 0x7f3c880a5250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ca0107570 0x7f3ca0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ca0100e50 0x7f3ca019e8b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 --2- 172.21.15.87:0/279621267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ca0101810 0x7f3ca019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 >> 172.21.15.87:0/279621267 conn(0x7f3ca00766b0 msgr2=0x7f3ca00ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 shutdown_connections 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:21.351+0000 7f3ca5dde700 1 -- 172.21.15.87:0/279621267 wait complete. 2024-06-06T08:14:21.352 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-06-06T08:14:21.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:21 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:21.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:21 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:21.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:21 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:22.444 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:22 smithi087 ceph-mon[32276]: pgmap v841: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:22.445 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:22 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:22.445 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:22 smithi087 ceph-mon[32276]: osdmap e149: 12 total, 12 up, 12 in 2024-06-06T08:14:22.476 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-06T08:14:22.718 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:14:22.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:22 smithi195 ceph-mon[37040]: pgmap v841: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:22.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:22 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:22.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:22 smithi195 ceph-mon[37040]: osdmap e149: 12 total, 12 up, 12 in 2024-06-06T08:14:22.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:22 smithi181 ceph-mon[36790]: pgmap v841: 97 pgs: 97 active+clean; 316 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:22.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:22 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/279621267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:22.806 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:22 smithi181 ceph-mon[36790]: osdmap e149: 12 total, 12 up, 12 in 2024-06-06T08:14:24.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 -- 172.21.15.87:0/1548250750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101390 msgr2=0x7f2bf8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/1548250750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101390 0x7f2bf8101810 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009990 tx=0x7f2bec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 -- 172.21.15.87:0/1548250750 shutdown_connections 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/1548250750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 0x7f2bf81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/1548250750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101d50 0x7f2bf810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/1548250750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101390 0x7f2bf8101810 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 -- 172.21.15.87:0/1548250750 >> 172.21.15.87:0/1548250750 conn(0x7f2bf8078340 msgr2=0x7f2bf8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 -- 172.21.15.87:0/1548250750 shutdown_connections 2024-06-06T08:14:24.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.384+0000 7f2bff3e8700 1 -- 172.21.15.87:0/1548250750 wait complete. 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 Processor -- start 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 -- start start 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 0x7f2bf8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 0x7f2bf81a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf81116a0 con 0x7f2bf8101d50 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8111520 con 0x7f2bf8107400 2024-06-06T08:14:24.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bff3e8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8111820 con 0x7f2bf8101390 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bfc983700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bfc983700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37392/0 (socket says 172.21.15.87:37392) 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bfc983700 1 -- 172.21.15.87:0/507662180 learned_addr learned my addr 172.21.15.87:0/507662180 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.385+0000 7f2bfd184700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 0x7f2bf8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfd985700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 0x7f2bf81a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 -- 172.21.15.87:0/507662180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 msgr2=0x7f2bf8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 0x7f2bf8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 -- 172.21.15.87:0/507662180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 msgr2=0x7f2bf81a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 0x7f2bf81a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 -- 172.21.15.87:0/507662180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf81a5020 con 0x7f2bf8101d50 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfd184700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 0x7f2bf8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfc983700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec027070 con 0x7f2bf8101d50 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec004050 con 0x7f2bf8101d50 2024-06-06T08:14:24.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00f460 con 0x7f2bf8101d50 2024-06-06T08:14:24.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf81a51f0 con 0x7f2bf8101d50 2024-06-06T08:14:24.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf81a56a0 con 0x7f2bf8101d50 2024-06-06T08:14:24.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf8069180 con 0x7f2bf8101d50 2024-06-06T08:14:24.391 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:24 smithi087 ceph-mon[32276]: pgmap v843: 33 pgs: 33 active+clean; 316 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:24.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2bec02a070 con 0x7f2bf8101d50 2024-06-06T08:14:24.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 0x7f2be00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:14:24.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 8582+0+0 (secure 0 0 0) 0x7f2bec0d07e0 con 0x7f2bf8101d50 2024-06-06T08:14:24.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfd184700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 0x7f2be00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:14:24.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.386+0000 7f2bfd184700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 0x7f2be00a52d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009d50 tx=0x7f2bf0009040 comp rx=0 tx=0).ready entity=mgr.24800 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:14:24.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.392+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2bec031070 con 0x7f2bf8101d50 2024-06-06T08:14:24.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:24.635+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2bf802dca0 con 0x7f2bf8101d50 2024-06-06T08:14:24.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:24 smithi195 ceph-mon[37040]: pgmap v843: 33 pgs: 33 active+clean; 316 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:24.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:24 smithi181 ceph-mon[36790]: pgmap v843: 33 pgs: 33 active+clean; 316 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:25.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.365+0000 7f2bf67fc700 1 -- 172.21.15.87:0/507662180 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v150) ==== 178+0+0 (secure 0 0 0) 0x7f2bec013070 con 0x7f2bf8101d50 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 msgr2=0x7f2be00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 0x7f2be00a52d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009d50 tx=0x7f2bf0009040 comp rx=0 tx=0).stop 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 msgr2=0x7f2bf8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec0047b0 comp rx=0 tx=0).stop 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 shutdown_connections 2024-06-06T08:14:25.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf8107400 0x7f2bf81a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:6800/4062973588,v1:172.21.15.87:6801/4062973588] conn(0x7f2be00a2e10 0x7f2be00a52d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bf8101390 0x7f2bf8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 --2- 172.21.15.87:0/507662180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bf8101d50 0x7f2bf8199fa0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.368+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 >> 172.21.15.87:0/507662180 conn(0x7f2bf8078340 msgr2=0x7f2bf80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.369+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 shutdown_connections 2024-06-06T08:14:25.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-06-06T08:14:25.369+0000 7f2bff3e8700 1 -- 172.21.15.87:0/507662180 wait complete. 2024-06-06T08:14:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data_ec' removed 2024-06-06T08:14:25.609 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:25 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:25.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:25 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:25.805 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:25 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:14:26.065 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:14:26.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:14:26.268 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:14:26.312 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-06T08:14:26.313 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:14:26.333 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:14:26.347 DEBUG:teuthology.orchestra.run.smithi195:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:14:26.369 INFO:tasks.cephadm:Stopping all daemons... 2024-06-06T08:14:26.369 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-06T08:14:26.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a 2024-06-06T08:14:26.555 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:26 smithi181 ceph-mon[36790]: pgmap v844: 33 pgs: 33 active+clean; 316 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:26 smithi181 ceph-mon[36790]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:26.556 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:26 smithi181 ceph-mon[36790]: osdmap e150: 12 total, 12 up, 12 in 2024-06-06T08:14:26.722 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 ceph-mon[32276]: pgmap v844: 33 pgs: 33 active+clean; 316 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:26.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 ceph-mon[32276]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:26.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 ceph-mon[32276]: osdmap e150: 12 total, 12 up, 12 in 2024-06-06T08:14:26.723 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 systemd[1]: Stopping Ceph mon.a for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:26.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:26 smithi195 ceph-mon[37040]: pgmap v844: 33 pgs: 33 active+clean; 316 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:14:26.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:26 smithi195 ceph-mon[37040]: from='client.? 172.21.15.87:0/507662180' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:14:26.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:26 smithi195 ceph-mon[37040]: osdmap e150: 12 total, 12 up, 12 in 2024-06-06T08:14:27.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T08:14:26.722+0000 7f610c6e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:27.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a[32272]: 2024-06-06T08:14:26.722+0000 7f610c6e6700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-06T08:14:27.109 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:26 smithi087 podman[125904]: 2024-06-06 08:14:26.937450118 +0000 UTC m=+0.338646547 container died d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T08:14:27.990 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:27 smithi087 podman[125904]: 2024-06-06 08:14:27.656465882 +0000 UTC m=+1.057662304 container cleanup d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T08:14:27.990 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:27 smithi087 bash[125904]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a 2024-06-06T08:14:28.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Jun 06 08:14:27 smithi087 podman[125917]: 2024-06-06 08:14:27.991281849 +0000 UTC m=+1.047588802 container remove d7ad1a2e106ca1228dbf1da7ede73ca0c40f353a8d2671413a811457bfff30e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:14:28.243 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-06T08:14:28.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:28.297 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-06T08:14:28.297 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-06T08:14:28.297 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.b 2024-06-06T08:14:28.685 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:28 smithi181 systemd[1]: Stopping Ceph mon.b for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:29.011 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-b[36785]: 2024-06-06T08:14:28.686+0000 7fe3d6bdb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:29.012 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-b[36785]: 2024-06-06T08:14:28.686+0000 7fe3d6bdb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-06T08:14:29.012 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:28 smithi181 podman[78951]: 2024-06-06 08:14:28.727163725 +0000 UTC m=+0.169255944 container died 39452bd0d8b23086d5921c0f7fca37e0eadb711d8160693d0afcb27385c71065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-b, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2) 2024-06-06T08:14:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:29 smithi181 podman[78951]: 2024-06-06 08:14:29.10333966 +0000 UTC m=+0.545431887 container cleanup 39452bd0d8b23086d5921c0f7fca37e0eadb711d8160693d0afcb27385c71065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:14:29.306 INFO:journalctl@ceph.mon.b.smithi181.stdout:Jun 06 08:14:29 smithi181 bash[78951]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-b 2024-06-06T08:14:29.514 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-06T08:14:29.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:29.585 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-06T08:14:29.585 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-06T08:14:29.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.c 2024-06-06T08:14:29.945 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:29 smithi195 systemd[1]: Stopping Ceph mon.c for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:30.199 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c[37024]: 2024-06-06T08:14:29.945+0000 7f9d37ad2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:30.200 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c[37024]: 2024-06-06T08:14:29.945+0000 7f9d37ad2700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-06T08:14:30.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:30 smithi195 podman[81866]: 2024-06-06 08:14:30.201275166 +0000 UTC m=+0.353521524 container died a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T08:14:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:30 smithi195 podman[81866]: 2024-06-06 08:14:30.557321746 +0000 UTC m=+0.709568100 container cleanup a034b190713c275de0f021ff2d3542e4837a5362b3ae8563f77896e7df0f074f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:14:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Jun 06 08:14:30 smithi195 bash[81866]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mon-c 2024-06-06T08:14:30.991 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-06T08:14:31.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:31.032 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-06T08:14:31.032 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-06T08:14:31.032 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T08:14:31.347 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 08:14:31 smithi087 systemd[1]: Stopping Ceph mgr.x for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:31.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 08:14:31 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x[32503]: 2024-06-06T08:14:31.347+0000 7f5852563700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:31.609 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 08:14:31 smithi087 podman[126025]: 2024-06-06 08:14:31.379164703 +0000 UTC m=+0.154002072 container died 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:14:32.012 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 08:14:31 smithi087 podman[126025]: 2024-06-06 08:14:31.75333974 +0000 UTC m=+0.528177094 container cleanup 596db31ad0949a465a7c54e1d93f4c750f17516b4ecf0a6a9d4fdc7f152db012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-06T08:14:32.012 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Jun 06 08:14:31 smithi087 bash[126025]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-x 2024-06-06T08:14:32.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-06T08:14:32.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:32.209 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-06T08:14:32.209 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-06T08:14:32.209 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.y 2024-06-06T08:14:32.544 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 08:14:32 smithi181 systemd[1]: Stopping Ceph mgr.y for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:32.806 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 08:14:32 smithi181 podman[79073]: 2024-06-06 08:14:32.545492638 +0000 UTC m=+0.144990918 container died 330057fe4e478927f31c73ee7963dde2b1bdd9ffa98ea80fada6befa0734587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T08:14:33.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 08:14:32 smithi181 podman[79073]: 2024-06-06 08:14:32.984128839 +0000 UTC m=+0.583627110 container cleanup 330057fe4e478927f31c73ee7963dde2b1bdd9ffa98ea80fada6befa0734587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T08:14:33.306 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Jun 06 08:14:32 smithi181 bash[79073]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-y 2024-06-06T08:14:33.476 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-06T08:14:33.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:33.512 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-06T08:14:33.512 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-06T08:14:33.512 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.z 2024-06-06T08:14:33.841 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:33 smithi195 systemd[1]: Stopping Ceph mgr.z for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:34.154 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:33 smithi195 podman[81990]: 2024-06-06 08:14:33.841951415 +0000 UTC m=+0.134778042 container stop f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:14:34.155 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:33 smithi195 podman[81990]: 2024-06-06 08:14:33.867650157 +0000 UTC m=+0.160476781 container died f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True) 2024-06-06T08:14:34.471 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:34 smithi195 podman[81990]: 2024-06-06 08:14:34.263364109 +0000 UTC m=+0.556190730 container cleanup f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T08:14:34.471 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:34 smithi195 bash[81990]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z 2024-06-06T08:14:34.748 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:34 smithi195 podman[82003]: 2024-06-06 08:14:34.471914518 +0000 UTC m=+0.619679849 container remove f0c623244cee216d34b3d36aec87f5179cd04437280920fa48d7cfd04a1f9a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T08:14:34.748 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Jun 06 08:14:34 smithi195 systemd[1]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-06-06T08:14:34.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-06T08:14:34.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:34.799 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-06T08:14:34.799 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-06T08:14:34.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.0 2024-06-06T08:14:35.109 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:34 smithi087 systemd[1]: Stopping Ceph osd.0 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:35.609 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0[44748]: 2024-06-06T08:14:35.123+0000 7efcbd33b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:35.609 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0[44748]: 2024-06-06T08:14:35.123+0000 7efcbd33b700 -1 osd.0 150 *** Got signal Terminated *** 2024-06-06T08:14:35.609 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0[44748]: 2024-06-06T08:14:35.123+0000 7efcbd33b700 -1 osd.0 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:14:41.024 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:40 smithi087 podman[126146]: 2024-06-06 08:14:40.734028505 +0000 UTC m=+5.734258785 container died 41a24b68d17b1a0ad7bb869b4531c0c99fd1ccb52f616af1bbb8e57fc7dafbae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T08:14:41.359 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:41 smithi087 podman[126146]: 2024-06-06 08:14:41.133142243 +0000 UTC m=+6.133372515 container cleanup 41a24b68d17b1a0ad7bb869b4531c0c99fd1ccb52f616af1bbb8e57fc7dafbae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:14:41.359 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:41 smithi087 bash[126146]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0 2024-06-06T08:14:41.752 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:41 smithi087 podman[126163]: 2024-06-06 08:14:41.384720629 +0000 UTC m=+0.647777186 container remove 41a24b68d17b1a0ad7bb869b4531c0c99fd1ccb52f616af1bbb8e57fc7dafbae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T08:14:42.021 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:41 smithi087 podman[126251]: 2024-06-06 08:14:41.655806624 +0000 UTC m=+0.017621337 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:14:42.021 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:41 smithi087 podman[126251]: 2024-06-06 08:14:41.760889347 +0000 UTC m=+0.122704059 container create aab94dd2973e3ff37d1bac96aad8302994fe39d875317f2a889a5c97c1e1ae03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T08:14:42.333 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:42 smithi087 podman[126251]: 2024-06-06 08:14:42.022315719 +0000 UTC m=+0.384130442 container init aab94dd2973e3ff37d1bac96aad8302994fe39d875317f2a889a5c97c1e1ae03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-06T08:14:42.334 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:42 smithi087 podman[126251]: 2024-06-06 08:14:42.025924743 +0000 UTC m=+0.387739442 container start aab94dd2973e3ff37d1bac96aad8302994fe39d875317f2a889a5c97c1e1ae03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-06T08:14:42.334 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:42 smithi087 podman[126251]: 2024-06-06 08:14:42.122061985 +0000 UTC m=+0.483876685 container attach aab94dd2973e3ff37d1bac96aad8302994fe39d875317f2a889a5c97c1e1ae03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T08:14:42.604 INFO:journalctl@ceph.osd.0.smithi087.stdout:Jun 06 08:14:42 smithi087 podman[126251]: 2024-06-06 08:14:42.334769967 +0000 UTC m=+0.696584678 container died aab94dd2973e3ff37d1bac96aad8302994fe39d875317f2a889a5c97c1e1ae03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-06T08:14:42.943 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-06T08:14:42.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:42.985 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-06T08:14:42.985 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-06T08:14:42.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.1 2024-06-06T08:14:43.305 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:43 smithi181 systemd[1]: Stopping Ceph osd.1 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:43.604 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1[41406]: 2024-06-06T08:14:43.337+0000 7f4e265db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:43.604 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1[41406]: 2024-06-06T08:14:43.337+0000 7f4e265db700 -1 osd.1 150 *** Got signal Terminated *** 2024-06-06T08:14:43.604 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1[41406]: 2024-06-06T08:14:43.337+0000 7f4e265db700 -1 osd.1 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:14:49.142 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:48 smithi181 podman[79210]: 2024-06-06 08:14:48.808913581 +0000 UTC m=+5.618360711 container died 1de2d1bcb2ec5a709905a0b105fa6edf7320b30c37156dcb2cf8fb3ce6bc50d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T08:14:49.464 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:49 smithi181 podman[79210]: 2024-06-06 08:14:49.238975517 +0000 UTC m=+6.048422639 container cleanup 1de2d1bcb2ec5a709905a0b105fa6edf7320b30c37156dcb2cf8fb3ce6bc50d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:14:49.464 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:49 smithi181 bash[79210]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1 2024-06-06T08:14:49.806 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:49 smithi181 podman[79227]: 2024-06-06 08:14:49.465546107 +0000 UTC m=+0.642353165 container remove 1de2d1bcb2ec5a709905a0b105fa6edf7320b30c37156dcb2cf8fb3ce6bc50d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:14:50.185 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:49 smithi181 podman[79313]: 2024-06-06 08:14:49.828528747 +0000 UTC m=+0.016826257 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:14:50.185 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:49 smithi181 podman[79313]: 2024-06-06 08:14:49.942025507 +0000 UTC m=+0.130322990 container create 2932bca5b03b0b88329591840050083e5ea7085427e6777158f58ccb4acf7281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-06T08:14:50.547 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:50 smithi181 podman[79313]: 2024-06-06 08:14:50.186405375 +0000 UTC m=+0.374702867 container init 2932bca5b03b0b88329591840050083e5ea7085427e6777158f58ccb4acf7281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-06T08:14:50.548 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:50 smithi181 podman[79313]: 2024-06-06 08:14:50.191438452 +0000 UTC m=+0.379735950 container start 2932bca5b03b0b88329591840050083e5ea7085427e6777158f58ccb4acf7281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:14:50.548 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:50 smithi181 podman[79313]: 2024-06-06 08:14:50.294774003 +0000 UTC m=+0.483071505 container attach 2932bca5b03b0b88329591840050083e5ea7085427e6777158f58ccb4acf7281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T08:14:50.806 INFO:journalctl@ceph.osd.1.smithi181.stdout:Jun 06 08:14:50 smithi181 podman[79313]: 2024-06-06 08:14:50.549139204 +0000 UTC m=+0.737436710 container died 2932bca5b03b0b88329591840050083e5ea7085427e6777158f58ccb4acf7281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-06T08:14:51.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-06T08:14:51.224 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:51.224 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-06T08:14:51.225 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-06T08:14:51.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.2 2024-06-06T08:14:51.509 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:51 smithi195 systemd[1]: Stopping Ceph osd.2 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:14:52.009 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T08:14:51.624+0000 7f7eb52dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:14:52.009 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T08:14:51.624+0000 7f7eb52dc700 -1 osd.2 150 *** Got signal Terminated *** 2024-06-06T08:14:52.010 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2[42168]: 2024-06-06T08:14:51.624+0000 7f7eb52dc700 -1 osd.2 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:14:57.467 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:57 smithi195 podman[82126]: 2024-06-06 08:14:57.150268022 +0000 UTC m=+5.731754237 container died 6f6233c306f3d5ee16f65c59d1e624d5665469f4130c67b7e79d15b78a2ed8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-06T08:14:57.759 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:57 smithi195 podman[82126]: 2024-06-06 08:14:57.574695752 +0000 UTC m=+6.156181949 container cleanup 6f6233c306f3d5ee16f65c59d1e624d5665469f4130c67b7e79d15b78a2ed8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:14:57.760 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:57 smithi195 bash[82126]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2 2024-06-06T08:14:58.131 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:57 smithi195 podman[82144]: 2024-06-06 08:14:57.818154237 +0000 UTC m=+0.651919630 container remove 6f6233c306f3d5ee16f65c59d1e624d5665469f4130c67b7e79d15b78a2ed8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-06T08:14:58.404 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.035081287 +0000 UTC m=+0.017371527 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:14:58.404 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.143977453 +0000 UTC m=+0.126267672 container create 098bc6dbee1f0330d75d9f7838bec2aa8ea5e542268c97ba2c91bc30ddec2edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T08:14:58.405 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.404787307 +0000 UTC m=+0.387077524 container init 098bc6dbee1f0330d75d9f7838bec2aa8ea5e542268c97ba2c91bc30ddec2edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-06T08:14:58.736 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.408943717 +0000 UTC m=+0.391233942 container start 098bc6dbee1f0330d75d9f7838bec2aa8ea5e542268c97ba2c91bc30ddec2edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T08:14:58.736 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.496739881 +0000 UTC m=+0.479030102 container attach 098bc6dbee1f0330d75d9f7838bec2aa8ea5e542268c97ba2c91bc30ddec2edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T08:14:59.009 INFO:journalctl@ceph.osd.2.smithi195.stdout:Jun 06 08:14:58 smithi195 podman[82232]: 2024-06-06 08:14:58.737506007 +0000 UTC m=+0.719796228 container died 098bc6dbee1f0330d75d9f7838bec2aa8ea5e542268c97ba2c91bc30ddec2edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T08:14:59.389 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-06T08:14:59.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:14:59.428 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-06T08:14:59.428 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-06T08:14:59.428 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.3 2024-06-06T08:14:59.745 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:14:59 smithi087 systemd[1]: Stopping Ceph osd.3 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:00.109 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:14:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:14:59.745+0000 7f055cdf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:00.109 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:14:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:14:59.745+0000 7f055cdf4700 -1 osd.3 150 *** Got signal Terminated *** 2024-06-06T08:15:00.109 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:14:59 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:14:59.745+0000 7f055cdf4700 -1 osd.3 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:01.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:01.225+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:01.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:01 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:01.554+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:02.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:02.234+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:02.609 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:02 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:15:02.174+0000 7f05553e0700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:39.686324+0000 front 2024-06-06T08:14:39.686297+0000 (oldest deadline 2024-06-06T08:15:01.386442+0000) 2024-06-06T08:15:02.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:02 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:02.543+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:03.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:03.199+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:03.609 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:03 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:15:03.191+0000 7f05553e0700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:39.686324+0000 front 2024-06-06T08:14:39.686297+0000 (oldest deadline 2024-06-06T08:15:01.386442+0000) 2024-06-06T08:15:03.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:03 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:03.549+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:04.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:04.174+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:04.609 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:04 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3[51325]: 2024-06-06T08:15:04.146+0000 7f05553e0700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:39.686324+0000 front 2024-06-06T08:14:39.686297+0000 (oldest deadline 2024-06-06T08:15:01.386442+0000) 2024-06-06T08:15:05.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:04 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:04.596+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:05.495 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:05 smithi087 podman[126371]: 2024-06-06 08:15:05.201403026 +0000 UTC m=+5.572050568 container died 243bd9af573675c4c14749dd015fc68982389a5d7af45297f8eb34f6a9efd889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-06T08:15:05.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:05.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:05.859 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:05 smithi087 podman[126371]: 2024-06-06 08:15:05.578965638 +0000 UTC m=+5.949613179 container cleanup 243bd9af573675c4c14749dd015fc68982389a5d7af45297f8eb34f6a9efd889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T08:15:05.859 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:05 smithi087 bash[126371]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3 2024-06-06T08:15:06.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:05 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:05.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:06.203 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:05 smithi087 podman[126387]: 2024-06-06 08:15:05.872173558 +0000 UTC m=+0.668009472 container remove 243bd9af573675c4c14749dd015fc68982389a5d7af45297f8eb34f6a9efd889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-06T08:15:06.499 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.107157958 +0000 UTC m=+0.017330460 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:06.500 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.23150317 +0000 UTC m=+0.141675659 container create 4f654537671f1f453b514c072567b615f42535e1de6ebf66f3b97858abf144c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T08:15:06.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:06.166+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:06.787 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.500850307 +0000 UTC m=+0.411022790 container init 4f654537671f1f453b514c072567b615f42535e1de6ebf66f3b97858abf144c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T08:15:06.787 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.504130762 +0000 UTC m=+0.414303259 container start 4f654537671f1f453b514c072567b615f42535e1de6ebf66f3b97858abf144c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T08:15:06.787 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.600835247 +0000 UTC m=+0.511007737 container attach 4f654537671f1f453b514c072567b615f42535e1de6ebf66f3b97858abf144c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-06T08:15:07.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:06 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:06.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:07.109 INFO:journalctl@ceph.osd.3.smithi087.stdout:Jun 06 08:15:06 smithi087 podman[126476]: 2024-06-06 08:15:06.788475888 +0000 UTC m=+0.698648387 container died 4f654537671f1f453b514c072567b615f42535e1de6ebf66f3b97858abf144c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:07.184 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:06.932+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:07.445 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:07 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:07.169+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:07.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-06T08:15:07.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:07.185+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:07.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:07.517 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-06T08:15:07.517 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-06T08:15:07.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.4 2024-06-06T08:15:07.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:07 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:07.585+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:07.807 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:07 smithi181 systemd[1]: Stopping Ceph osd.4 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:08.234 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:07.959+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:08.305 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:07 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T08:15:07.834+0000 7faee4573700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:08.306 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:07 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T08:15:07.834+0000 7faee4573700 -1 osd.4 150 *** Got signal Terminated *** 2024-06-06T08:15:08.306 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:07 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T08:15:07.834+0000 7faee4573700 -1 osd.4 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:08.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:08.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:08.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:08 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:08.218+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:09.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:08 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:08.604+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:09.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:08.940+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:09.552 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:09 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:09.170+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:09.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:09 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:09.273+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:09.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:09 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:09.553+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:10.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:09 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:09.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:10.257 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:09 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:09.953+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:10.257 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:09 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:09.804+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:10.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:10.257+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:10.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:10.257+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:10.559 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:10 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:10.217+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:10.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:10 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:10.542+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:10.858 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:10 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:10.560+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:11.227 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:10.945+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:11.227 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:10 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:10.807+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:11.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:11 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:11.228+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:11.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:11 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:11.228+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:11.582 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:11.216+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:11.806 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:11 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T08:15:11.588+0000 7faee0b67700 -1 osd.4 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.333026+0000 front 2024-06-06T08:14:46.333058+0000 (oldest deadline 2024-06-06T08:15:11.533848+0000) 2024-06-06T08:15:11.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:11 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:11.533+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:11.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:11 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:11.583+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:12.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:11 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:11.911+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:12.260 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:11 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:11.857+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:12.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:11 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:11.986+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:12.546 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:12 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:12.204+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:12.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:12 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:12.266+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:12.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:12 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:12.266+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:12.858 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:12 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:12.547+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:13.012 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:12 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4[48038]: 2024-06-06T08:15:12.598+0000 7faee0b67700 -1 osd.4 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.333026+0000 front 2024-06-06T08:14:46.333058+0000 (oldest deadline 2024-06-06T08:15:11.533848+0000) 2024-06-06T08:15:13.013 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:12 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:12.578+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:13.222 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:12 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:12.871+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:13.223 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:12 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:12.885+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:13.268 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:13 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:13.013+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:13.500 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:13 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:13.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:13.500 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:13 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:13.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:13.514 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:13.209+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:13.515 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:13.209+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:13.555 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:13 smithi181 podman[79436]: 2024-06-06 08:15:13.269940799 +0000 UTC m=+5.551323530 container stop d594e75dd16eeec80073277f8af171f33d957640736c6bf95ac5050a6ab00f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:15:13.556 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:13 smithi181 podman[79436]: 2024-06-06 08:15:13.299008751 +0000 UTC m=+5.580391502 container died d594e75dd16eeec80073277f8af171f33d957640736c6bf95ac5050a6ab00f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-06T08:15:13.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:13 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:13.515+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:13.903 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:13 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:13.578+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:13.904 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:13 smithi181 podman[79436]: 2024-06-06 08:15:13.684982328 +0000 UTC m=+5.966365113 container cleanup d594e75dd16eeec80073277f8af171f33d957640736c6bf95ac5050a6ab00f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-06T08:15:13.904 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:13 smithi181 bash[79436]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4 2024-06-06T08:15:14.205 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:13 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:13.888+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:14.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:13 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:13.880+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:14.306 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:13 smithi181 podman[79454]: 2024-06-06 08:15:13.902887832 +0000 UTC m=+0.627161588 container remove d594e75dd16eeec80073277f8af171f33d957640736c6bf95ac5050a6ab00f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-06T08:15:14.307 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:14 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:14.023+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:14.482 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:14.188+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:14.482 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:14.188+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:14.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:14 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:14.206+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:14.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:14 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:14.206+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:14.646 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:14 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:14.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:14.647 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79539]: 2024-06-06 08:15:14.245405719 +0000 UTC m=+0.017272417 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:14.647 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79539]: 2024-06-06 08:15:14.353929658 +0000 UTC m=+0.125796350 container create 15f553c665f474b36d245dd2cfa1b791f0237be0b84c075a472583b8f46e46fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:15:14.647 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79539]: 2024-06-06 08:15:14.548609234 +0000 UTC m=+0.320475941 container init 15f553c665f474b36d245dd2cfa1b791f0237be0b84c075a472583b8f46e46fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:15:14.647 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79539]: 2024-06-06 08:15:14.552072958 +0000 UTC m=+0.323939660 container start 15f553c665f474b36d245dd2cfa1b791f0237be0b84c075a472583b8f46e46fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:15:14.648 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79539]: 2024-06-06 08:15:14.648308635 +0000 UTC m=+0.420175381 container attach 15f553c665f474b36d245dd2cfa1b791f0237be0b84c075a472583b8f46e46fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:15:14.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:14 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:14.481+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:15.037 INFO:journalctl@ceph.osd.4.smithi181.stdout:Jun 06 08:15:14 smithi181 podman[79565]: 2024-06-06 08:15:14.870333352 +0000 UTC m=+0.017938439 container died 15f553c665f474b36d245dd2cfa1b791f0237be0b84c075a472583b8f46e46fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-06T08:15:15.216 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:14 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:14.873+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:15.216 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:14 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:14.835+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:15.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:15 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:15.038+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:15.449 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-06T08:15:15.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:15.489 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-06T08:15:15.489 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-06T08:15:15.489 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.5 2024-06-06T08:15:15.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:15.216+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:15.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:15.216+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:15.512 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:15 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:15.213+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:15.513 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:15 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:15.213+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:15.803 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:15 smithi195 systemd[1]: Stopping Ceph osd.5 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:15.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:15 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:15.500+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:15.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:15 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:15.500+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:15.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:15 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:15.513+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:16.167 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:15.827+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:16.168 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:15.803+0000 7f543103d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:16.168 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:15.803+0000 7f543103d700 -1 osd.5 150 *** Got signal Terminated *** 2024-06-06T08:15:16.168 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:15.803+0000 7f543103d700 -1 osd.5 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:16.168 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:15 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:15.827+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:16.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:16 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:15.999+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:16.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:16 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:16.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:16.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:16 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:16.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:16.538 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:16.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:16.538 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:16.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:16.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:16 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:16.492+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:16.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:16 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:16.492+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:16.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:16.538+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:17.217 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:16 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:16.798+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:17.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:16 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:16.784+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:17.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:16 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:16.952+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:17.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:17 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:17.217+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:17.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:17 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:17.217+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:17.588 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:17.202+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:17.588 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:17.202+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:17.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:17 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:17.517+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:17.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:17 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:17.517+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:17.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:17 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:17.588+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:18.244 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:17 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:17.800+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:18.245 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:17 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:17.805+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:18.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:17 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:17.950+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:18.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:18.245+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:18.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:18.245+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:18.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:18.245+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:18.608 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:18.241+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:18.608 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:18.241+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:18.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:18.241+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:18.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:18 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:18.480+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:18.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:18 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:18.480+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:18.861 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:18.609+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:18.861 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:18.609+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:19.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:18.832+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:19.219 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:18.776+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:19.219 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:18 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:18.776+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:53.921029+0000 front 2024-06-06T08:14:53.921055+0000 (oldest deadline 2024-06-06T08:15:18.021156+0000) 2024-06-06T08:15:19.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:18 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:18.936+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:19.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:19.218+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:19.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:19.218+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:19.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:19.218+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:19.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:19.271+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:19.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:19.271+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:19.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:19.271+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:19.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:19 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:19.497+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:19.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:19 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:19.497+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:19.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:19 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:19.497+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:20.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:19.621+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:20.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:19 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:19.621+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:20.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:19.848+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:20.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:19.848+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:20.260 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:19.783+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:20.260 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:19 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:19.783+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:53.921029+0000 front 2024-06-06T08:14:53.921055+0000 (oldest deadline 2024-06-06T08:15:18.021156+0000) 2024-06-06T08:15:20.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:19 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:19.933+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:20.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:19 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:19.933+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:20.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:20.228+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:20.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:20.228+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:20.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:20.228+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:20.747 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:20.265+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:20.747 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:20.265+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:20.747 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:20.265+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:20.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:20 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:20.545+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:20.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:20 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:20.545+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:20.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:20 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:20.545+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:21.009 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:20.747+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:47.618991+0000 front 2024-06-06T08:14:47.618963+0000 (oldest deadline 2024-06-06T08:15:09.319082+0000) 2024-06-06T08:15:21.009 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5[49120]: 2024-06-06T08:15:20.747+0000 7f5429629700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:53.921029+0000 front 2024-06-06T08:14:53.921055+0000 (oldest deadline 2024-06-06T08:15:18.021156+0000) 2024-06-06T08:15:21.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:20.866+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:21.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:20 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:20.866+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:21.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:20.618+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:21.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:20 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:20.618+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:21.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:20 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:20.916+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:21.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:20 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:20.916+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:21.538 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:21 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:21.262+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:21.538 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:21 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:21.262+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:21.538 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:21 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:21.262+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:21.538 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:21 smithi195 podman[82348]: 2024-06-06 08:15:21.272373007 +0000 UTC m=+5.584747341 container died 6546d8a218fd2106aa58fbbbe8bab428190e454a63a8cfd3fbc37106c0d9b3e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T08:15:21.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:21.265+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:21.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:21.265+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:21.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:21.265+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:21.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:21 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:21.549+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:21.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:21 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:21.549+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:21.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:21 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:21.549+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:21.911 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:21 smithi195 podman[82348]: 2024-06-06 08:15:21.629175233 +0000 UTC m=+5.941549565 container cleanup 6546d8a218fd2106aa58fbbbe8bab428190e454a63a8cfd3fbc37106c0d9b3e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T08:15:21.912 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:21 smithi195 bash[82348]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5 2024-06-06T08:15:22.110 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:21.634+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:22.111 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:21.634+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:22.222 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:21 smithi195 podman[82363]: 2024-06-06 08:15:21.922290129 +0000 UTC m=+0.647161420 container remove 6546d8a218fd2106aa58fbbbe8bab428190e454a63a8cfd3fbc37106c0d9b3e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T08:15:22.222 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:21 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:21.912+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:22.222 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:21 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:21.912+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:22.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:21 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:21.898+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:22.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:21 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:21.898+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:22.499 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:22 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:22.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:22.500 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:22 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:22.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:22.500 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:22 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:22.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:22.500 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.141697625 +0000 UTC m=+0.017570581 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:22.500 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.265023726 +0000 UTC m=+0.140896712 container create 208a1f146a410267c0cf7a1ea33216c444f255e7b204491f3e5c503a409d2148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T08:15:22.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:22 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:22.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:22.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:22 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:22.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:22.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:22 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:22.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:22.759 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.500796192 +0000 UTC m=+0.376669127 container init 208a1f146a410267c0cf7a1ea33216c444f255e7b204491f3e5c503a409d2148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T08:15:22.760 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.504022634 +0000 UTC m=+0.379895577 container start 208a1f146a410267c0cf7a1ea33216c444f255e7b204491f3e5c503a409d2148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T08:15:22.760 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.592560522 +0000 UTC m=+0.468433468 container attach 208a1f146a410267c0cf7a1ea33216c444f255e7b204491f3e5c503a409d2148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T08:15:22.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:22 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:22.554+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:22.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:22 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:22.554+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:22.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:22 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:22.554+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:23.086 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:22 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:22.923+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:23.087 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:22 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:22.923+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:23.087 INFO:journalctl@ceph.osd.5.smithi195.stdout:Jun 06 08:15:22 smithi195 podman[82454]: 2024-06-06 08:15:22.787867504 +0000 UTC m=+0.663740461 container died 208a1f146a410267c0cf7a1ea33216c444f255e7b204491f3e5c503a409d2148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T08:15:23.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:22 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:22.663+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:23.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:22 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:22.663+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:23.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:22 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:22.940+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:23.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:22 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:22.940+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:23.383 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:23.239+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:23.383 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:23.239+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:23.383 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:23.239+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:23.412 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:23.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:23.412 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:23.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:23.412 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:23.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:23.433 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-06T08:15:23.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:23.512 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-06T08:15:23.512 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-06T08:15:23.512 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.6 2024-06-06T08:15:23.681 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 systemd[1]: Stopping Ceph osd.6 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:23.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:23 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:23.512+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:23.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:23 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:23.512+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:23.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:23 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:23.512+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:24.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:23.680+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:24.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:23.680+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:24.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:23.831+0000 7f1b5fbc5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:24.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:23.831+0000 7f1b5fbc5700 -1 osd.6 150 *** Got signal Terminated *** 2024-06-06T08:15:24.110 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:23 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:23.831+0000 7f1b5fbc5700 -1 osd.6 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:24.226 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:23.961+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:24.226 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:23 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:23.961+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:24.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:23 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:23.913+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:24.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:23 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:23.913+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:24.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:24 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:24.226+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:24.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:24 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:24.226+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:24.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:24 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:24.226+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:24.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:24 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:24.285+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:24.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:24 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:24.285+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:24.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:24 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:24.285+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:24.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:24.498+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:24.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:24.498+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:24.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:24.498+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:25.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:24 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:24.714+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:25.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:24 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:24.714+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:25.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:24 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:24.973+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:25.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:24 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:24.973+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:25.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:25.215+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:25.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:25.215+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:25.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:25.215+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:25.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:24.905+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:25.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:24 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:24.905+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:25.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:25.253+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:25.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:25.253+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:25.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:25.253+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:25.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:25.534+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:25.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:25.534+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:25.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:25.534+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:25.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:25.534+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:26.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:25.707+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:26.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:25 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:25.707+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:26.234 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:25.952+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:26.235 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:25.952+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:26.235 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:25 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:25.952+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:26.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:25.856+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:26.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:25.856+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:26.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:26.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:26.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:26.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:26.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:26.235+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:26.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:26.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:26.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:26.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:26.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:26.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:26.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:26.508+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:26.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:26.508+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:26.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:26.508+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:26.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:26.508+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:27.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:26.745+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:27.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:26.745+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:27.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:26 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:26.745+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.455514+0000 front 2024-06-06T08:15:00.455540+0000 (oldest deadline 2024-06-06T08:15:25.755760+0000) 2024-06-06T08:15:27.256 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:26.993+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:27.256 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:26.993+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:27.256 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:26 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:26.993+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:27.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:26.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:27.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:26 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:26.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:27.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:27.257+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:27.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:27.257+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:27.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:27.257+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:27.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:27.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:27.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:27.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:27.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:27.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:27.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:27.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:27.805 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:27.536+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:27.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:27.536+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:27.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:27.536+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:27.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:27.536+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:28.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:27.778+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:28.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:27.778+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:28.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:27 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:27.778+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.455514+0000 front 2024-06-06T08:15:00.455540+0000 (oldest deadline 2024-06-06T08:15:25.755760+0000) 2024-06-06T08:15:28.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:27.955+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:28.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:27.955+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:28.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:27 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:27.955+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:28.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:27.861+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:28.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:27 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:27.862+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:28.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:28.267+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:28.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:28.267+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:28.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:28.267+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:28.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:28.267+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:28.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:28.291+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:28.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:28.291+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:28.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:28.291+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:29.055 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:28.830+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:29.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:28.830+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:29.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:28.830+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:29.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:28.584+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:29.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:28.584+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:29.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:28.584+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:29.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:28 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:28.584+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:29.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:28.809+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:43.651661+0000 front 2024-06-06T08:14:43.651557+0000 (oldest deadline 2024-06-06T08:15:09.551531+0000) 2024-06-06T08:15:29.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:28.809+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.753330+0000 front 2024-06-06T08:14:54.753352+0000 (oldest deadline 2024-06-06T08:15:18.154123+0000) 2024-06-06T08:15:29.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:28 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6[57810]: 2024-06-06T08:15:28.809+0000 7f1b581b1700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.455514+0000 front 2024-06-06T08:15:00.455540+0000 (oldest deadline 2024-06-06T08:15:25.755760+0000) 2024-06-06T08:15:29.253 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:28.918+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:29.254 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:28.918+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:29.254 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:28 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:28.918+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:29.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:29.255+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:29.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:29.255+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:29.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:29.255+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:29.573 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:29 smithi087 podman[126591]: 2024-06-06 08:15:29.271554529 +0000 UTC m=+5.562232607 container died 53b1e07a4526244a92236dcdb1bbbc94da3d375a6e01ca625dd97f0db36abc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:15:29.573 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:29 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:29.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:29.573 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:29 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:29.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:29.574 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:29 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:29.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:29.574 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:29 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:29.231+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:29.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:29 smithi087 podman[126591]: 2024-06-06 08:15:29.673844818 +0000 UTC m=+5.964522945 container cleanup 53b1e07a4526244a92236dcdb1bbbc94da3d375a6e01ca625dd97f0db36abc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:15:29.859 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:29 smithi087 bash[126591]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6 2024-06-06T08:15:30.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:29.825+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:30.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:29.825+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:30.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:29.825+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:30.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:29.602+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:30.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:29.602+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:30.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:29.602+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:30.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:29 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:29.602+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:30.204 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:29 smithi087 podman[126609]: 2024-06-06 08:15:29.908372936 +0000 UTC m=+0.633848177 container remove 53b1e07a4526244a92236dcdb1bbbc94da3d375a6e01ca625dd97f0db36abc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-06-06T08:15:30.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:29.939+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:30.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:29.939+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:30.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:29 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:29.939+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:30.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:30.217+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:30.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:30.217+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:30.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:30.217+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:30.561 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.134095781 +0000 UTC m=+0.017500362 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:30.561 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.25924932 +0000 UTC m=+0.142653894 container create 937879252daf80c5afa0ddfd638c0107c2aae11d23d700aa50f25e618dbc67c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:15:30.561 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.562380307 +0000 UTC m=+0.445784880 container init 937879252daf80c5afa0ddfd638c0107c2aae11d23d700aa50f25e618dbc67c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True) 2024-06-06T08:15:30.562 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:30 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:30.205+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:30.562 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:30 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:30.205+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:30.562 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:30 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:30.205+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:30.562 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:30 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:30.205+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:30.854 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.565904483 +0000 UTC m=+0.449309050 container start 937879252daf80c5afa0ddfd638c0107c2aae11d23d700aa50f25e618dbc67c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T08:15:30.855 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.653928653 +0000 UTC m=+0.537333240 container attach 937879252daf80c5afa0ddfd638c0107c2aae11d23d700aa50f25e618dbc67c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:31.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:30.853+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:31.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:30.853+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:31.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:30.853+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:31.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:30.623+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:31.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:30.623+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:31.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:30.623+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:31.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:30 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:30.623+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:31.109 INFO:journalctl@ceph.osd.6.smithi087.stdout:Jun 06 08:15:30 smithi087 podman[126698]: 2024-06-06 08:15:30.856337551 +0000 UTC m=+0.739742131 container died 937879252daf80c5afa0ddfd638c0107c2aae11d23d700aa50f25e618dbc67c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:15:31.258 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:30.903+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:31.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:30.903+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:31.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:30 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:30.903+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:31.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:31.258+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:31.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:31.258+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:31.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:31.258+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:31.514 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:31 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:31.237+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:31.515 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:31 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:31.237+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:31.515 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:31 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:31.238+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:31.515 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:31 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:31.238+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:31.539 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-06T08:15:31.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:31.578 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-06T08:15:31.579 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-06T08:15:31.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.7 2024-06-06T08:15:31.887 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 systemd[1]: Stopping Ceph osd.7 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:31.887 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.837+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:31.887 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.837+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:31.887 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.837+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:31.887 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:31.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:31.888 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:31.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:31.888 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:31.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:31.888 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:31.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:32.243 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:31.863+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:32.244 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:31.863+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:32.244 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:31 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:31.863+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:32.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.887+0000 7f08a2e63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:32.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.887+0000 7f08a2e63700 -1 osd.7 150 *** Got signal Terminated *** 2024-06-06T08:15:32.306 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:31.887+0000 7f08a2e63700 -1 osd.7 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:32.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:32.244+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:32.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:32.244+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:32.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:32.244+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:32.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:32 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:32.275+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:32.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:32 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:32.275+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:32.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:32 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:32.275+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:32.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:32 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:32.275+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:33.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:32.869+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:33.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:32.869+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:33.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:32.869+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:33.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:32.650+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:33.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:32.650+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:33.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:32.650+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:33.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:32 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:32.650+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:33.239 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:32.827+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:33.239 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:32.827+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:33.239 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:32 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:32.827+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:33.508 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:33.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:33.508 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:33.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:33.508 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:33.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:33.608 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:33 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:33.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:33.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:33 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:33.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:33.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:33 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:33.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:33.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:33 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:33.247+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:34.055 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:33.899+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:34.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:33.899+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:34.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:33.899+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:34.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:33.661+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:34.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:33.661+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:34.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:33.661+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:34.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:33 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:33.661+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:34.222 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:33.803+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:34.222 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:33.803+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:34.223 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:33 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:33.803+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:34.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:34.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:34.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:34.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:34.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:34.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:34.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:34.222+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:34.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:34.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:34.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:34.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:34.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:34.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:34.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:34 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:34.220+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:35.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:34.758+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:35.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:34.758+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:35.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:34.758+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:35.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:34 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:34.758+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:35.055 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:34.885+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:35.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:34.885+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:35.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:34.885+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:35.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:34.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:35.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:34.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:35.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:34.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:35.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:34 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:34.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:35.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:35.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:35.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:35.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:35.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:35.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:35.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:35.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:35.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:35 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:35.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:35.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:35.269+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:35.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:35.269+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:35.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:35.269+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:35.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:35.269+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:36.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:35.759+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:36.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:35.759+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:36.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:35.759+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:36.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:35 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:35.759+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:36.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:35.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:45.849557+0000 front 2024-06-06T08:14:45.849588+0000 (oldest deadline 2024-06-06T08:15:11.149737+0000) 2024-06-06T08:15:36.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:35.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.751939+0000 front 2024-06-06T08:14:55.751819+0000 (oldest deadline 2024-06-06T08:15:19.251792+0000) 2024-06-06T08:15:36.056 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:35.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:03.352894+0000 front 2024-06-06T08:15:03.352855+0000 (oldest deadline 2024-06-06T08:15:28.652844+0000) 2024-06-06T08:15:36.057 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7[54686]: 2024-06-06T08:15:35.900+0000 7f089b44f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.054548+0000 front 2024-06-06T08:15:12.054576+0000 (oldest deadline 2024-06-06T08:15:34.954759+0000) 2024-06-06T08:15:36.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:35.719+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:36.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:35.719+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:36.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:35.719+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:36.058 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:35.719+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:36.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:36.253+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:36.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:36.253+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:36.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:36.253+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:36.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:36.253+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:36.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:36 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:36.157+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:36.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:36 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:36.157+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:36.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:36 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:36.157+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:36.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:36 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:36.157+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:36.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:36 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:36.157+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:37.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:36 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:36.684+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:37.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:36 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:36.684+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:37.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:36 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:36.684+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:37.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:36 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:36.684+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:37.208 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:36.782+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:37.208 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:36.782+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:37.209 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:36.782+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:37.209 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:36 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:36.782+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:37.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:37.208+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:37.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:37.208+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:37.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:37.208+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:37.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:37.208+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:37.596 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:37 smithi181 podman[79644]: 2024-06-06 08:15:37.31205843 +0000 UTC m=+5.542783152 container died 8a1dab867c0eba59186a9e6f59683909f1224f43a742aed6f1000c34869eb6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T08:15:37.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:37 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:37.118+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:37.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:37 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:37.118+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:37.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:37 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:37.118+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:37.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:37 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:37.118+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:37.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:37 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:37.118+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:37.988 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:37 smithi181 podman[79644]: 2024-06-06 08:15:37.679583788 +0000 UTC m=+5.910308525 container cleanup 8a1dab867c0eba59186a9e6f59683909f1224f43a742aed6f1000c34869eb6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T08:15:37.988 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:37 smithi181 bash[79644]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7 2024-06-06T08:15:37.988 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:37 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:37.695+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:37.988 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:37 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:37.695+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:37.988 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:37 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:37.695+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:37.989 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:37 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:37.695+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:38.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:37.756+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:38.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:37.756+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:38.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:37.756+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:38.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:37 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:37.756+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:38.305 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:37 smithi181 podman[79671]: 2024-06-06 08:15:37.989185192 +0000 UTC m=+0.664437447 container remove 8a1dab867c0eba59186a9e6f59683909f1224f43a742aed6f1000c34869eb6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T08:15:38.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:38.223+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:38.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:38.223+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:38.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:38.223+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:38.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:38.223+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:38.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:38 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:38.141+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:38.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:38 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:38.141+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:38.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:38 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:38.141+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:38.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:38 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:38.141+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:38.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:38 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:38.141+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:38.625 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.256232744 +0000 UTC m=+0.017272388 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:38.625 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.373555583 +0000 UTC m=+0.134595224 container create 98a64e90721a2a7af4b925b435c08e4843b221df22595a4641f9d3ef5da12c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T08:15:38.625 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.626680861 +0000 UTC m=+0.387720511 container init 98a64e90721a2a7af4b925b435c08e4843b221df22595a4641f9d3ef5da12c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-06T08:15:38.920 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.62994288 +0000 UTC m=+0.390982526 container start 98a64e90721a2a7af4b925b435c08e4843b221df22595a4641f9d3ef5da12c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T08:15:38.920 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.709471385 +0000 UTC m=+0.470511029 container attach 98a64e90721a2a7af4b925b435c08e4843b221df22595a4641f9d3ef5da12c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T08:15:38.920 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:38 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:38.685+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:38.920 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:38 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:38.685+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:38.921 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:38 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:38.685+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:38.921 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:38 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:38.685+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:39.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:38.714+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:39.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:38.714+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:39.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:38.714+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:39.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:38 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:38.714+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:39.303 INFO:journalctl@ceph.osd.7.smithi181.stdout:Jun 06 08:15:38 smithi181 podman[79761]: 2024-06-06 08:15:38.921336184 +0000 UTC m=+0.682375826 container died 98a64e90721a2a7af4b925b435c08e4843b221df22595a4641f9d3ef5da12c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T08:15:39.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:39.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:39.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:39.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:39.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:39.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:39.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:39.239+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:39.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-06T08:15:39.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:39.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:39.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:39.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:39.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:39.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:39.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:39.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:39.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:39 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:39.191+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:39.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:39.629 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-06T08:15:39.630 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-06T08:15:39.630 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.8 2024-06-06T08:15:39.948 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 systemd[1]: Stopping Ceph osd.8 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:39.949 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.699+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:39.949 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.699+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:39.949 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.699+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:39.949 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.699+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:40.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:39.659+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:40.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:39.659+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:40.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:39.659+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:40.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:39.659+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:40.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:39.659+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:40.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.949+0000 7f28c9dd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:40.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.949+0000 7f28c9dd7700 -1 osd.8 150 *** Got signal Terminated *** 2024-06-06T08:15:40.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:39 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:39.949+0000 7f28c9dd7700 -1 osd.8 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:40.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:40.178+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:40.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:40.178+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:40.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:40.178+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:40.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:40.178+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:40.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:40 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:40.178+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:40.736 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:40.283+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:40.737 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:40.283+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:40.737 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:40.283+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:40.737 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:40.283+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:41.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:40.737+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:41.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:40.737+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:41.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:40.737+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:41.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:40 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:40.737+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:41.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:40 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:40.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:41.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:40 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:40.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:41.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:40 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:40.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:41.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:40 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:40.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:41.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:40 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:40.699+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:41.589 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:41.180+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:41.590 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:41.180+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:41.590 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:41.180+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:41.590 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:41.180+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:41.590 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:41 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:41.180+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:41.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:41.271+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:41.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:41.271+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:41.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:41.271+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:41.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:41.271+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:42.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:41 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:41.686+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:42.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:41 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:41.686+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:42.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:41 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:41.686+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:42.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:41 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:41.686+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:42.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:41 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:41.686+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:42.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:41.771+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:42.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:41.771+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:42.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:41.771+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:42.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:41 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:41.771+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:42.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:42.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:42.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:42.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:42.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:42.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:42 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:42.197+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:42.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:42.315+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:42.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:42.315+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:42.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:42.315+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:42.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:42.315+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:43.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:42 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:42.679+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:43.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:42 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:42.679+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:43.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:42 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:42.679+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:43.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:42 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:42.679+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:43.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:42 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:42.679+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:43.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:42.778+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:43.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:42.778+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:43.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:42.778+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:43.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:42 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:42.779+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:43.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:43.611 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:43.611 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:43.611 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:43.611 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:43.611 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:43 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:43.201+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:43.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:43.329+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:43.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:43.329+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:43.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:43.329+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:43.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:43.329+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:44.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:44.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:44.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:44.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:44.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:44.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:43 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:43.642+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:44.259 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:43.775+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:44.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:43.775+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:44.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:43.775+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:44.260 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:43 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:43.775+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:44.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:44.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:44.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:44.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:44.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:44.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:44 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:44.227+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:44.732 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:44.354+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:44.733 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:44.354+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:44.733 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:44.354+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:44.733 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:44.354+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:44.733 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:44.354+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:45.009 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:44.732+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:36.221643+0000 front 2024-06-06T08:14:36.221678+0000 (oldest deadline 2024-06-06T08:15:06.222332+0000) 2024-06-06T08:15:45.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:44.732+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.424134+0000 front 2024-06-06T08:14:54.424104+0000 (oldest deadline 2024-06-06T08:15:19.124292+0000) 2024-06-06T08:15:45.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:44.732+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:02.025803+0000 front 2024-06-06T08:15:02.025770+0000 (oldest deadline 2024-06-06T08:15:25.525831+0000) 2024-06-06T08:15:45.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:44.732+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.827106+0000 front 2024-06-06T08:15:10.827048+0000 (oldest deadline 2024-06-06T08:15:34.327134+0000) 2024-06-06T08:15:45.010 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8[55638]: 2024-06-06T08:15:44.732+0000 7f28c63cb700 -1 osd.8 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:20.228252+0000 front 2024-06-06T08:15:20.228233+0000 (oldest deadline 2024-06-06T08:15:44.328489+0000) 2024-06-06T08:15:45.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:45.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:45.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:45.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:45.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:45.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:44 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:44.630+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:45.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:45.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:45.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:45.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:45.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:45.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:45 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:45.198+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:45.759 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:45 smithi195 podman[82571]: 2024-06-06 08:15:45.554761739 +0000 UTC m=+5.718752918 container died 390b739675e029e3e802d94a9c6dc61ce1a27d835d3ec34b1286ded7b846623e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:45.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:45 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:45.365+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:45.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:45 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:45.365+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:45.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:45 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:45.365+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:45.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:45 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:45.365+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:45.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:45 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:45.365+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:46.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:46.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:46.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:46.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:46.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:46.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:45 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:45.599+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:46.241 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:45 smithi195 podman[82571]: 2024-06-06 08:15:45.95760984 +0000 UTC m=+6.121601019 container cleanup 390b739675e029e3e802d94a9c6dc61ce1a27d835d3ec34b1286ded7b846623e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T08:15:46.241 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:45 smithi195 bash[82571]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8 2024-06-06T08:15:46.509 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82586]: 2024-06-06 08:15:46.242721847 +0000 UTC m=+0.673148486 container remove 390b739675e029e3e802d94a9c6dc61ce1a27d835d3ec34b1286ded7b846623e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T08:15:46.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:46.343+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:46.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:46.343+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:46.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:46.343+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:46.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:46.343+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:46.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:46 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:46.344+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:46.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:46.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:46.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:46.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:46.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:46.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:46 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:46.149+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:46.837 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82676]: 2024-06-06 08:15:46.460672167 +0000 UTC m=+0.017206515 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:46.837 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82676]: 2024-06-06 08:15:46.568814784 +0000 UTC m=+0.125349132 container create 0c0f584243d26eaefe9f8081fa4ad299d7c52423263be9fa27512be7d62f6716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-06T08:15:47.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:47.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:47.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:47.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:47.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:47.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:46 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:46.565+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:47.130 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82676]: 2024-06-06 08:15:46.837844573 +0000 UTC m=+0.394378924 container init 0c0f584243d26eaefe9f8081fa4ad299d7c52423263be9fa27512be7d62f6716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-06-06T08:15:47.130 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82676]: 2024-06-06 08:15:46.841646137 +0000 UTC m=+0.398180496 container start 0c0f584243d26eaefe9f8081fa4ad299d7c52423263be9fa27512be7d62f6716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-06T08:15:47.130 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:46 smithi195 podman[82676]: 2024-06-06 08:15:46.912804315 +0000 UTC m=+0.469338687 container attach 0c0f584243d26eaefe9f8081fa4ad299d7c52423263be9fa27512be7d62f6716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:47.431 INFO:journalctl@ceph.osd.8.smithi195.stdout:Jun 06 08:15:47 smithi195 podman[82676]: 2024-06-06 08:15:47.131461973 +0000 UTC m=+0.687996329 container died 0c0f584243d26eaefe9f8081fa4ad299d7c52423263be9fa27512be7d62f6716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-06T08:15:47.431 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:47.297+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:47.432 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:47.297+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:47.432 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:47.297+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:47.432 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:47.297+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:47.432 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:47.297+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:47.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:47.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:47.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:47.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:47.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:47.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:47.155+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:47.754 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-06T08:15:47.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:47.792 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-06T08:15:47.792 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-06T08:15:47.792 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.9 2024-06-06T08:15:48.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:48.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:48.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:48.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:48.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:48.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:47 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:47.586+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:48.109 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:47 smithi087 systemd[1]: Stopping Ceph osd.9 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:48.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.116+0000 7f37cefc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:48.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.116+0000 7f37cefc4700 -1 osd.9 150 *** Got signal Terminated *** 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.116+0000 7f37cefc4700 -1 osd.9 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:48.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:48 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:48.137+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:48.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:48 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:48.307+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:48.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:48 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:48.307+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:48.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:48 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:48.307+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:48.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:48 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:48.307+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:48.761 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:48 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:48.307+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:49.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:49.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:49.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:49.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:49.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:49.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:48 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:48.561+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:49.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:49.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:49.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:49.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:49.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:49.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:49 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:49.144+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:49.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:49 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:49.346+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:49.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:49 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:49.346+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:49.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:49 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:49.346+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:49.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:49 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:49.346+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:49.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:49 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:49.346+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:50.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:50.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:50.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:50.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:50.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:50.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:49 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:49.601+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:50.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:50.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:50.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:50.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:50.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:50.610 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:50 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:50.120+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:50.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:50 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:50.334+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:50.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:50 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:50.334+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:50.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:50 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:50.334+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:50.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:50 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:50.334+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:50.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:50 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:50.334+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:51.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:51.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:50 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:50.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:51.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:51.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:51.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:51.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:51.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:51.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:51 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:51.096+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:51.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:51.288+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:51.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:51.288+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:51.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:51.288+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:51.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:51.288+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:51.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:51.288+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:52.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:51 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:51.563+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:52.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:52.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:52.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:52.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:52.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:52.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:52 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:52.064+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:52.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:52.286+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:52.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:52.286+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:52.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:52.286+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:52.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:52.286+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:52.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:52 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:52.286+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:52.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:52.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:52.806 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:52.807 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:52.807 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:52.807 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:52.807 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:52 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:52.540+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:53.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:37.712920+0000 front 2024-06-06T08:14:37.713005+0000 (oldest deadline 2024-06-06T08:15:06.513600+0000) 2024-06-06T08:15:53.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:46.514405+0000 front 2024-06-06T08:14:46.514174+0000 (oldest deadline 2024-06-06T08:15:12.414118+0000) 2024-06-06T08:15:53.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:55.315395+0000 front 2024-06-06T08:14:55.315523+0000 (oldest deadline 2024-06-06T08:15:17.615387+0000) 2024-06-06T08:15:53.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:15:00.516522+0000 front 2024-06-06T08:15:00.516617+0000 (oldest deadline 2024-06-06T08:15:26.416646+0000) 2024-06-06T08:15:53.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:10.418216+0000 front 2024-06-06T08:15:10.418194+0000 (oldest deadline 2024-06-06T08:15:35.118105+0000) 2024-06-06T08:15:53.360 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9[64679]: 2024-06-06T08:15:53.043+0000 7f37cb5b8700 -1 osd.9 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:19.119798+0000 front 2024-06-06T08:15:19.119730+0000 (oldest deadline 2024-06-06T08:15:42.019446+0000) 2024-06-06T08:15:53.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:53.243+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:53.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:53.243+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:53.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:53.243+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:53.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:53.243+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:53.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:53 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:53.243+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:53.859 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:53 smithi087 podman[126805]: 2024-06-06 08:15:53.725745909 +0000 UTC m=+5.733563386 container died 968e02a61d2d4a2ad7bb652dbc60cde81b27b2b110e9700243aed31b43fc60dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:15:54.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:54.058 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:54.058 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:54.058 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:54.058 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:54.059 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:54.059 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:53 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:53.589+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:54.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:54 smithi087 podman[126805]: 2024-06-06 08:15:54.167923612 +0000 UTC m=+6.175741079 container cleanup 968e02a61d2d4a2ad7bb652dbc60cde81b27b2b110e9700243aed31b43fc60dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-06T08:15:54.359 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:54 smithi087 bash[126805]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9 2024-06-06T08:15:54.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:54.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:54.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:54.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:54.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:54.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:54 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:54.194+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:15:54.789 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:54 smithi087 podman[126827]: 2024-06-06 08:15:54.452955054 +0000 UTC m=+0.730533433 container remove 968e02a61d2d4a2ad7bb652dbc60cde81b27b2b110e9700243aed31b43fc60dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:15:55.055 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:54 smithi087 podman[126922]: 2024-06-06 08:15:54.693091747 +0000 UTC m=+0.016925130 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:15:55.055 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:54 smithi087 podman[126922]: 2024-06-06 08:15:54.804047813 +0000 UTC m=+0.127881195 container create aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T08:15:55.055 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 podman[126922]: 2024-06-06 08:15:55.056320444 +0000 UTC m=+0.380153825 container init aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:55.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:54 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:54.588+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:55.351 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 podman[126922]: 2024-06-06 08:15:55.059713829 +0000 UTC m=+0.383547190 container start aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:15:55.352 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 podman[126922]: 2024-06-06 08:15:55.139626168 +0000 UTC m=+0.463459540 container attach aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:55.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:55.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:55.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:55.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:55.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:55.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:55.198+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:15:55.609 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 podman[126922]: 2024-06-06 08:15:55.352255619 +0000 UTC m=+0.676088998 container died aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-06T08:15:55.882 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 podman[126948]: 2024-06-06 08:15:55.867849954 +0000 UTC m=+0.509644866 container remove aeedd8f7c2fb3f9015fd2a429e9ee8700179b616da4d5ee0ef04efa3843a8d21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T08:15:55.883 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 systemd[1]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.9.service: Deactivated successfully. 2024-06-06T08:15:55.883 INFO:journalctl@ceph.osd.9.smithi087.stdout:Jun 06 08:15:55 smithi087 systemd[1]: Stopped Ceph osd.9 for 3f55109c-23d7-11ef-bc9b-c7b262605968. 2024-06-06T08:15:55.885 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-06T08:15:55.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:15:55.924 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-06T08:15:55.924 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-06T08:15:55.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.10 2024-06-06T08:15:55.950 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:55.951 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:55.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:56.214 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:55 smithi181 systemd[1]: Stopping Ceph osd.10 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:15:56.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:56.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:56.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:56.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:56.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:56.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:56 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:56.197+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:15:56.556 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.215+0000 7f3ed5167700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:15:56.556 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.215+0000 7f3ed5167700 -1 osd.10 150 *** Got signal Terminated *** 2024-06-06T08:15:56.556 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.215+0000 7f3ed5167700 -1 osd.10 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:57.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:57.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:56 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:56.587+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:57.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:57.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:57.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:57.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:57.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:57.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:57 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:57.209+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:15:58.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:58.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:58.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:58.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:58.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:58.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:58.057 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:57 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:57.629+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:58.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:58.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:58.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:58.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:58.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:58.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:58 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:58.164+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:15:59.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:15:59.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:58 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:58.672+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:15:59.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:15:59.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:15:59.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:15:59.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:15:59.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:15:59.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:15:59 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:15:59.139+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:00.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:16:00.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:15:59 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:15:59.625+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:16:00.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:00.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:00.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:00.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:00.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:00.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:00.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:00 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:00.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:01.055 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:35.893268+0000 front 2024-06-06T08:14:35.893225+0000 (oldest deadline 2024-06-06T08:15:01.193302+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:42.894317+0000 front 2024-06-06T08:14:42.894258+0000 (oldest deadline 2024-06-06T08:15:14.694953+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.695711+0000 front 2024-06-06T08:14:54.695681+0000 (oldest deadline 2024-06-06T08:15:19.395658+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-06-06T08:14:59.396366+0000 front 2024-06-06T08:14:59.396448+0000 (oldest deadline 2024-06-06T08:15:25.296455+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:12.798772+0000 front 2024-06-06T08:15:12.798749+0000 (oldest deadline 2024-06-06T08:15:38.698945+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.699639+0000 front 2024-06-06T08:15:18.699651+0000 (oldest deadline 2024-06-06T08:15:43.399604+0000) 2024-06-06T08:16:01.056 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:00 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10[61689]: 2024-06-06T08:16:00.639+0000 7f3ed175b700 -1 osd.10 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:27.501077+0000 front 2024-06-06T08:15:27.501016+0000 (oldest deadline 2024-06-06T08:15:49.800890+0000) 2024-06-06T08:16:01.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:01.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:01 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:01.230+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:01.872 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:01 smithi181 podman[79868]: 2024-06-06 08:16:01.599558014 +0000 UTC m=+5.480959441 container died 6b813d6a614d081f42e593e9ba38fd71a6a65d5ecf9da8ee78ac72b9724c4d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-06T08:16:02.259 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:01 smithi181 podman[79868]: 2024-06-06 08:16:01.964517999 +0000 UTC m=+5.845919423 container cleanup 6b813d6a614d081f42e593e9ba38fd71a6a65d5ecf9da8ee78ac72b9724c4d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-06T08:16:02.259 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:01 smithi181 bash[79868]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10 2024-06-06T08:16:02.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:02.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:02.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:02.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:02.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:02.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:02.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:02 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:02.188+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:02.555 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79884]: 2024-06-06 08:16:02.260343251 +0000 UTC m=+0.655319428 container remove 6b813d6a614d081f42e593e9ba38fd71a6a65d5ecf9da8ee78ac72b9724c4d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T08:16:02.871 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79974]: 2024-06-06 08:16:02.569681601 +0000 UTC m=+0.112299851 container create eaff36920f31bfe2adf4c478a74174816111d38173bd1ce12a80e34a76b43cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-06T08:16:02.871 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79974]: 2024-06-06 08:16:02.474313978 +0000 UTC m=+0.016932232 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:16:02.871 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79974]: 2024-06-06 08:16:02.788988961 +0000 UTC m=+0.331607213 container init eaff36920f31bfe2adf4c478a74174816111d38173bd1ce12a80e34a76b43cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:16:02.871 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79974]: 2024-06-06 08:16:02.792281174 +0000 UTC m=+0.334899419 container start eaff36920f31bfe2adf4c478a74174816111d38173bd1ce12a80e34a76b43cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:16:02.871 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:02 smithi181 podman[79974]: 2024-06-06 08:16:02.87186367 +0000 UTC m=+0.414481924 container attach eaff36920f31bfe2adf4c478a74174816111d38173bd1ce12a80e34a76b43cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:16:03.306 INFO:journalctl@ceph.osd.10.smithi181.stdout:Jun 06 08:16:03 smithi181 podman[79974]: 2024-06-06 08:16:03.081652915 +0000 UTC m=+0.624271160 container died eaff36920f31bfe2adf4c478a74174816111d38173bd1ce12a80e34a76b43cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:16:03.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:03.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:03.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:03.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:03.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:03.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:03.511 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:03.220+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:03.627 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-06T08:16:03.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:03.705 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-06T08:16:03.705 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-06T08:16:03.705 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.11 2024-06-06T08:16:04.009 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:03 smithi195 systemd[1]: Stopping Ceph osd.11 for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:04.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.018+0000 7fce6a842700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.018+0000 7fce6a842700 -1 osd.11 150 *** Got signal Terminated *** 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.018+0000 7fce6a842700 -1 osd.11 150 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:04.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:04.511 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:04.511 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:04.511 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:04 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:04.210+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:05.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:05.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:05 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:05.168+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:06.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:06.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:06 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:06.195+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-06-06T08:15:41.597397+0000 front 2024-06-06T08:15:41.597414+0000 (oldest deadline 2024-06-06T08:16:05.697421+0000) 2024-06-06T08:16:07.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:07.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:07.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:07 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:07.156+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-06-06T08:15:41.597397+0000 front 2024-06-06T08:15:41.597414+0000 (oldest deadline 2024-06-06T08:16:05.697421+0000) 2024-06-06T08:16:08.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-06-06T08:14:38.786021+0000 front 2024-06-06T08:14:38.785927+0000 (oldest deadline 2024-06-06T08:15:01.085990+0000) 2024-06-06T08:16:08.509 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-06-06T08:14:44.587371+0000 front 2024-06-06T08:14:44.587214+0000 (oldest deadline 2024-06-06T08:15:09.887193+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-06-06T08:14:54.488890+0000 front 2024-06-06T08:14:54.488923+0000 (oldest deadline 2024-06-06T08:15:17.989012+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-06-06T08:15:07.791958+0000 front 2024-06-06T08:15:07.791933+0000 (oldest deadline 2024-06-06T08:15:33.691849+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-06-06T08:15:18.695059+0000 front 2024-06-06T08:15:18.694976+0000 (oldest deadline 2024-06-06T08:15:43.395162+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-06-06T08:15:28.696089+0000 front 2024-06-06T08:15:28.696188+0000 (oldest deadline 2024-06-06T08:15:53.996041+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-06-06T08:15:33.996461+0000 front 2024-06-06T08:15:33.996449+0000 (oldest deadline 2024-06-06T08:15:59.296470+0000) 2024-06-06T08:16:08.510 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:08 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11[62637]: 2024-06-06T08:16:08.114+0000 7fce62e2e700 -1 osd.11 150 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-06-06T08:15:41.597397+0000 front 2024-06-06T08:15:41.597414+0000 (oldest deadline 2024-06-06T08:16:05.697421+0000) 2024-06-06T08:16:09.802 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:09 smithi195 podman[82789]: 2024-06-06 08:16:09.522669558 +0000 UTC m=+5.614250669 container stop 2feb2349b4caba083e453003d6947369401f299883b1d6d93cb3fbb747f94f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-06T08:16:09.802 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:09 smithi195 podman[82789]: 2024-06-06 08:16:09.548236045 +0000 UTC m=+5.639817165 container died 2feb2349b4caba083e453003d6947369401f299883b1d6d93cb3fbb747f94f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T08:16:10.085 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:09 smithi195 podman[82789]: 2024-06-06 08:16:09.893435819 +0000 UTC m=+5.985016923 container cleanup 2feb2349b4caba083e453003d6947369401f299883b1d6d93cb3fbb747f94f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T08:16:10.085 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:09 smithi195 bash[82789]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11 2024-06-06T08:16:10.506 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82807]: 2024-06-06 08:16:10.086481459 +0000 UTC m=+0.557406650 container remove 2feb2349b4caba083e453003d6947369401f299883b1d6d93cb3fbb747f94f48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T08:16:10.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82899]: 2024-06-06 08:16:10.410197 +0000 UTC m=+0.016553435 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:16:10.759 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82899]: 2024-06-06 08:16:10.545533025 +0000 UTC m=+0.151889457 container create 222eedd773354f1600a3d8a69bbd8c373bc60664e688c835b3c493362529e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-06T08:16:10.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82899]: 2024-06-06 08:16:10.731344857 +0000 UTC m=+0.337701288 container init 222eedd773354f1600a3d8a69bbd8c373bc60664e688c835b3c493362529e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T08:16:10.760 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82899]: 2024-06-06 08:16:10.734383795 +0000 UTC m=+0.340740227 container start 222eedd773354f1600a3d8a69bbd8c373bc60664e688c835b3c493362529e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:16:11.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:10 smithi195 podman[82899]: 2024-06-06 08:16:10.814806031 +0000 UTC m=+0.421162485 container attach 222eedd773354f1600a3d8a69bbd8c373bc60664e688c835b3c493362529e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-06T08:16:11.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:Jun 06 08:16:11 smithi195 podman[82899]: 2024-06-06 08:16:11.022241956 +0000 UTC m=+0.628598389 container died 222eedd773354f1600a3d8a69bbd8c373bc60664e688c835b3c493362529e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:16:11.633 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-06T08:16:11.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:11.671 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-06T08:16:11.671 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-06T08:16:11.671 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.a 2024-06-06T08:16:12.019 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:11 smithi087 systemd[1]: Stopping Ceph mds.a for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:12.359 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:12 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a[71521]: 2024-06-06T08:16:12.019+0000 7f3ba3a99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:12.359 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:12 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a[71521]: 2024-06-06T08:16:12.019+0000 7f3ba3a99700 -1 mds.a *** got signal Terminated *** 2024-06-06T08:16:15.109 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:14 smithi087 podman[127026]: 2024-06-06 08:16:14.820745055 +0000 UTC m=+2.952368149 container died 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-06T08:16:15.536 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:15 smithi087 podman[127026]: 2024-06-06 08:16:15.319682412 +0000 UTC m=+3.451305550 container cleanup 6563e6d28f6561128959282f4e2f124ebdb8e91f9492f2b22fadbbdb76edc0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T08:16:15.536 INFO:journalctl@ceph.mds.a.smithi087.stdout:Jun 06 08:16:15 smithi087 bash[127026]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-a 2024-06-06T08:16:15.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-06T08:16:15.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:15.705 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-06T08:16:15.706 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-06T08:16:15.706 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.d 2024-06-06T08:16:16.109 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:15 smithi087 systemd[1]: Stopping Ceph mds.d for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:16.109 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d[70253]: 2024-06-06T08:16:16.032+0000 7f0faadcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:16.109 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:16 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d[70253]: 2024-06-06T08:16:16.032+0000 7f0faadcf700 -1 mds.d *** got signal Terminated *** 2024-06-06T08:16:17.781 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:17 smithi087 podman[127147]: 2024-06-06 08:16:17.392035595 +0000 UTC m=+1.460904182 container died 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-06T08:16:17.781 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:17 smithi087 podman[127147]: 2024-06-06 08:16:17.782047375 +0000 UTC m=+1.850915964 container cleanup 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-06T08:16:18.109 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:17 smithi087 bash[127147]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d 2024-06-06T08:16:18.109 INFO:journalctl@ceph.mds.d.smithi087.stdout:Jun 06 08:16:18 smithi087 podman[127159]: 2024-06-06 08:16:18.025378588 +0000 UTC m=+0.630297554 container remove 16f81635e3bddec9fd2a71ef72514849661e9dab28bf1651b04f312f765ac4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-06T08:16:18.162 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-06T08:16:18.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:18.197 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-06T08:16:18.198 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-06T08:16:18.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.g 2024-06-06T08:16:18.510 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:18 smithi087 systemd[1]: Stopping Ceph mds.g for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:18.859 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-g[71161]: 2024-06-06T08:16:18.511+0000 7eff3be84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:18.859 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:18 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-g[71161]: 2024-06-06T08:16:18.511+0000 7eff3be84700 -1 mds.g *** got signal Terminated *** 2024-06-06T08:16:20.109 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:19 smithi087 podman[127269]: 2024-06-06 08:16:19.626736601 +0000 UTC m=+1.215826623 container died 7c814bdfef9df44ef9f84f0f4b02b91971c943216f211673a0b94bc0dbc8b0d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T08:16:20.528 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:20 smithi087 podman[127269]: 2024-06-06 08:16:20.295195851 +0000 UTC m=+1.884285918 container cleanup 7c814bdfef9df44ef9f84f0f4b02b91971c943216f211673a0b94bc0dbc8b0d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T08:16:20.528 INFO:journalctl@ceph.mds.g.smithi087.stdout:Jun 06 08:16:20 smithi087 bash[127269]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-g 2024-06-06T08:16:20.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-06T08:16:20.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:20.716 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-06T08:16:20.716 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-06T08:16:20.716 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.j 2024-06-06T08:16:21.056 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:20 smithi087 systemd[1]: Stopping Ceph mds.j for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:21.359 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j[69794]: 2024-06-06T08:16:21.056+0000 7fae66d50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:21.359 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:21 smithi087 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j[69794]: 2024-06-06T08:16:21.056+0000 7fae66d50700 -1 mds.j *** got signal Terminated *** 2024-06-06T08:16:24.834 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:24 smithi087 podman[127391]: 2024-06-06 08:16:24.550069392 +0000 UTC m=+3.615566779 container died 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T08:16:25.109 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:25 smithi087 podman[127391]: 2024-06-06 08:16:25.024206967 +0000 UTC m=+4.089704343 container cleanup 2d054ce0c32f5295125f5b924c1705918ffdec6d0be64765eca1ecc7bc4427ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-06T08:16:25.109 INFO:journalctl@ceph.mds.j.smithi087.stdout:Jun 06 08:16:25 smithi087 bash[127391]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-j 2024-06-06T08:16:25.447 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-06T08:16:25.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:25.479 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-06T08:16:25.479 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-06T08:16:25.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.b 2024-06-06T08:16:25.765 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:25 smithi181 systemd[1]: Stopping Ceph mds.b for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:26.056 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: 2024-06-06T08:16:25.766+0000 7f950aa33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:26.056 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:25 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b[67349]: 2024-06-06T08:16:25.766+0000 7f950aa33700 -1 mds.b *** got signal Terminated *** 2024-06-06T08:16:30.040 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:29 smithi181 podman[80091]: 2024-06-06 08:16:29.779599974 +0000 UTC m=+4.104708264 container died b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-06T08:16:30.305 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:30 smithi181 podman[80091]: 2024-06-06 08:16:30.14044794 +0000 UTC m=+4.465556227 container cleanup b6416607acde57532339c0c2e94d7b63cffbc8dec03ca4bdfa27c41c62661f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:16:30.306 INFO:journalctl@ceph.mds.b.smithi181.stdout:Jun 06 08:16:30 smithi181 bash[80091]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-b 2024-06-06T08:16:30.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-06T08:16:30.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:30.776 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-06T08:16:30.776 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-06T08:16:30.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.e 2024-06-06T08:16:31.306 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:30 smithi181 systemd[1]: Stopping Ceph mds.e for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:31.306 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: 2024-06-06T08:16:31.092+0000 7f76252d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:31.306 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:31 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e[66729]: 2024-06-06T08:16:31.092+0000 7f76252d5700 -1 mds.e *** got signal Terminated *** 2024-06-06T08:16:34.702 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:34 smithi181 podman[80214]: 2024-06-06 08:16:34.424062765 +0000 UTC m=+3.429551044 container died 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T08:16:35.044 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:34 smithi181 podman[80214]: 2024-06-06 08:16:34.802322293 +0000 UTC m=+3.807810571 container cleanup 7c6028ecd49f2dbef845548edb9e38840331c7c274d874e205385da9b6ea0cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-06T08:16:35.044 INFO:journalctl@ceph.mds.e.smithi181.stdout:Jun 06 08:16:34 smithi181 bash[80214]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-e 2024-06-06T08:16:35.163 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-06T08:16:35.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:35.195 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-06T08:16:35.195 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-06T08:16:35.195 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.h 2024-06-06T08:16:35.305 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:35 smithi181 systemd[1]: Stopping Ceph mds.h for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:35.805 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T08:16:35.512+0000 7f08bb705700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:35.805 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:35 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h[67045]: 2024-06-06T08:16:35.512+0000 7f08bb705700 -1 mds.h *** got signal Terminated *** 2024-06-06T08:16:38.715 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:38 smithi181 podman[80335]: 2024-06-06 08:16:38.42881303 +0000 UTC m=+3.014624664 container died f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:16:39.055 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:38 smithi181 podman[80335]: 2024-06-06 08:16:38.797661427 +0000 UTC m=+3.383473050 container cleanup f1dd816f140b9fe1fef598010fb632b8c244be4cd24c2a79f02b2d01598855bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T08:16:39.056 INFO:journalctl@ceph.mds.h.smithi181.stdout:Jun 06 08:16:38 smithi181 bash[80335]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-h 2024-06-06T08:16:39.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-06T08:16:39.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:39.264 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-06T08:16:39.264 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-06T08:16:39.265 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.k 2024-06-06T08:16:39.555 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:39 smithi181 systemd[1]: Stopping Ceph mds.k for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:40.055 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k[66414]: 2024-06-06T08:16:39.585+0000 7f03de779700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:40.056 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:39 smithi181 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k[66414]: 2024-06-06T08:16:39.585+0000 7f03de779700 -1 mds.k *** got signal Terminated *** 2024-06-06T08:16:43.843 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:43 smithi181 podman[80470]: 2024-06-06 08:16:43.592958093 +0000 UTC m=+4.112581902 container died 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:16:44.134 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:43 smithi181 podman[80470]: 2024-06-06 08:16:43.917991477 +0000 UTC m=+4.437615274 container cleanup 57008d0756ab4e3a6c90e92440ba2a35d522ecca741ea93ca6427350d23425fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T08:16:44.135 INFO:journalctl@ceph.mds.k.smithi181.stdout:Jun 06 08:16:43 smithi181 bash[80470]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-k 2024-06-06T08:16:44.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-06T08:16:44.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:44.379 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-06T08:16:44.380 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-06T08:16:44.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.c 2024-06-06T08:16:44.717 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:44 smithi195 systemd[1]: Stopping Ceph mds.c for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:44.974 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: 2024-06-06T08:16:44.717+0000 7f75b1263700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:44.974 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:44 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c[67975]: 2024-06-06T08:16:44.717+0000 7f75b1263700 -1 mds.c *** got signal Terminated *** 2024-06-06T08:16:46.709 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:46 smithi195 podman[83003]: 2024-06-06 08:16:46.448664769 +0000 UTC m=+1.869664586 container died 1291b2f081478e0840241dce59f82c0e39ecc307112badb4a76603bdbc759d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:16:47.009 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:46 smithi195 podman[83003]: 2024-06-06 08:16:46.80119493 +0000 UTC m=+2.222194736 container cleanup 1291b2f081478e0840241dce59f82c0e39ecc307112badb4a76603bdbc759d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T08:16:47.010 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:46 smithi195 bash[83003]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c 2024-06-06T08:16:47.363 INFO:journalctl@ceph.mds.c.smithi195.stdout:Jun 06 08:16:47 smithi195 podman[83027]: 2024-06-06 08:16:47.094131795 +0000 UTC m=+0.636007165 container remove 1291b2f081478e0840241dce59f82c0e39ecc307112badb4a76603bdbc759d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T08:16:47.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-06T08:16:47.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:47.415 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-06T08:16:47.415 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-06T08:16:47.415 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.f 2024-06-06T08:16:47.745 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:47 smithi195 systemd[1]: Stopping Ceph mds.f for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:48.009 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: 2024-06-06T08:16:47.745+0000 7f1c18921700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:48.009 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:47 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f[67662]: 2024-06-06T08:16:47.745+0000 7f1c18921700 -1 mds.f *** got signal Terminated *** 2024-06-06T08:16:50.697 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:50 smithi195 podman[83138]: 2024-06-06 08:16:50.430710326 +0000 UTC m=+2.786074324 container died e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-06T08:16:51.009 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:50 smithi195 podman[83138]: 2024-06-06 08:16:50.787906601 +0000 UTC m=+3.143270598 container cleanup e585d789d9b2ddb3c291ecda0616716cfb7b124a10315d0a446e1fc7e098c5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-06T08:16:51.009 INFO:journalctl@ceph.mds.f.smithi195.stdout:Jun 06 08:16:50 smithi195 bash[83138]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-f 2024-06-06T08:16:51.282 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-06T08:16:51.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:51.313 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-06T08:16:51.314 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-06T08:16:51.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.i 2024-06-06T08:16:51.632 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:51 smithi195 systemd[1]: Stopping Ceph mds.i for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:52.009 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: 2024-06-06T08:16:51.633+0000 7f1a5529f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:52.009 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:51 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i[68275]: 2024-06-06T08:16:51.633+0000 7f1a5529f700 -1 mds.i *** got signal Terminated *** 2024-06-06T08:16:54.759 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:54 smithi195 podman[83259]: 2024-06-06 08:16:54.427292959 +0000 UTC m=+2.890879055 container died c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T08:16:55.042 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:54 smithi195 podman[83259]: 2024-06-06 08:16:54.842294398 +0000 UTC m=+3.305880492 container cleanup c4bbaed80256f2ab518cb4dabfbcf46b45f4d74c211cd66cbdaa6a69d3b202cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T08:16:55.042 INFO:journalctl@ceph.mds.i.smithi195.stdout:Jun 06 08:16:54 smithi195 bash[83259]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-i 2024-06-06T08:16:55.170 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-06T08:16:55.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:55.202 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-06T08:16:55.202 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-06T08:16:55.203 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.l 2024-06-06T08:16:55.407 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:55 smithi195 systemd[1]: Stopping Ceph mds.l for 3f55109c-23d7-11ef-bc9b-c7b262605968... 2024-06-06T08:16:55.759 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l[67359]: 2024-06-06T08:16:55.520+0000 7f0ba7d38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:16:55.760 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:55 smithi195 ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l[67359]: 2024-06-06T08:16:55.520+0000 7f0ba7d38700 -1 mds.l *** got signal Terminated *** 2024-06-06T08:16:57.053 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:56 smithi195 podman[83378]: 2024-06-06 08:16:56.800625551 +0000 UTC m=+1.379031192 container died 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T08:16:57.420 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:57 smithi195 podman[83378]: 2024-06-06 08:16:57.162797953 +0000 UTC m=+1.741203584 container cleanup 6b718d9adcf4a8fada5801d9410ec757cb6c930dddb1b4d070b372905512a0ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T08:16:57.421 INFO:journalctl@ceph.mds.l.smithi195.stdout:Jun 06 08:16:57 smithi195 bash[83378]: ceph-3f55109c-23d7-11ef-bc9b-c7b262605968-mds-l 2024-06-06T08:16:57.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3f55109c-23d7-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-06T08:16:57.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:16:57.578 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-06T08:16:57.578 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:16:57.769 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:17:00.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:17:00.612 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:17:03.097 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:17:03.286 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:17:05.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:17:05.702 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:17:05.732 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:17:05.758 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-06T08:17:05.760 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi087/crash 2024-06-06T08:17:05.761 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:17:06.007 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:17:06.007 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:17:06.009 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi181/crash 2024-06-06T08:17:06.010 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:17:06.080 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:17:06.080 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:17:06.082 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi195/crash 2024-06-06T08:17:06.083 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:17:06.116 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:17:06.117 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:17:06.117 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-06T08:17:06.118 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-06T08:17:06.169 INFO:teuthology.orchestra.run.smithi087.stdout:2024-06-06T07:50:01.737424+0000 mds.h (mds.2) 2 : cluster [WRN] Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/2cbbdaee-7f6a-426d-a0fe-6de3fe3b0542/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-06-06T08:17:06.171 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-06T08:17:06.171 INFO:tasks.cephadm:Compressing logs... 2024-06-06T08:17:06.171 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:17:06.174 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:17:06.176 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:17:06.198 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:17:06.198 INFO:teuthology.orchestra.run.smithi087.stderr:: No such file or directory 2024-06-06T08:17:06.198 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-06T08:17:06.203 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.a.log: 84.8% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log.gzgzip 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr: -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-06T08:17:06.204 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log: 87.0% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T08:17:06.205 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:17:06.205 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-06T08:17:06.206 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.6% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-06T08:17:06.206 INFO:teuthology.orchestra.run.smithi087.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-06T08:17:06.209 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:: No such file or directory 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-06T08:17:06.210 INFO:teuthology.orchestra.run.smithi195.stderr:gzip/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-06T08:17:06.211 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:17:06.212 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.11.log: /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-06T08:17:06.212 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:17:06.217 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-06T08:17:06.217 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-06T08:17:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-06T08:17:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-06T08:17:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-06T08:17:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:17:06.228 INFO:teuthology.orchestra.run.smithi087.stderr: 90.3% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:17:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-06T08:17:06.350 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 84.6% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T08:17:06.351 INFO:teuthology.orchestra.run.smithi195.stderr: 87.0% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T08:17:06.370 INFO:teuthology.orchestra.run.smithi087.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:17:06.384 INFO:teuthology.orchestra.run.smithi195.stderr: 94.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-06T08:17:06.386 INFO:teuthology.orchestra.run.smithi087.stderr: 94.6% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-06T08:17:06.419 INFO:teuthology.orchestra.run.smithi195.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:17:06.420 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:17:06.439 INFO:teuthology.orchestra.run.smithi087.stderr: 94.6% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-06T08:17:06.439 INFO:teuthology.orchestra.run.smithi181.stderr: 92.7% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-06T08:17:06.440 INFO:teuthology.orchestra.run.smithi087.stderr: 94.6% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-06T08:17:06.447 INFO:teuthology.orchestra.run.smithi195.stderr: 91.1% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:17:06.513 INFO:teuthology.orchestra.run.smithi195.stderr: 90.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:17:06.544 INFO:teuthology.orchestra.run.smithi195.stderr: 92.8% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-06T08:17:06.700 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:17:06.834 INFO:teuthology.orchestra.run.smithi087.stderr: 90.3% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-06T08:17:07.634 INFO:teuthology.orchestra.run.smithi181.stderr: 84.7% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T08:17:07.635 INFO:teuthology.orchestra.run.smithi181.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:17:07.677 INFO:teuthology.orchestra.run.smithi181.stderr: 90.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:17:07.820 INFO:teuthology.orchestra.run.smithi087.stderr: 90.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-06T08:17:07.980 INFO:teuthology.orchestra.run.smithi181.stderr: 87.0% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T08:17:08.868 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-06T08:17:15.098 INFO:teuthology.orchestra.run.smithi195.stderr: 95.0% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-06T08:17:48.934 INFO:teuthology.orchestra.run.smithi087.stderr: 91.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-06T08:17:58.991 INFO:teuthology.orchestra.run.smithi087.stderr: 91.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-06T08:18:01.089 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-06T08:18:05.948 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-06T08:18:05.950 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-06-06T08:18:05.950 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m59.763s 2024-06-06T08:18:05.950 INFO:teuthology.orchestra.run.smithi087.stderr:user 3m21.007s 2024-06-06T08:18:05.951 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m6.392s 2024-06-06T08:18:06.537 INFO:teuthology.orchestra.run.smithi181.stderr: 93.2% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-06T08:18:39.812 INFO:teuthology.orchestra.run.smithi195.stderr: 91.3% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-06T08:18:42.111 INFO:teuthology.orchestra.run.smithi195.stderr: 94.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-06T08:19:30.714 INFO:teuthology.orchestra.run.smithi195.stderr: 91.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-06T08:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr: 95.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-06T08:20:20.164 INFO:teuthology.orchestra.run.smithi195.stderr: 91.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-06T08:20:49.371 INFO:teuthology.orchestra.run.smithi195.stderr: 91.2% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-06T08:20:54.386 INFO:teuthology.orchestra.run.smithi195.stderr: 93.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-06T08:20:54.388 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-06-06T08:20:54.389 INFO:teuthology.orchestra.run.smithi195.stderr:real 3m48.198s 2024-06-06T08:20:54.389 INFO:teuthology.orchestra.run.smithi195.stderr:user 4m42.514s 2024-06-06T08:20:54.389 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m10.470s 2024-06-06T08:24:35.186 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-06T08:25:08.811 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-06T08:29:16.939 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-06T08:30:24.464 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-06T08:30:29.513 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-06T08:36:01.278 INFO:teuthology.orchestra.run.smithi181.stderr: 93.1% -- replaced with /var/log/ceph/3f55109c-23d7-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-06T08:36:01.332 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-06-06T08:36:01.333 INFO:teuthology.orchestra.run.smithi181.stderr:real 18m55.144s 2024-06-06T08:36:01.333 INFO:teuthology.orchestra.run.smithi181.stderr:user 9m25.806s 2024-06-06T08:36:01.333 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m25.758s 2024-06-06T08:36:01.334 INFO:tasks.cephadm:Archiving logs... 2024-06-06T08:36:01.335 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi087/log 2024-06-06T08:36:01.336 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:36:36.210 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi181/log 2024-06-06T08:36:36.211 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:38:05.322 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi195/log 2024-06-06T08:38:05.323 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:38:51.208 INFO:tasks.cephadm:Removing cluster... 2024-06-06T08:38:51.209 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force 2024-06-06T08:38:51.401 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:38:52.066 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force 2024-06-06T08:38:53.675 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:38:54.528 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3f55109c-23d7-11ef-bc9b-c7b262605968 --force 2024-06-06T08:38:54.726 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 3f55109c-23d7-11ef-bc9b-c7b262605968 2024-06-06T08:38:55.367 INFO:tasks.cephadm:Removing cephadm ... 2024-06-06T08:38:55.367 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:38:55.387 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:38:55.417 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:38:55.432 INFO:tasks.cephadm:Teardown complete 2024-06-06T08:38:55.432 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-06T08:38:55.449 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-06T08:38:55.450 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:38:55.452 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:38:55.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:38:55.575 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-06T08:38:55.586 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-06T08:38:55.587 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:38:55.589 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:38:55.592 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:38:55.604 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:38:55.605 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:38:55.606 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:38:55.611 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:38:55.611 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-06-06T08:38:55.611 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +74us[ +72us] +/- 69ms 2024-06-06T08:38:55.611 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1927us[-1927us] +/- 64ms 2024-06-06T08:38:55.611 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +2243us[+2240us] +/- 75ms 2024-06-06T08:38:55.612 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:38:55.614 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:38:55.614 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-06-06T08:38:55.615 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +63us[ +54us] +/- 69ms 2024-06-06T08:38:55.615 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1923us[-1923us] +/- 64ms 2024-06-06T08:38:55.615 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +2244us[+2244us] +/- 75ms 2024-06-06T08:38:55.615 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:38:55.678 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:38:55.678 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-06-06T08:38:55.678 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 +67us[ +68us] +/- 70ms 2024-06-06T08:38:55.678 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -1954us[-1949us] +/- 64ms 2024-06-06T08:38:55.678 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +2216us[+2221us] +/- 75ms 2024-06-06T08:38:55.679 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:38:55.688 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-06T08:38:55.699 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-06T08:38:55.700 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-06T08:38:55.710 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:38:55.923 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:38:56.053 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:38:56.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:38:56.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:56.188 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:38:56.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:56.217 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:38:56.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:56.244 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-06T08:38:56.255 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-06T08:38:56.266 INFO:teuthology.task.internal:Duration was 4222.908936 seconds 2024-06-06T08:38:56.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-06T08:38:56.277 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-06T08:38:56.277 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:38:56.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:38:56.282 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:38:56.330 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:38:56.333 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:38:56.355 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:38:56.652 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-06T08:38:56.652 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-06-06T08:38:56.652 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:38:56.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-06-06T08:38:56.675 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:38:56.702 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-06-06T08:38:56.703 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:38:56.731 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-06T08:38:56.731 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:38:56.733 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:38:56.745 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:38:56.802 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-06T08:38:56.802 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:38:56.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:38:56.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:38:57.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-06T08:38:57.872 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-06T08:38:57.872 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:38:57.897 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:38:57.931 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:38:57.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-06T08:38:57.973 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:57.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:57.977 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:57.996 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-06-06T08:38:58.020 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-06-06T08:38:58.025 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-06-06T08:38:58.049 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:58.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:58.065 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:58.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:58.079 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:38:58.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:38:58.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-06T08:38:58.108 INFO:teuthology.task.internal:Transferring archived files... 2024-06-06T08:38:58.110 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi087 2024-06-06T08:38:58.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:38:58.173 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi181 2024-06-06T08:38:58.174 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:38:58.209 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743210/remote/smithi195 2024-06-06T08:38:58.209 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:38:58.246 INFO:teuthology.task.internal:Removing archive directory... 2024-06-06T08:38:58.246 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:38:58.249 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:38:58.251 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:38:58.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-06T08:38:58.308 INFO:teuthology.task.internal:Not uploading archives. 2024-06-06T08:38:58.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-06T08:38:58.322 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-06T08:38:58.322 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:38:58.324 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:38:58.326 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:38:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: 265818 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:38 /home/ubuntu/cephtest 2024-06-06T08:38:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:38 /home/ubuntu/cephtest 2024-06-06T08:38:58.351 INFO:teuthology.orchestra.run.smithi195.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:38 /home/ubuntu/cephtest 2024-06-06T08:38:58.352 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-06T08:38:58.364 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-06T08:38:58.393 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4222.908935546875 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi087 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bc91305e1d1a4c4ba7d6c9baacf9bed2 status: fail success: false 2024-06-06T08:38:58.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T08:38:58.476 INFO:teuthology.run:FAIL